A method for dynamically allocating computing resource of LoRA fine-tuning

Abstract

The present application relates to the field of resource optimization, and more particularly to a LoRA fine-tuning computing resource dynamic allocation method, by obtaining the video memory state and computing unit load data stream of the computing cluster, the prior features representing the resource fragmentation degree and load level are obtained according to the historical time series data analysis, and the bottleneck label is set for various computing tasks accordingly; during task execution, differentiated scheduling optimization is carried out according to the label type and real-time resource features, the video memory bottleneck task is temporarily withheld until it is reduced to the predetermined threshold value before execution by monitoring the real-time video memory state features; the computing bottleneck task is started when the load meets the conditions by monitoring the real-time load state features. The present application realizes the perception response of the dynamic resource state of the cluster and the targeted scheduling of the task bottleneck type, alleviates the congestion and idling caused by video memory fragmentation and uneven resource distribution, and improves the resource utilization balance and the overall execution efficiency of the LoRA fine-tuning task.

Description

Technical Field

[0001] This invention relates to the field of resource optimization, and in particular to a method for dynamic allocation of computing resources with LoRA fine-tuning. Background Technology

[0002] As the scale of deep learning models continues to expand, parameter-efficient fine-tuning techniques, represented by LoRA (Low-Rank Adaptation), have been widely applied in large computing clusters to execute massive model adaptation tasks. In such distributed training scenarios, dynamic scheduling of computing tasks and efficient allocation of cluster resources are crucial to ensuring overall throughput and hardware utilization. Currently, facing significant differences in memory requirements, computational intensity, and multi-GPU communication overhead among different LoRA fine-tuning tasks, as well as dynamic issues such as memory fragmentation and uneven load distribution between nodes caused by task execution, common cluster task schedulers often adopt static allocation strategies based on first-come, first-served, fixed priority, or simple resource reservation. These methods typically perform one-time scheduling based solely on the pre-set resource request amount of the task, lacking continuous awareness and dynamic response capabilities regarding the real-time resource status of the cluster. Therefore, how to enable resource allocation strategies to dynamically adapt to changes in the internal resource status of the cluster and to perform differentiated scheduling for different task bottleneck types has become a key issue in improving the execution efficiency of LoRA fine-tuning tasks and the utilization efficiency of cluster resources.

[0003] Chinese Patent Publication No. CN117785482A discloses a computing power scheduling system and method for a computing power network, comprising: performing predictive analysis on data requests from a target data source within a preset future time period to obtain a corresponding trend prediction index; receiving data requests from the target data source and sending the data requests to a computing power resource scheduling module; performing computing power scheduling allocation calculations based on the trend prediction index, the data requests, and the specific circumstances of the computing power network module; and the computing power network module executing computing power allocation and calculation based on the results of the computing power scheduling allocation calculations. This method performs predictive analysis on data requests for a preset future time period based on historical data request patterns, incorporates the obtained trend prediction index into the computing power allocation process, and considers the influence of the computing power network equipment itself and the specific category of the data request during computing power allocation, thereby significantly improving the efficiency and capability of computing power resource mobilization.

[0004] However, the following problems still exist in the existing technology:

[0005] 1. In the existing technology, when scheduling LoRA fine-tuning tasks, the allocation is usually made only once based on the static resource requirements declared by the task, without continuously sensing the changes in the video memory status caused by the dynamic release of tasks and the accumulation of fragmentation during the operation of the cluster. As a result, a large amount of video memory resources cannot be effectively utilized due to fragmentation. Video memory intensive tasks often block for a long time while waiting for large blocks of continuous video memory, which reduces the overall task throughput of the cluster.

[0006] 2. Existing technologies lack mechanisms for identifying and differentiating resource bottleneck types for computing tasks. Current scheduling methods often apply the same resource checking and execution strategies to all tasks, failing to distinguish between memory-sensitive and computationally-sensitive tasks. Therefore, when resources are scarce, they cannot prioritize the effective supply of bottleneck resources, resulting in an imbalance between computing and memory resource utilization, with some resources idle while others are overloaded.

[0007] 3. Existing technologies do not fully consider the impact of resource distribution topology characteristics on task execution efficiency in multi-node environments. For example, they do not incorporate cross-node resource status factors such as inter-node memory access latency and communication bandwidth usage into scheduling decision factors. This may result in tasks being scheduled to a combination of nodes with sufficient local memory but high communication overhead, increasing task completion time and affecting the execution efficiency of multi-GPU parallel tasks. Summary of the Invention

[0008] To address this, the present invention provides a LoRA-based dynamic allocation method for computing resources, which overcomes the problems of uneven resource utilization and low task execution efficiency caused by the failure to consider dynamic memory status, differences in task bottlenecks, and resource distribution and access efficiency among multiple nodes in the prior art.

[0009] To achieve the above objectives, this invention provides a method for dynamic allocation of computing resources using LoRA fine-tuning, comprising:

[0010] Acquire the memory status data stream and computing unit load data stream when the computing cluster executes various computing tasks during historical operating cycles;

[0011] Based on the analysis of the memory state data stream, the prior memory state characteristics are analyzed to characterize the fragmentation degree of the memory space of the computing cluster. Based on the analysis of the unit load data stream, the prior load state characteristics are analyzed to characterize the load degree of the computing resources of the computing cluster.

[0012] Bottleneck labels are set for various computing tasks based on prior memory state characteristics and prior load state characteristics.

[0013] Receive and parse the queue of computing tasks to be processed, and extract the resource requirement characteristics of each computing task;

[0014] Resource optimization during task execution is performed based on the prior resource requirements of computational tasks and corresponding bottleneck labels, including:

[0015] The current task is temporarily suspended, and the real-time memory status characteristics in the computing cluster are monitored. The current task is executed after the memory status characteristics decrease to a predetermined threshold.

[0016] Determine whether to execute the current task based on the real-time load status characteristics in the computing cluster;

[0017] The predetermined threshold is determined based on the memory state characteristics corresponding to the computing task.

[0018] Specifically, the video memory status data stream includes,

[0019] Timing data on the number of nodes with a large amount of continuous free video memory and timing data on the inter-node video memory access delay.

[0020] Furthermore, the computing unit load data stream includes,

[0021] Time-series data on GPU core utilization and available core count for each computing node within the computing cluster.

[0022] Furthermore, the step of parsing prior memory state features based on memory state data streams to characterize the fragmentation level of the computing cluster's memory space includes:

[0023] Based on the time-series data of the number of nodes with continuous large capacity of free video memory, determine the average number of free nodes within a preset statistical period.

[0024] Based on the inter-node memory access latency timing data with continuous large capacity of free memory, determine the average access latency within a preset statistical period.

[0025] The ratio of the number of available nodes to the total number of computing nodes in the cluster is determined as the first memory parameter.

[0026] The ratio of the preset latency standard value to the average mutual access latency is determined as the second memory parameter;

[0027] The first memory parameter and the second memory parameter are weighted and summed, and the weighted sum is used as the prior memory state feature.

[0028] Furthermore, the method of characterizing the load level of computing cluster resources by parsing prior load state features based on unit load data streams includes:

[0029] Based on the GPU computing core utilization time-series data, determine the average computing core utilization within a preset statistical period;

[0030] Based on the time-series data of the number of available computing cores, determine the average number of available cores within a preset statistical period;

[0031] The average utilization rate of the computing core is determined as the first load parameter;

[0032] The ratio of the average number of available cores to the total number of computing cores in the cluster is determined as the second load parameter;

[0033] The first load parameter and the second load parameter are weighted and summed, and the weighted summation result is used as the prior load state feature.

[0034] Furthermore, the step of setting bottleneck labels for various computing tasks based on prior memory state characteristics and prior load state characteristics includes:

[0035] Based on historical operating data, determine the relative magnitude relationship between the prior memory state characteristics and the prior load state characteristics when executing various types of computing tasks;

[0036] If, when performing a certain type of computing task, the prior memory state characteristic value is less than the prior load state characteristic value corresponding to the computing task, then a memory bottleneck label is set for the computing task.

[0037] If, when performing a certain type of computing task, the prior load state characteristic value is less than or equal to the prior memory state characteristic value corresponding to that task, then a computing bottleneck label is set for that computing task.

[0038] Furthermore, the resource requirement characteristics corresponding to each computing task are extracted, including:

[0039] The configuration parameters of the computing task are analyzed to obtain the expected memory usage and expected core utilization of the computing task.

[0040] The expected video memory usage and the expected computing core utilization rate are used as the resource requirement characteristics corresponding to the computing task.

[0041] Furthermore, the step of temporarily suspending the current task, monitoring the real-time memory status characteristics in the computing cluster, and waiting for the memory status characteristics to decrease to a predetermined threshold before executing the current task includes:

[0042] If the bottleneck label for the current task is a video memory bottleneck label, then the execution of that task will be temporarily suspended.

[0043] The real-time memory status data stream of the computing cluster is acquired in real time, and the real-time memory status characteristics are analyzed based on the real-time memory status data stream.

[0044] If the real-time video memory status feature is less than or equal to the predetermined unlock threshold, video memory resources are allocated to the current task and the task is started.

[0045] Furthermore, the predetermined unlock threshold is determined based on the expected video memory usage of the current task.

[0046] Furthermore, the step of determining whether to execute the current task based on the real-time load status characteristics in the computing cluster includes:

[0047] In response to the bottleneck label of the current task being a computing bottleneck label, the real-time computing unit load data stream of the computing cluster is obtained in real time, and the real-time load status characteristics are analyzed based on the real-time computing unit load data stream of the computing cluster.

[0048] Based on the mapping relationship between expected computing core utilization and real-time load status characteristics, a predetermined computing threshold is determined.

[0049] If the real-time load status characteristics are less than or equal to the predetermined calculation threshold, then the current task is executed;

[0050] The mapping relationship is preset.

[0051] Compared with existing technologies, this invention acquires the memory status and computing unit load data stream of the computing cluster, analyzes historical time-series data to obtain prior features characterizing the degree of resource fragmentation and load level, and sets bottleneck labels for various computing tasks accordingly. During task execution, differentiated scheduling optimization is performed based on the label type and real-time resource characteristics: by monitoring real-time memory status characteristics, memory bottleneck tasks are temporarily suspended until they decrease to a predetermined threshold before execution; by monitoring real-time load status characteristics, computing bottleneck tasks are started when the load meets the conditions. This invention achieves perception and response to the dynamic resource status of the cluster and targeted scheduling for task bottleneck types, alleviating blocking and idleness caused by memory fragmentation and uneven resource distribution, and improving the balance of resource utilization and the overall execution efficiency of LoRA fine-tuning tasks.

[0052] In particular, this invention evaluates the memory characteristics of a computing cluster based on prior memory status. Prior memory status characteristics are an indicator that comprehensively quantifies the fragmentation of the computing cluster's memory space. This indicator is constructed by statistically analyzing two key time-series data points: "the number of nodes with contiguous large-capacity free memory" and "inter-node memory access latency." The former directly reflects the scale of nodes in the cluster available for large-block memory allocation; a higher value indicates better concentration and availability of memory resources at the node level. The latter characterizes the efficiency of data exchange between preferred nodes; lower latency indicates less communication overhead caused by resource dispersion. Prior memory status characteristics can reflect the fragmentation of the computing cluster's memory space, thereby comprehensively evaluating the system's memory characteristics.

[0053] In particular, this invention considers a mechanism for analyzing prior memory state characteristics and prior load state characteristics based on historical time-series data. In reality, the resource state of a computing cluster changes dynamically over time and is significantly affected by historical task execution patterns. Traditional static scheduling strategies cannot characterize and utilize such historical dependencies, leading to a disconnect between resource assessment and actual conditions. This invention obtains prior characteristics by analyzing memory state data streams and computing unit load data streams, providing a benchmark for characterizing the resource status of the computing cluster for subsequent task scheduling.

[0054] In particular, this invention considers a mechanism for setting bottleneck labels for computational tasks and implementing differentiated scheduling optimization based on the label type. In practice, different types of LoRA fine-tuning tasks have fundamentally different sensitivities to memory and computational resources, and using a uniform scheduling strategy can easily lead to resource mismatch. This invention sets memory bottleneck or computational bottleneck labels for tasks based on the relative size relationship of prior features, and triggers different resource monitoring and execution condition judgment mechanisms according to the label type, achieving targeted adaptation of the scheduling strategy to the inherent bottleneck characteristics of the task.

[0055] In particular, this invention provides a dynamic scheduling mechanism for memory bottleneck tasks. In practical applications, the degree of memory fragmentation changes dynamically with task execution and release. If memory-intensive tasks are scheduled when fragmentation is severe, task failure or prolonged blocking is likely due to insufficient allocation of contiguous memory. To address this, this invention monitors the memory status characteristics of the cluster in real time and adopts a temporary suspension strategy for tasks marked as memory bottlenecks. Resources are only allocated and execution is initiated when the real-time memory status characteristic value drops to a predetermined unlock threshold determined by the expected memory usage of the task. This mechanism effectively avoids invalid scheduling attempts when memory conditions are not met, thereby improving the first-time execution success rate of memory bottleneck tasks and reducing invalid waiting time in the task queue.

[0056] In particular, this invention considers a mechanism for determining the execution timing of computationally bottleneck tasks based on real-time load status characteristics. In reality, the load of computing clusters fluctuates. Deploying computationally intensive tasks under high load may exacerbate competition and prolong execution time, while under low load may lead to idle computing resources. This invention achieves a balance between computing resource utilization efficiency and task execution efficiency by monitoring real-time load status characteristics and comparing them with conditions set according to task requirements to determine the timing of task initiation. Attached Figure Description

[0057] Figure 1 This is a schematic diagram illustrating the steps of the LoRA fine-tuning dynamic allocation method for computing resources according to an embodiment of the present invention;

[0058] Figure 2A logic block diagram illustrating the setting of bottleneck labels for various computing tasks in an embodiment of the invention;

[0059] Figure 3 This is a block diagram of the execution judgment logic for the memory bottleneck tagging task in an embodiment of the invention. Detailed Implementation

[0060] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0061] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0062] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0063] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0064] Please see Figure 1 The diagram illustrates the steps of the LoRA fine-tuning dynamic allocation method for computing resources according to an embodiment of the present invention. The LoRA fine-tuning dynamic allocation method for computing resources according to the present invention includes:

[0065] Step S1: Obtain the memory status data stream and computing unit load data stream when executing various computing tasks during the historical running cycle of the computing cluster;

[0066] Step S2: Based on the memory state data stream, analyze the prior memory state characteristics to characterize the fragmentation degree of the memory space of the computing cluster; based on the unit load data stream, analyze the prior load state characteristics to characterize the load degree of the computing resources of the computing cluster.

[0067] Step S3: Set bottleneck labels for various computing tasks based on prior memory state characteristics and prior load state characteristics;

[0068] Step S4: Receive and parse the queue of computing tasks to be processed, and extract the resource requirement features corresponding to each computing task;

[0069] Step S5 involves optimizing resource usage during the task queue execution process based on the prior resource requirements of the computation task and the corresponding bottleneck labels. This includes:

[0070] The current task is temporarily suspended, and the real-time memory status characteristics in the computing cluster are monitored. The current task is executed after the memory status characteristics decrease to a predetermined threshold.

[0071] Determine whether to execute the current task based on the real-time load status characteristics in the computing cluster;

[0072] The predetermined threshold is determined based on the memory state characteristics corresponding to the computing task.

[0073] Specifically, the video memory status data stream includes,

[0074] Timing data on the number of nodes with a large amount of continuous free video memory and timing data on the inter-node video memory access delay.

[0075] Specifically, for any node in the computing cluster, its video memory address space is scanned to obtain all free memory blocks; if at least two free memory blocks exist, the node is determined to have "contiguous large capacity of free video memory". The size of the memory block should be determined by those skilled in the art based on the video memory size of the nodes in the computing cluster, and is typically set to 2% to 5% of the total capacity of a single card's video memory. In this embodiment of the invention, the memory block corresponding to a node with 24GB of video memory is set to 1024MB, and the memory block corresponding to a node with 72GB of video memory is set to 2048MB.

[0076] Specifically, the computing unit load data stream includes,

[0077] Time-series data on GPU core utilization and available core count for each computing node within the computing cluster.

[0078] Specifically, the method of parsing prior memory state features based on memory state data streams to characterize the fragmentation level of the computing cluster's memory space includes:

[0079] Based on the time-series data of the number of nodes with continuous large capacity of free video memory, determine the average number of free nodes within a preset statistical period.

[0080] Based on the inter-node memory access latency timing data with continuous large capacity of free memory, determine the average access latency within a preset statistical period.

[0081] The ratio of the number of available nodes to the total number of computing nodes in the cluster is determined as the first memory parameter.

[0082] The ratio of the preset latency standard value to the average mutual access latency is determined as the second memory parameter;

[0083] The first memory parameter and the second memory parameter are weighted and summed, and the weighted sum is used as the prior memory state feature.

[0084] Specifically, the preset latency standard value is set based on the theoretical minimum latency value of the hardware interconnect topology and communication protocol of the computing cluster. Those skilled in the art should determine the theoretical minimum latency value based on the actual operating conditions of the deployed computing cluster; further details are omitted here.

[0085] Understandably, a preset latency standard value is used as a benchmark for evaluating the relative performance of real-time inter-node access latency. When the real-time inter-node access latency is close to or better than this standard value, it indicates that the data access efficiency between currently available nodes is high.

[0086] Specifically, the weighting weight of the first memory parameter is 0.35, and the weighting weight of the second memory parameter is 0.65. This weighting reflects that when evaluating the memory space status, the second memory parameter, which reflects the efficiency of data access between nodes, is given more emphasis. This ensures that the parsed prior memory status features can more accurately characterize the combined bottleneck degree formed by memory resource fragmentation and communication overhead in a multi-GPU environment.

[0087] Specifically, the method of characterizing the load level of computing cluster resources by parsing prior load state characteristics based on unit load data streams includes:

[0088] Based on the GPU computing core utilization time-series data, determine the average computing core utilization within a preset statistical period;

[0089] Based on the time-series data of the number of available computing cores, determine the average number of available cores within a preset statistical period;

[0090] The average utilization rate of the computing core is determined as the first load parameter;

[0091] The ratio of the average number of available cores to the total number of computing cores in the cluster is determined as the second load parameter;

[0092] The first load parameter and the second load parameter are weighted and summed, and the weighted summation result is used as the prior load state feature.

[0093] Specifically, the weighting weight of the first load parameter is 0.7, and the weighting weight of the second load parameter is 0.3. This weighting allocation reflects the use of the actual utilization rate of computing cores as the primary basis for assessing the load level, while also taking into account the supplementary reference of the number of available cores, thereby more accurately representing the real-time load pressure and potential scalability of computing resources.

[0094] Please see Figure 2 As shown, Figure 2 This is a logical block diagram illustrating the process of setting bottleneck labels for various computing tasks according to an embodiment of the invention. The step of setting bottleneck labels for various computing tasks based on prior memory state characteristics and prior load state characteristics includes...

[0095] Based on historical operating data, determine the relative magnitude relationship between the prior memory state characteristics and the prior load state characteristics when executing various types of computing tasks;

[0096] If, when performing a certain type of computing task, the prior memory state characteristic value is less than the prior load state characteristic value corresponding to the computing task, then a memory bottleneck label is set for the computing task.

[0097] If, when performing a certain type of computing task, the prior load state characteristic value is less than or equal to the prior memory state characteristic value corresponding to that task, then a computing bottleneck label is set for that computing task.

[0098] Specifically, extracting the resource requirement features corresponding to each computing task includes,

[0099] The configuration parameters of the computing task are analyzed to obtain the expected memory usage and expected core utilization of the computing task.

[0100] The expected video memory usage and the expected computing core utilization rate are used as the resource requirement characteristics corresponding to the computing task.

[0101] Please see Figure 3 As shown, Figure 3 This is a block diagram illustrating the execution judgment logic for a task tagged as a memory bottleneck according to an embodiment of the invention. The step of temporarily suspending the current task, monitoring the real-time memory status characteristics in the computing cluster, and waiting for the memory status characteristics to decrease to a predetermined threshold before executing the current task includes...

[0102] If the bottleneck label for the current task is a video memory bottleneck label, then the execution of that task will be temporarily suspended.

[0103] The real-time memory status data stream of the computing cluster is acquired in real time, and the real-time memory status characteristics are analyzed based on the real-time memory status data stream.

[0104] If the real-time video memory status feature is less than or equal to the predetermined unlock threshold, then video memory resources are allocated to the current task and the task is started.

[0105] Specifically, the predetermined unlock threshold is determined based on the expected video memory usage of the current task.

[0106] Specifically, the predetermined unlock threshold is set through the following rule: a threshold range is pre-defined, where the upper limit corresponds to the memory requirements of tasks with the minimum expected memory usage, and the lower limit corresponds to the memory requirements of tasks with the maximum expected memory usage. For any current task, the specific predetermined unlock threshold is dynamically determined based on the task's expected memory usage, using linear interpolation between the upper and lower limits. This setting method provides the system with a clear and predictable threshold mapping rule, enabling tasks with different memory requirements to correspond to different levels of strictness in execution conditions, thereby satisfying the resource guarantee requirements of large memory tasks while also considering the overall task scheduling efficiency of the system.

[0107] Specifically, in this embodiment of the invention, the upper limit of the threshold interval is set to 0.85, and the lower limit is set to 0.65. The predetermined unlocking threshold is determined according to the following linear interpolation formula:

[0108] The predetermined unlock threshold = upper limit - [(expected VRAM usage - minimum expected VRAM usage) / (maximum expected VRAM usage - minimum expected VRAM usage)] × (upper limit - lower limit).

[0109] The minimum expected video memory usage and the maximum expected video memory usage are the minimum and maximum values ​​obtained by statistically analyzing the expected video memory usage of all tasks in the computing task queue.

[0110] Specifically, determining whether to execute the current task based on real-time load status characteristics in the computing cluster includes:

[0111] In response to the bottleneck label of the current task being a computing bottleneck label, the real-time computing unit load data stream of the computing cluster is obtained in real time, and the real-time load status characteristics are analyzed based on the real-time computing unit load data stream of the computing cluster.

[0112] Based on the mapping relationship between expected computing core utilization and real-time load status characteristics, a predetermined computing threshold is determined.

[0113] If the real-time load status characteristics are less than or equal to the predetermined calculation threshold, then the current task is executed;

[0114] The mapping relationship is preset.

[0115] Specifically, the mapping relationship is as follows: a threshold range is pre-defined, where the lower limit corresponds to the load state required for a task with zero expected computing core utilization, and the upper limit corresponds to the load state required for a task with maximum expected computing core utilization. For any current task, its specific predetermined computing threshold will be dynamically determined by linear interpolation between the lower and upper limits based on the task's expected computing core utilization.

[0116] Specifically, in embodiments of the present invention, the lower limit of the threshold interval is set to 0.60, and the upper limit is set to 0.90. The predetermined calculation threshold is determined according to the following linear interpolation formula:

[0117] The predetermined calculation threshold = lower limit + (expected computing core utilization / maximum expected computing core utilization) × (upper limit - lower limit).

[0118] The maximum expected computing core occupancy rate is the maximum value obtained by statistically analyzing the expected computing core occupancy rates of all bottleneck tasks in the computing task queue.

[0119] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A method for dynamically allocating computing resources with LoRA fine-tuning, characterized in that, include: Acquire the memory status data stream and computing unit load data stream when the computing cluster executes various computing tasks during historical operating cycles; Based on the analysis of the memory state data stream, the prior memory state characteristics are analyzed to characterize the fragmentation degree of the memory space of the computing cluster. Based on the analysis of the unit load data stream, the prior load state characteristics are analyzed to characterize the load degree of the computing resources of the computing cluster. Bottleneck labels are set for various computing tasks based on prior memory state characteristics and prior load state characteristics, including, Based on historical operating data, determine the relative magnitude relationship between the prior memory state characteristics and the prior load state characteristics when executing various types of computing tasks; If, when performing a certain type of computing task, the prior memory state characteristic value is less than the prior load state characteristic value corresponding to the computing task, then a memory bottleneck label is set for the computing task. If, when performing a certain type of computing task, the prior load state characteristic value is less than or equal to the prior memory state characteristic value corresponding to that task, then a computing bottleneck label is set for that computing task. Receive and parse the queue of computing tasks to be processed, and extract the resource requirement characteristics of each computing task; Resource optimization during task execution is performed based on the prior resource requirements of computational tasks and corresponding bottleneck labels, including: The current task is temporarily suspended, and the real-time memory status characteristics in the computing cluster are monitored. The current task is executed after the memory status characteristics decrease to a predetermined threshold. Determine whether to execute the current task based on the real-time load status characteristics in the computing cluster; The predetermined threshold is determined based on the memory state characteristics corresponding to the computing task; The video memory status data stream includes timing data of the number of nodes with a large amount of continuous free video memory and timing data of the video memory access delay between nodes with a large amount of continuous free video memory. The method of parsing prior memory state features based on memory state data stream to characterize the fragmentation degree of the computing cluster's memory space includes: determining the average number of free nodes within a preset statistical period based on the time-series data of the number of nodes with continuous large capacity of free memory; determining the average inter-memory access latency within a preset statistical period based on the time-series data of the inter-memory access latency between nodes with continuous large capacity of free memory; determining the ratio of the statistical value of the number of free nodes to the total number of computing nodes in the cluster as a first memory parameter; determining the ratio of the preset latency standard value to the average inter-memory access latency as a second memory parameter; performing a weighted summation of the first memory parameter and the second memory parameter; and using the weighted summation result as the prior memory state feature. The step of temporarily suspending the current task, monitoring the real-time memory status characteristics in the computing cluster, and waiting for the memory status characteristics to decrease to a predetermined threshold before executing the current task includes... If the bottleneck label of the current task is a video memory bottleneck label, the execution of the task is temporarily suspended, the real-time video memory status data stream of the computing cluster is obtained in real time, and the real-time video memory status characteristics are analyzed based on the real-time video memory status data stream. If the real-time video memory status feature is less than or equal to the predetermined unlock threshold, video memory resources are allocated to the current task and the task is started; The predetermined unlock threshold is determined based on the expected video memory usage of the current task.

2. The method for dynamic allocation of computing resources for LoRA fine-tuning according to claim 1, characterized in that, The computing unit load data stream includes, Time-series data on GPU core utilization and available core count for each computing node within the computing cluster.

3. The method for dynamic allocation of computing resources for LoRA fine-tuning according to claim 2, characterized in that, The method of parsing prior load state characteristics based on unit load data streams to characterize the load level of computing cluster resources includes: Based on the GPU computing core utilization time-series data, determine the average computing core utilization within a preset statistical period; Based on the time-series data of the number of available computing cores, determine the average number of available cores within a preset statistical period; The average utilization rate of the computing core is determined as the first load parameter; The ratio of the average number of available cores to the total number of computing cores in the cluster is determined as the second load parameter; The first load parameter and the second load parameter are weighted and summed, and the weighted summation result is used as the prior load state feature.

4. The method for dynamic allocation of computing resources for LoRA fine-tuning according to claim 1, characterized in that, Extracting the resource requirement features corresponding to each computing task includes, The configuration parameters of the computing task are analyzed to obtain the expected memory usage and expected core utilization of the computing task. The expected video memory usage and the expected computing core utilization rate are used as the resource requirement characteristics corresponding to the computing task.

5. The method for dynamic allocation of computing resources for LoRA fine-tuning according to claim 1, characterized in that, The step of determining whether to execute the current task based on the real-time load status characteristics in the computing cluster includes... In response to the bottleneck label of the current task being a computing bottleneck label, the real-time computing unit load data stream of the computing cluster is obtained in real time, and the real-time load status characteristics are analyzed based on the real-time computing unit load data stream of the computing cluster. Based on the mapping relationship between expected computing core utilization and real-time load status characteristics, a predetermined computing threshold is determined. If the real-time load status characteristics are less than or equal to the predetermined calculation threshold, then the current task is executed; The mapping relationship is preset.

Images