Resource allocation method, apparatus, device, and storage medium

Abstract

The application belongs to the technical field of computers and specifically relates to a resource allocation method, device, equipment and storage medium, which comprises the following steps: S1. acquiring real-time state data of a target resource occupation task and multiple clusters; S2. calculating a cluster resource utilization rate based on a total resource configuration amount and a resource usage amount, calculating a residual resource proportion of the cluster based on a resource residual amount and a resource demand parameter, and calculating a load sensitivity coefficient of the target resource occupation task based on a task priority, a resource demand parameter, an execution time threshold and a task dependency identifier; S3. screening each cluster according to preset screening conditions corresponding to different task guarantee levels, determining a cluster that meets the preset screening conditions as a candidate cluster, monitoring in real time and triggering resource migration based on a quantitative migration condition, and forming a closed-loop process of "data acquisition-index calculation-screening allocation-preemption coordination-monitoring migration".

Description

Technical Field

[0001] This invention belongs to the field of computer technology, specifically relating to resource allocation methods, devices, equipment, and storage media. Background Technology

[0002] With the large-scale deployment of cloud computing, edge computing, and distributed systems, resource allocation in multi-cluster environments has become a core technology supporting complex businesses such as big data processing, artificial intelligence training, and cloud service deployment. Its core objective is to achieve multi-dimensional optimization of task execution efficiency, system stability, and energy utilization under limited hardware resource constraints. In recent years, the industry has developed a series of resource allocation schemes based on load balancing, priority scheduling, and historical data prediction. These schemes provide technical support for basic resource scheduling. However, with the diversification of task types (compute-intensive / IO-intensive / storage-intensive), the dynamic fluctuations of cluster load, and the increasing complexity of inter-task dependencies, existing technologies have gradually revealed structural defects that make them difficult to adapt to complex scenarios. This leads to problems such as insufficient resource utilization, high task failure rates, and energy waste. Therefore, improved resource allocation methods, devices, equipment, and storage media have been designed. Summary of the Invention

[0003] In view of the above-mentioned shortcomings in the prior art, the present invention provides a resource allocation method, apparatus, device and storage medium to solve the problems in the background art.

[0004] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0005] A resource allocation method includes the following steps;

[0006] S1. Obtain the target resource occupancy task and real-time status data of multiple clusters. The real-time status data includes the resource usage, remaining resource, energy efficiency data and resource fragmentation of each cluster. The target resource occupancy task carries task priority, resource requirement parameters, execution time threshold and task dependency identifier.

[0007] S2. Calculate the cluster resource utilization rate based on the total resource allocation and resource usage; calculate the remaining resource ratio of the cluster based on the remaining resource and resource requirement parameters; calculate the load sensitivity coefficient of the target resource occupation task based on the task priority, resource requirement parameters, execution time threshold and task dependency identifier.

[0008] S3. Based on the value range of the load sensitivity coefficient of the target resource occupation task, the target resource occupation task is divided into different protection levels. Each cluster is screened according to the preset screening conditions corresponding to the different task protection levels, and the clusters that meet the preset screening conditions are determined as candidate clusters.

[0009] S4. Based on the preset balancing strategy, the energy efficiency data of the candidate clusters and the energy consumption nonlinear correction model, determine the target cluster from the candidate clusters and allocate the resources of the target cluster to the target resource occupation task.

[0010] By acquiring real-time data of the target task and the cluster, the load sensitivity coefficient and key cluster indicators are calculated. Based on the coefficient, the task guarantee level is divided and candidate clusters are screened. The target cluster is determined by combining the preset balancing strategy and the energy consumption nonlinear correction model. When there is no candidate cluster, a resource preemption negotiation mechanism based on the task dependency chain topology is triggered. Resource migration is triggered in real time based on quantitative migration conditions, forming a closed-loop process of "data collection - indicator calculation - screening and allocation - preemption coordination - monitoring and migration".

[0011] According to another aspect of the present invention, a resource allocation device is also provided, comprising: a data acquisition module for acquiring real-time status data of a target resource-occupying task and multiple clusters, including total resource allocation, resource usage, remaining resource, energy efficiency data, resource fragmentation, and task-related identifiers; an index calculation module for calculating cluster resource utilization, remaining resource ratio, and load sensitivity coefficient of the target resource-occupying task; a level classification and screening module for classifying task protection levels according to the load sensitivity coefficient, calling corresponding preset screening conditions, and determining candidate clusters; a model training and application module for training an energy consumption nonlinear correction model and calculating the corrected energy efficiency ratio and comprehensive score based on the model; a resource allocation module for determining the target cluster based on the comprehensive score and allocating resources to the target resource-occupying task; a dependency topology construction module for constructing a task resource dependency chain topology to provide dependency relationship data for resource preemption; a preemption negotiation module for implementing a resource preemption negotiation mechanism, including preemptible task screening, release cost calculation, resource coordination, and verification feedback; and a monitoring and migration module for real-time monitoring of the target cluster status and task execution progress, triggering resource migration when migration conditions are met.

[0012] According to another aspect of the present invention, an electronic device is also provided, including a memory and a processor; the memory stores a computer program, and when the processor executes the computer program, it implements the methods of various embodiments of the present invention.

[0013] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the methods of various embodiments of the present invention.

[0014] Compared with the prior art, the present invention has the following beneficial effects:

[0015] 1. By acquiring real-time data of the target task and the cluster, the load sensitivity coefficient and key cluster indicators are calculated. Based on the coefficient, the task guarantee level is divided and candidate clusters are screened. The target cluster is determined by combining the preset balancing strategy and the energy consumption nonlinear correction model. When there is no candidate cluster, a resource preemption negotiation mechanism based on the task dependency chain topology is triggered. Real-time monitoring and resource migration are triggered based on quantitative migration conditions, forming a closed-loop process of "data collection - indicator calculation - screening and allocation - preemption coordination - monitoring and migration".

[0016] 2. Based on the task resource dependency chain topology analysis, the relationship between tasks is analyzed, the dependency strength and release cost are quantified, and the release of resources for tasks with low sensitivity, weak dependency and high execution progress is prioritized to avoid blindly seizing and damaging the core task chain, and to ensure that the impact of the seizure operation on the overall system is minimized. Attached Figure Description

[0017] Figure 1 This is a first flowchart of the resource allocation method of the present invention;

[0018] Figure 2 This is the second flowchart of the resource allocation method of the present invention. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present invention, the technical solution of the present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0020] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual images. They should not be construed as limiting the scope of this application. To better illustrate the embodiments of the present invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual dimensions of the product. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0021] According to an embodiment of the present invention, an embodiment of a resource allocation method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0022] The resource allocation method embodiments provided in this application can be executed in a mobile terminal, computer terminal, or similar computing device. A computer terminal may include one or more processors (processors may include, but are not limited to, microprocessors such as MCUs or programmable logic devices such as FPGAs), and a memory for storing data. In addition, it may include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera.

[0023] It should be noted that the aforementioned one or more processors and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits can be implemented wholly or partially as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuit can be a single, independent processing module, or wholly or partially integrated into any other element in the computer terminal. As involved in the embodiments of this application, the data processing circuit serves as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0024] The memory can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the resource allocation method in this embodiment of the invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the IP address allocation method of the aforementioned application. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to a computer terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0025] The display can be, for example, a touchscreen liquid crystal display (LCD), which allows users to interact with the user interface of the computer terminal.

[0026] refer to Figure 1 A flowchart of a resource allocation method according to an embodiment of the present invention, the method comprising the following steps;

[0027] S1. Obtain real-time status data of the target resource usage task and multiple clusters;

[0028] The real-time status data includes the resource usage, remaining resources, energy efficiency data, and resource fragmentation of each cluster. The target resource occupation task carries the task priority, resource requirement parameters, execution time threshold, and task dependency identifier.

[0029] The core function of the target resource allocation task, which requests the use of resources across multiple clusters, is as the "demand subject" for resource allocation. It must carry task priority, resource requirement parameters, execution duration thresholds, and task dependency identifiers, allowing the system to accurately identify the intensity and priority of the task's resource demand and avoid resource waste caused by "indiscriminate allocation." A cluster is a resource collection formed by multiple nodes (physical servers or virtual machines) with independent computing, storage, and networking capabilities connected through a high-speed network. It integrates dispersed hardware resources to form scalable computing and storage capabilities, supporting concurrent execution of multiple tasks while achieving resource expansion, load balancing, and fault redundancy. It solves the problems of limited single-node resources and low availability, providing an operable resource foundation for differentiated resource allocation, low-impact preemption, and dynamic migration. The cluster needs to provide real-time status data such as total resource configuration, usage, and energy efficiency to adapt to the differentiated needs of multiple tasks and multi-scenario scheduling logic.

[0030] Task priority is an indicator that quantifies the importance of a task in terms of the amount of resources it consumes. Its value ranges from [0,1], with higher values ​​indicating greater task importance. For example, high-priority tasks ultimately correspond to core support levels (load sensitivity coefficient ∈ (0.7,1.0]).

[0031] Resource requirements parameters refer to the types and quantities of hardware resources necessary for the target task to run, with core parameters including the number of processor cores, memory capacity, and storage capacity. These parameters directly determine whether cluster resources can be adapted to the task, avoiding insufficient resource allocation (leading to task crashes) or excessive resource allocation (causing waste), and are explicitly specified when the task is initiated.

[0032] The execution time threshold refers to the maximum allowed execution time for a task, and is a key indicator for measuring the time urgency of a task. It prevents tasks from occupying resources for extended periods, leading to low resource utilization. Furthermore, by quantifying time urgency, it provides supplementary information for prioritizing resources for core tasks (such as short-term critical tasks requiring rapid allocation of high-quality resources). By standardizing the time urgency dimension through the ratio to the system's maximum allowed execution time, it adapts to the different time requirements of different types of tasks.

[0033] Task dependency identifiers are unique markers that indicate the dependency relationships between a target resource-consuming task and other tasks (upstream and downstream). They can resolve resource interactions and data flow relationships between tasks. They are the core input for constructing the "task resource dependency chain topology," used to calculate dependency strength and dependency impact factors. They differentiate the tightness of dependencies between tasks, preventing the disruption of strong dependency chains during resource preemption (e.g., preempting upstream task resources leading to a lack of data input for downstream tasks), and minimizing the impact of preemption on the overall system.

[0034] For computationally intensive tasks, the parameter settings (priority P=0.9, execution time threshold T=24h) are based on the configuration of core business tasks in actual cloud computing (such as real-time data analysis and high-frequency transaction calculation). The dependency coefficient D=0.1 corresponds to "depending on 2 downstream tasks", ensuring that the scenario is consistent with the actual business needs.

[0035] S2. Calculate the cluster resource utilization rate based on the total resource allocation and resource usage; calculate the remaining resource ratio of the cluster based on the remaining resource and resource requirement parameters; calculate the load sensitivity coefficient of the target resource occupation task based on the task priority, resource requirement parameters, execution time threshold and task dependency identifier.

[0036] Cluster resource utilization is the arithmetic mean of processor utilization, memory utilization, and storage utilization. Remaining resource ratio is the minimum of the following three ratios: the ratio of remaining processor cores to required processor cores, the ratio of remaining memory capacity to required memory capacity, and the ratio of remaining storage capacity to required storage capacity.

[0037] The load sensitivity coefficient S is calculated using the following formula:

[0038] ,in β is the dynamic weighting coefficient corresponding to the task type, which includes compute-intensive, I / O-intensive, and storage-intensive tasks; β is the dynamic weighting coefficient corresponding to the cluster resource utilization range, which is divided into low load, medium load, and high load. Cluster resource utilization refers to the average resource utilization rate of the cluster, which is a quantitative indicator that comprehensively reflects the overall load status of the cluster. It is calculated as the arithmetic mean of cluster processor utilization, memory utilization, and storage utilization, as shown in the following formula. Where: U_cpu is the cluster processor utilization rate (actual number of processor cores used / total number of processor cores); U_mem is the cluster memory utilization rate (actual memory capacity used / total memory capacity); U_stor is the cluster storage utilization rate (actual storage capacity used / total storage capacity); the cluster resource utilization rate ranges from [0,1], corresponding to the percentage range [0%,100%], and the larger the value, the higher the cluster load.

[0039] The interval division method uses a fixed quantization threshold to clearly divide three load intervals, with the specific division rules as follows: Low load interval: U < 0.3 (i.e., average cluster resource utilization < 30%), at which point the cluster resources are mostly idle and can handle new tasks; Medium load interval: 0.3 ≤ U ≤ 0.7 (i.e., 30% ≤ average cluster resource utilization ≤ 70%), at which point the cluster load is balanced and task execution efficiency is optimal; High load interval: U > 0.7 (i.e., average cluster resource utilization > 70%), at which point cluster resources are scarce, task lag is likely to occur, and resource migration or preemption mechanisms need to be triggered.

[0040] γ is a dynamic weighting coefficient corresponding to the task execution stage, which includes initialization, running, and closing. The division of task execution stages is based on the quantitative ratio of "executed time to a preset execution time threshold." The lifecycle stage of the task is defined by the relative relationship between the current execution progress and the preset time threshold. Specifically, let (T_executed) be the current executed time of the task (in hours), T be the task execution time threshold (in hours), and the division ratio be... The specific stages are divided as follows: Initialization stage <0.3 (i.e., execution time <30%), the task is in the environment deployment, data loading, and dependency initialization stage, and has not yet entered the core business logic execution stage; 0.3≤ during execution. ≤0.7 (i.e., 30%-70% of the execution time has elapsed), during this stage the task is in the core business logic execution state, resource requirements are stable, and it cannot be easily interrupted; Final stage >0.7 (meaning the execution time is >70%), at this stage the task is in the state of result summarization, data archiving, and log output. Resource requirements gradually decrease, and releasing resources has minimal impact on task completion.

[0041] δ is a fixed weight coefficient corresponding to the task dependency coefficient, and α+β+γ+δ=1; P is the task priority quantification value, ranging from 0 to 1, preset by the system according to task importance (1 for core tasks, 0.5 for ordinary tasks, and 0 for low-priority tasks); R is the total resource requirement of the target resource-consuming task, in GB (memory / storage) or cores (processors), converted according to a unified standard resource unit. This is to address the heterogeneity between CPU (time-based resource) and memory (space-based resource) by introducing a "Standard Computing Unit (SCU)" as a unified conversion benchmark to solve the problem that the two types of resources cannot be directly superimposed for calculation. The specific rule is as follows: 1 Standard Computing Unit (SCU) = 1 CPU core (base frequency 2.0GHz) × 1 hour + 1GB memory × 1 hour; Combining the "time consumption" of CPU and the "space consumption" of memory, the unit is unified through the "duration" dimension to ensure the comparability of the converted data. The total resource requirement R (unit: SCU) of the target resource-consuming task is calculated according to the following formula: ,in The number of CPU cores required for the task; The numerator represents the required memory capacity for the task (in GB); T represents the task execution time threshold (in hours); the numerator is the total CPU and memory resource usage of the task throughout its execution cycle, divided by 1 SCU to obtain a uniformly quantified total resource requirement. Note that storage resource requirements are not included in the SCU conversion; their sufficiency is independently verified through the "remaining resource ratio" (i.e., remaining storage capacity / required storage capacity ≥ the corresponding redundancy requirement of the guarantee level); the baseline CPU frequency of 2.0GHz is an industry-standard benchmark. If there is a difference between the cluster CPU frequency and the benchmark value, it can be adjusted through the "frequency correction factor" (e.g., for a CPU frequency of 3.0GHz, the correction factor is 1.5, i.e., 1 3.0GHz CPU ≈ 1.5 2.0GHz CPUs) to ensure conversion accuracy.

[0042] R1 represents the average total resource requirement of similar tasks in the same historical period, obtained through statistics from the past 3 months of historical data; T represents the execution time threshold of the target resource-consuming task, in hours. The maximum allowed execution time threshold is 72 hours in this embodiment; D is the task dependency coefficient, which ranges from 0 to 0.2 in this embodiment. It is 0 when there is no dependency, 0.1 when there is a dependency on 1-2 tasks, and 0.2 when there is a dependency on 3 or more tasks.

[0043] It should be noted that the dynamic weight coefficients are obtained through training using the dynamic weight coefficient determination method described below; specifically, δ is a fixed weight that does not participate in dynamic optimization.

[0044] Step 1: Historical data collection and preprocessing. Collect scenario dimension data, weight combination data, and corresponding resource allocation success rate data that affect weight adaptation, and construct a training dataset.

[0045] Data collection dimensions:

[0046] Scene segmentation dimensions:

[0047] Task types: compute-intensive, storage-intensive, and hybrid-intensive; Cluster load range: low load (utilization <30%), medium load (30% ≤ utilization ≤ 70%), and high load (utilization > 70%) (3 categories);

[0048] Task execution phases: initialization phase, mid-execution phase, and finalization phase (3 categories); Weight combination data: combinations of values ​​for α, β, and γ (satisfying α+β+γ=1, and α, β, γ∈(0,1)), generating 100 candidate weight combinations through grid search; For each weight combination, in the corresponding scenario, calculate the success probability of historical tasks after load sensitivity coefficient calculation, guarantee level classification, and resource allocation (success is defined as "task completed normally without resource shortage or overload interruption"). Data requirements: Sample size ≥ 5000 records, covering all candidate weight combinations in 27 scenarios, ensuring data representativeness (e.g., at least 100 samples per scenario); remove outliers in allocation success rate (e.g., 0% success rate due to cluster failure), group data by scenario, ensuring each group is independent and free from cross-contamination.

[0049] Step 2: Scenario-based grouping and definition of objective function

[0050] Scenario grouping: The preprocessed dataset is divided into 27 independent scenario groups according to "task type × cluster load range × task execution stage". Each data group corresponds to only one specific scenario (such as "computation-intensive + medium load + initialization stage") to ensure the accuracy of scenario for weight matching.

[0051] Objective function definition: With "maximizing the success rate of resource allocation" as the core objective, construct the optimization objective function as follows:

[0052] Constraints: α+β+γ=1, and α>0, β>0, γ>0 (ensuring that all three dimensions participate in the weight calculation and no dimension is omitted).

[0053] Step 3: Iterative optimization of the particle swarm optimization algorithm

[0054] The Particle Swarm Optimization (PSO) algorithm is used to iteratively optimize the weight combinations for each scene group. This algorithm has the advantages of fast convergence speed and strong global search capability, and is suitable for multi-dimensional optimization of weight parameters. Algorithm parameter initialization: Particle encoding: Each particle corresponds to a set of weight combinations (α, β, γ), and the particle dimension is 3; Population size: 50 particles are set (i.e., 50 sets of weight combinations to be optimized) to ensure comprehensive search scope; Number of iterations: The maximum number of iterations is set to 1000 to avoid local optima due to insufficient iterations; Learning factors: Acceleration coefficients c1=1.5 (individual learning factor) and c2=1.5 (global learning factor) to balance individual search and global search; Inertia weight: The initial inertia weight ω=0.9, which decreases linearly to 0.4 with the number of iterations, focusing on global exploration in the early stage and local convergence in the later stage.

[0055] Iterative optimization process: Under constraints, 50 sets of weight combinations (initial particle positions) are randomly generated; each weight combination is substituted into the historical data of the corresponding scenario to calculate the objective function value (assignment success rate), which is used as the particle's fitness; the historical best fitness (individual best pbest) and the global best fitness (global best gbest) of the population are recorded; the particle's position (weight combination) is updated according to the inertia weight, individual learning factor, and global learning factor, as shown in the following formula: Where v_i is the particle velocity, x_i is the particle position (weight combination), and r1 and r2 are random numbers in the range [0,1] to ensure the randomness of the search; the updated weight combination must satisfy α+β+γ=1, and if it does not, it is normalized (e.g., α'=α / (α+β+γ), β'=β / (α+β+γ), γ'=γ / (α+β+γ)); iteration termination: when the maximum number of iterations is reached, or when the global optimal fitness has not improved for 50 consecutive iterations, the iteration stops, and the global optimal particle position at this time is the optimal weight combination for this scene.

[0056] Step 4: Test set validation and weight solidification

[0057] Dataset partitioning: Historical data for each scene group is divided into a training set (for algorithm optimization) and a test set (for validation) in a 7:3 ratio to avoid overfitting. The optimized weight combination is substituted into the test set to calculate the allocation success rate, which must meet the requirements of "test ensemble power ≥ 90%" and "difference from training ensemble power ≤ 5%" to ensure the generalization ability of the weight combination. The validated weight combinations are then fixed into the system configuration file according to scene classification, forming a "scene-weight" mapping table (example shown in Table 1 below).

[0058]

[0059] Table 1 Scene-Weight Mapping Table

[0060] Step 5: Continuous Iterative Updates

[0061] Update trigger conditions: Weight update is triggered every 3 months, or when the actual allocation success rate of a certain scenario is lower than 85% for 10 consecutive days, an emergency update is triggered; Update process: Re-collect historical data within the period (add ≥1000 new samples), repeat steps 1-4, iteratively optimize the weight combination, and replace the original fixed weights.

[0062] Further, S3. Based on the value range of the load sensitivity coefficient of the target resource occupancy task, the target resource occupancy task is divided into different protection levels. Each cluster is screened according to the preset screening conditions corresponding to the different task protection levels, and the clusters that meet the preset screening conditions are determined as candidate clusters.

[0063] The protection levels for the target resource occupancy task include elastic protection level, regular protection level, and core protection level. The preset screening criteria for elastic protection level tasks are: the remaining resources of the cluster fully meet the resource requirements of the target resource occupancy task, the cluster's resource fragmentation is lower than a first preset fragmentation threshold, and the cluster's resource utilization rate is within a first preset utilization rate range. The preset screening criteria for regular protection level tasks are: the remaining resources of the cluster meet the resource requirements of the target resource occupancy task and have basic redundancy, the cluster's resource fragmentation is lower than a second preset fragmentation threshold, and the cluster's resource utilization rate is within a second preset utilization rate range. The preset screening criteria for core protection level tasks are: the remaining resources of the cluster meet the resource requirements of the target resource occupancy task and have sufficient redundancy, the cluster's resource fragmentation is lower than a third preset fragmentation threshold, and the cluster's resource utilization rate is within a third preset utilization rate range. The first preset fragmentation threshold > the second preset fragmentation threshold > the third preset fragmentation threshold; the range of the first preset utilization rate range is wider than the second preset utilization rate range; the range of the second preset utilization rate range is wider than the third preset utilization rate range; the resource redundancy level corresponding to core protection level is higher than that of regular protection level; and the resource redundancy level corresponding to regular protection level is higher than that of elastic protection level.

[0064] The calculation of the resource fragmentation degree of the cluster includes:

[0065] Calculate the percentage of continuously available resources (Fd) for cluster processors, memory, and storage respectively;

[0066] The calculation formula is: ,in , is the resource dimension identifier, where 1 represents cluster processors, 2 represents cluster memory, and 3 represents cluster storage; nd is the total number of contiguous resource blocks in dimension d; Let d be the length of the i-th contiguous resource block in dimension d; This is the availability status identifier for the i-th contiguous resource block in dimension d, with a value of 1 indicating availability and a value of 0 indicating occupancy. Let d be the total number of resource units in dimension d; Let be the capacity of the j-th resource unit in dimension d; The standard resource unit capacity of dimension d;

[0067] The overall resource fragmentation degree F of the cluster is calculated based on the proportion of continuously available resources in the cluster's processors, memory, and storage. The calculation formula is as follows:

[0068] ,in , , These represent the percentage of continuously available resources for cluster processors, the percentage of continuously available resources for cluster memory, and the percentage of continuously available resources for cluster storage, respectively.

[0069] The load sensitivity coefficient S is a comprehensive indicator that integrates task priority, relative resource demand, time urgency, and task dependency coefficient. Its value directly reflects the dual core attributes of "the sensitivity of the task to resource fluctuations" and "the business value of the task".

[0070] In this embodiment, S∈[0,0.4]: the task is highly tolerant of resource fluctuations and has low business value, and resource requirements can be flexibly adjusted; S∈(0.4,0.7]: the task is somewhat sensitive to resource fluctuations and has medium business value, requiring conventional resource guarantees; S∈(0.7,1.0]: the task is highly sensitive to resource fluctuations and has core business value, requiring the highest level of resource guarantees. High S-value tasks (core guarantee level) are prone to task crashes or significant business losses due to resource fluctuations, and need to be selected based on the criteria of "low fragmentation, optimal load range, and high resource redundancy" to prioritize the allocation of high-quality and stable resources and avoid the risk of preemption; medium S-value tasks (conventional guarantee level): resource fluctuations have limited impact, and the selection criteria of balancing "guarantee strength and resource utilization" are adopted to adapt to conventional resource configuration; low S-value tasks (elastic guarantee level): resource fluctuations have little impact, and lenient selection criteria are acceptable. When necessary, resources can be released to guarantee high S-value tasks to achieve a balance between overall resource utilization efficiency and core task stability.

[0071] Elastic Assurance Level (load sensitivity coefficient S∈[0,0.4]), corresponding filtering conditions and preset values: resource fragmentation threshold: <0.2, resource utilization range: [30%,75%], remaining resource ratio threshold: ≥1.0. Task characteristics: strong tolerance to resource fluctuations, low business value (such as background statistics, non-core data backup), resource requirements can be adjusted elastically; the lenient fragmentation threshold (0.2) allows for resource dispersion, the wide utilization range (30%-75%) adapts to more clusters, and the remaining resource ratio ≥1.0 only meets basic needs, avoiding resource redundancy and waste; resource elasticity space is reserved for high-level tasks, and resources of this level of task can be released when necessary to prioritize core needs.

[0072] Standard protection level (load sensitivity coefficient S∈(0.4,0.7]), 1. Corresponding screening conditions and preset values, resource fragmentation threshold: <0.15, resource utilization range: [35%,70%], remaining resource ratio threshold: ≥1.1. This level is sensitive to resource fluctuations and has moderate business value (such as routine business processing and ordinary data analysis), requiring a balance between stability and resource utilization. The fragmentation threshold is tightened to 0.15 to reduce task lag caused by resource dispersion; the utilization range is narrowed (35%-70%) to avoid waste under low load and overload risk under high load; the remaining resource ratio ≥1.1 provides a small amount of redundancy to cope with small increases in task resource demand; ensuring normal task execution without consuming too many high-quality resources, while taking into account overall resource utilization efficiency.

[0073] Core protection level (load sensitivity coefficient S∈(0.7,1.0]), 1. Corresponding screening conditions and preset values, resource fragmentation threshold: <0.1, resource utilization range: [40%,60%], remaining resource ratio threshold: ≥1.2, task characteristics: highly sensitive to resource fluctuations, core business value (such as real-time transactions, key data calculations), resource instability can easily lead to significant business losses; the strict fragmentation threshold (0.1) ensures that resources are highly concentrated, avoiding performance bottlenecks caused by memory / processor dispersion; the narrow utilization range (40%-60%) locks in the optimal load state of the cluster, with the strongest stability; the remaining resource ratio ≥1.2 provides sufficient redundancy to cope with sudden resource demands of tasks, prioritizes the allocation of high-quality and stable resources, reduces the risk of preemption and migration, and ensures the continuity of core business.

[0074] S4. Based on the preset balancing strategy, the energy efficiency data of the candidate clusters and the energy consumption nonlinear correction model, determine the target cluster from the candidate clusters and allocate the resources of the target cluster to the target resource occupation task.

[0075] In this embodiment, all candidate clusters have met the preset screening criteria, namely, the remaining resources fully meet the target task requirements and comply with the corresponding redundancy requirements, the resource fragmentation degree is lower than a preset threshold, and the resource utilization rate is within the preset range of the corresponding guarantee level. The specific steps for determining the target cluster from the candidate clusters are as follows:

[0076] Step 1: Candidate Cluster Data Collection and Preprocessing

[0077] For each candidate cluster, two types of core data are collected and processed in real time: First, basic energy efficiency data, i.e., throughput per unit of energy consumption, is obtained by dividing the number of tasks completed per unit time by the energy consumption during the same period as counted by the power management module, with the unit being "number of tasks / kWh"; Second, basic load data, including processor utilization, memory utilization, and storage utilization. Processor utilization is calculated by dividing the number of processor cores used by the total number of processor cores, memory utilization by dividing the memory capacity used by the total memory capacity, and storage utilization by dividing the storage capacity used by the total storage capacity. All three are presented as percentages and need to be converted to decimals for subsequent calculations; Third, model-fixed parameters, i.e., the fitting coefficients k1, k2, and k3 of the energy consumption nonlinear correction model, which have been completed and fixed through prior training specifically for this cluster; Fourth, baseline parameters, i.e., the rated energy consumption of the cluster, which is obtained in advance through full-load testing of the cluster (e.g., the energy consumption of a 64-core cluster running at full load for 1 hour is 12kWh).

[0078] Step 2: Calculate the cluster resource balance

[0079] First, calculate the cluster average resource utilization rate. Add the processor utilization rate, memory utilization rate, and storage utilization rate together and divide by 3 to obtain the cluster average resource utilization rate U, which is in the range of [0,1] and serves as a unified load benchmark. Then, calculate the resource balance. Calculate the variance between the cluster average resource utilization rate and the processor utilization rate, memory utilization rate, and storage utilization rate. Take the square root of this variance and then take the reciprocal, which is the resource balance. The larger this value is, the more balanced the load is across the cluster's resource dimensions, which can avoid the situation where one resource dimension is overloaded while other dimensions are idle.

[0080] Step 3: Calculate the predicted energy consumption of the cluster

[0081] Using the average cluster resource utilization U obtained in step 2 as input, we substitute it into the trained and fixed energy consumption nonlinear correction model. The model expression is E=k1×U²+k2×U+k3, where E is the predicted energy consumption (unit: kWh), and k1, k2, and k3 are the specific fitting coefficients for this cluster. For storage-intensive clusters, an IO energy consumption impact term needs to be added to the model, adjusted to E=k1×U²+k2×U+k3+k4×D_io (D_io is the IO throughput, unit: GB / s, and k4 is the IO energy consumption coefficient).

[0082] The explanation is that the training of the energy consumption nonlinear correction model includes...

[0083] Data collection: Collect historical data of the target cluster for the past 6 months, with sampling every 15 minutes, a sample size of ≥1000 records, and coverage of the utilization rate U∈[0.2,0.8] interval. The core data includes the cluster's average resource utilization rate U and actual energy consumption value; for storage-intensive clusters, additionally collect IO throughput D_io.

[0084] Data preprocessing: Outliers were removed using box plots, and U, energy consumption (and D_io) were normalized to [0,1] to eliminate dimensional differences.

[0085] Initial parameter settings: For compute / hybrid intensive clusters, the preset values ​​are k1=0.5, k2=0.3, and k3=0.2; for storage intensive clusters, the additional preset IO energy consumption coefficient is k4=0.1.

[0086] Fitting optimization: The objective is to minimize the mean square error between predicted and actual energy consumption. Gradient descent is used to iteratively update the coefficients. The initial step size is 0.001, halving every 100 iterations, with a maximum of 1000 iterations. The termination condition is a loss function value < 10⁻⁻⁶. 6 Ensure that all coefficients are positive after the update.

[0087] Model validation: Divide the training set and validation set into a 7:3 ratio, calculate the coefficient of determination R², and if R² ≥ 0.9, the validation is successful; if not, ≥ 500 medium-to-high load samples need to be added for retraining.

[0088] Consolidation and Update: The validated coefficients are consolidated according to cluster type to form a "cluster-coefficient" mapping; new data is collected every 3 months for repeated training to adapt to dynamic changes.

[0089] Step 4: Calculate the overall score

[0090] First, calculate the corrected energy efficiency ratio. Multiply the unit energy consumption throughput collected in step 1 by the cluster's rated energy consumption obtained in step 1, and then divide by the predicted energy consumption E obtained in step 3. This indicator is used to unify the energy efficiency comparison benchmark for different clusters and eliminate the comparison bias caused by differences in hardware configuration. Next, calculate the comprehensive score using a weighted summation method. The formula is: Comprehensive Score (Score) = ω1 × Resource Balance + ω2 × Corrected Energy Efficiency Ratio, where ω1 = 0.3 (weight of load balancing) and ω2 = 0.7 (weight of energy efficiency).

[0091] Step 5: Filter target clusters

[0092] First, all candidate clusters are sorted from highest to lowest based on their overall score. After sorting, the cluster with the highest score is selected as the target cluster. If multiple candidate clusters have the same overall score (difference ≤ 0.1), the resource balance of each cluster is further compared, and the cluster with higher resource balance is selected to prioritize the stability of the target task execution. After finally determining the unique target cluster, a resource allocation instruction for the target resource-consuming task is issued to that cluster, specifying the specific resource parameters such as the number of processor cores, memory capacity, and storage capacity to be allocated.

[0093] The specific implementation steps for allocating resources to the target cluster by the target resource occupancy task are as follows:

[0094] Step 1: Extract precise resource requirements from the metadata of the target resource-consuming task: clearly define the required number of processor cores (e.g., 32 cores), memory capacity (e.g., 64GB), and storage capacity (e.g., 500GB), while associating auxiliary parameters such as task priority and execution duration threshold; combine the resource configuration characteristics of the target cluster (e.g., processor architecture, memory read / write speed, storage type) and solidify the allocation parameters: for example, if the storage requirement is of the "high IO" type, prioritize allocating SSD storage resources of the cluster; prioritize allocating high-frequency processor cores for compute-intensive tasks to avoid competing with other tasks for the same core resources.

[0095] Step 2: Through the cluster resource management module (such as Kubernetes Scheduler or YARN ResourceManager), send a "resource reservation request" to the target cluster to lock the specific resource unit that matches the target task requirements. The reservation lock duration is set to 5 minutes (to ensure the instruction execution window period; if it is not started within the timeout, it will be released). The system automatically checks whether the reserved resources are occupied, locked, or fragmented and cannot be integrated by other tasks. If there is no resource conflict, proceed to the next instruction issuance step. If there is a conflict (such as the reserved processor core being occupied by a temporary high-priority task), trigger "resource re-adaptation" to re-select available resource units from the remaining resources of the target cluster (prioritizing resources of the same type and performance). If multiple adaptations fail (≥3 times), fall back to the "resource preemption negotiation mechanism" (releasing low-sensitivity task resources and then re-reserving them).

[0096] Step 3: Generate standardized resource allocation instructions according to the cluster scheduling protocol (such as Kubernetes API, OpenStack Nova API). The instructions include core fields: task ID, target cluster ID, details of the allocated resource unit (processor core list, memory address range, storage path), upper limit of resource usage time (= task execution time threshold × 1.2, such as 24h × 1.2 = 28.8h), and priority identifier (corresponding to the task guarantee level). The instructions are sent to the resource management node of the target cluster through the cluster-dedicated scheduling channel (RESTful API interface, cluster internal scheduling bus), and the instruction sending time and channel status are recorded to ensure that no instructions are lost (using TCP protocol for transmission, with a timeout retransmission mechanism: 3 retransmission intervals of 10s, 30s, and 60s respectively).

[0097] Step 4: After receiving the instruction, the resource management node of the target cluster uses kernel-level resource isolation technology (such as Linux cgroups, Docker Namespace) to mount the reserved processor, memory, and storage resources to the dedicated task runtime space to achieve resource isolation (avoiding performance interference caused by sharing resources with other tasks); it assigns the minimum permissions to the task runtime space (such as only allowing reading and writing to the specified storage directory, and only using the allocated processor cores), prohibiting unauthorized access to other cluster resources; it calls the cluster task execution engine (such as Docker, Podman), loads the target task's runtime environment (such as operating system image, dependent libraries), and starts the task based on the allocated resources, while feeding back the task startup status (success / failure) and startup time to the scheduling system.

[0098] Step 5: Within 1 minute of startup, the scheduling system collects two core indicators through the cluster monitoring module: Resource occupancy status: confirming whether the actual amount of resources used by the task is consistent with the allocation instructions (e.g., processor occupancy of 32 cores, memory occupancy of 64GB); confirming that the task has no startup failures or resource shortage errors; if the verification passes, maintain the current allocation status and enter the task operation monitoring stage (linked to subsequent migration judgment logic); if the verification fails (e.g., the actual memory usage of the task exceeds the allocated amount): recalculate the incremental resource demand based on the load sensitivity coefficient; if the incremental demand is less than or equal to the remaining resource redundancy of the target cluster (e.g., the proportion of remaining core guarantee level resources is ≥1.2), then automatically add resources (e.g., allocate an additional 16GB of memory); if the incremental demand exceeds the redundancy, then trigger the target cluster re-selection process.

[0099] refer to Figure 1 The resource allocation method also includes the following steps:

[0100] S5. If no cluster simultaneously meets all the preset screening conditions, a resource preemption negotiation mechanism is triggered. Based on the task resource dependency chain topology and load sensitivity coefficient priority, the low-sensitivity task is coordinated to release resources, and then steps S3 and S4 are re-executed. The preset migration conditions are one of the following three situations: the target cluster is overloaded, the execution progress of the target resource-occupying task has not reached the preset progress threshold, the incremental resource demand of the target resource-occupying task exceeds the preset proportion of the initial allocation, and the remaining resources of the target cluster cannot adapt to the incremental demand.

[0101] S6. Monitor the target cluster status and the execution progress of the target resource consuming task in real time. When the preset migration conditions are met, based on the updated load sensitivity coefficient of the target resource consuming task, reclassify the target resource consuming task into the corresponding task protection level and execute steps S3 and S4 to complete the resource migration.

[0102] In this embodiment, the resource preemption negotiation mechanism when there are no candidate clusters is as follows: when no cluster simultaneously meets the preset screening conditions of "resource fragmentation threshold, resource utilization range, and remaining resource ratio threshold", a closed loop of "low impact screening → cost quantification → resource release → re-screening" is used to free up suitable resources for the target task, and then steps S3 (candidate cluster screening) and S4 (target cluster determination) are executed. The specific implementation is as follows:

[0103] Step 1: Trigger Condition Verification The system automatically determines that all clusters have failed the preset screening conditions, and after trying to relax the screening threshold 3 times (such as temporarily relaxing the core protection level fragmentation to 0.12) there are still no candidate clusters, the resource preemption negotiation mechanism is officially triggered.

[0104] Step 2: Construct a Task Resource Dependency Chain Topology. Based on the "task dependency identifiers" of all running tasks, analyze the upstream and downstream resource interactions and data flow relationships to construct a topology graph G=(V,E,W): V is the set of task nodes (including the target task and all running tasks); E is the set of dependency edges (identifying direct dependencies between tasks, such as the output of task A being the input of task B); W is the set of dependency strength weights (determined by formula...). The calculation is performed, where f is the resource interaction frequency, d is the data transfer volume, and ω∈[0,1] (the larger the value, the tighter the dependency). The final output is the "mean dependency strength" (arithmetic mean of all associated dependency strengths) for each task, providing a basis for filtering low-impact tasks.

[0105] Step 3: Filtering the Preemptible Task Pool. Based on the "load sensitivity coefficient priority" (high-sensitivity tasks are given priority), the filtering conditions (quantified thresholds) for preemptible tasks are set: 1. Load sensitivity coefficient S < 0.4 (elastic guarantee level, strong fault tolerance to resource fluctuations); 2. Task execution progress ≥ 70% (in the final stage, releasing resources has little impact on itself); 3. Average dependency strength ≤ 0.5 (weak dependency, releasing resources will not affect downstream task chains). The system automatically traverses all running tasks and filters out tasks that simultaneously meet the above conditions to form a preemptible task pool.

[0106] Step 4: Calculate the Corrected Release Cost. To avoid system fluctuations caused by blindly releasing resources, calculate the "corrected release cost" for tasks in the preemptible task pool. The formula is: Corrected Release Cost = Original Release Cost × (1 + Dependency Impact Factor) - Original Release Cost, C_0 = 0.6 × t_rollback + 0.4 × r_recover, where t_rollback is the task rollback time, and r_recover is the data recovery probability (e.g., 0.1). The weights 0.6 and 0.4 are set based on the assumption that "rollback time has a greater impact on business"; - Dependency Impact Factor: ω_t,i represents the dependency strength between task t and downstream task i, and S_i represents the load sensitivity coefficient of downstream task i, quantifying the impact of this task on downstream tasks.

[0107] Step 5: Resource Release and Verification 1. Task Sorting: Sort tasks in ascending order of release cost (prioritize releasing tasks with the least impact). 2. Resource Release: Based on the resource requirement type of the target task, instruction T1 releases the corresponding type of redundant resources (the release process is implemented through the cluster resource management module, using a safe flow of "pause task first → back up data → release resources"). 3. Re-verification: After releasing resources, re-collect the real-time status data of the cluster (remaining resources, fragmentation, utilization rate), and execute step S3 (candidate cluster screening). If the cluster meets the preset screening conditions of the target task, proceed to step S4 (target cluster determination); if it still does not meet the conditions, continue releasing the next task in the sorted order, iterating a maximum of 3 times. If there are still no candidate clusters after 3 iterations, prompt "cluster hardware resources need to be expanded".

[0108] The resource migration process when the preset migration conditions are met is implemented as follows: Step 1: Real-time monitoring mechanism - Monitoring objects: target cluster status (resource utilization rate, remaining resource amount), execution progress of target resource occupancy tasks and changes in resource requirements. Data is collected through cluster monitoring modules (such as Prometheus) to ensure timely capture of status changes; monitoring data (such as cluster utilization rate, task execution time, current resource requirements) are stored in real time to provide a basis for migration condition verification.

[0109] Step 2: Migration Condition Verification (meeting any one of the following conditions is sufficient) The system automatically verifies whether the following three preset migration scenarios are triggered: 1. Target cluster is overloaded: The average resource utilization of the cluster is greater than or equal to the preset overload threshold for 5 consecutive minutes, which is considered an overload; 2. Task execution progress has not reached the preset progress threshold: The execution time of the task / the execution time threshold × 100% < the preset progress threshold; 3. The incremental resource requirement of the task exceeds the preset ratio and the cluster cannot adapt: ​​The current resource requirement of the task - the initial allocation amount > the initial allocation amount × the preset ratio, and the remaining resources of the target cluster < the incremental resource requirement.

[0110] Step 3: Update the load sensitivity coefficient. Based on the current task status (execution stage, changes in resource requirements) and the target cluster status (load range), recalculate the load sensitivity coefficient S: - Input parameter update: task execution stage (e.g., from initialization stage to mid-execution stage, γ weight adjusted), relative resource requirements (current resource requirements / historical average requirements of similar tasks), current cluster load range (e.g., from low load to high load, β weight adjusted).

[0111] Step 4: Reclassify the protection level and execute S3 and S4. 1. Reclassify the level: Based on the updated S value, redetermine the task protection level (e.g., if the original core protection level S=0.778, the updated S=0.75, it is still the core protection level); 2. Re-filter: Execute step S3 (candidate cluster filtering), based on the preset conditions of the new protection level, filter candidate clusters from all clusters (including the original target cluster) (if the original target cluster still meets the conditions, it can be selected again); 3. Determine the new target cluster: Execute step S4.

[0112] Step 5: Resource Migration Execution and Verification. 1. Release of Original Cluster Resources: Instruct the original target cluster to safely release the resources occupied by the task (process: pause task → complete data backup → release resources → record release log); 2. Allocation of New Cluster Resources: According to the new target cluster determined in step S4, execute the resource allocation process (resource reservation → conflict detection → instruction issuance → mounting isolation → task startup); 3. Verification of Migration Results: Within 1 minute after startup, verify the resource occupation status (whether it is consistent with the requirements) and running status (no errors, no lag) of the task in the new cluster. If the verification passes, the migration is complete, and real-time monitoring continues; if the verification fails, re-execute S3 and S4, with a maximum of 2 retries. If it fails, trigger an alarm.

[0113] According to an embodiment of the present invention, a resource allocation device is provided. It should be noted that the device can be used to execute the above-described resource allocation method. The specific implementation scheme and application scenario of this embodiment are the same as those of the above embodiments, and will not be repeated here.

[0114] A resource allocation device includes: a data acquisition module for acquiring real-time status data of target resource-occupying tasks and multiple clusters, including total resource allocation, resource usage, remaining resources, energy efficiency data, resource fragmentation, and task-related identifiers; an index calculation module for calculating cluster resource utilization, remaining resource ratio, and load sensitivity coefficient of the target resource-occupying task; a level classification and screening module for classifying task protection levels based on the load sensitivity coefficient, calling corresponding preset screening conditions, and determining candidate clusters; a model training and application module for training an energy consumption nonlinear correction model and calculating the corrected energy efficiency ratio and comprehensive score based on the model; a resource allocation module for determining the target cluster based on the comprehensive score and allocating resources to the target resource-occupying task; a dependency topology construction module for constructing task resource dependency chain topology and providing dependency relationship data for resource preemption; a preemption negotiation module for implementing a resource preemption negotiation mechanism, including preemptible task screening, release cost calculation, resource coordination, and verification feedback; and a monitoring and migration module for real-time monitoring of the target cluster status and task execution progress, triggering resource migration when migration conditions are met.

[0115] Embodiments of this application also provide an electronic device, including a memory and a processor; the memory stores a computer program, and when the processor executes the computer program, it implements the above-described resource allocation method.

[0116] An embodiment of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described resource allocation method.

[0117] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0118] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0119] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0120] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0121] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A resource allocation method, characterized in that: Includes the following steps; S1. Obtain real-time status data of the target resource usage task and multiple clusters; The real-time status data includes the resource usage, remaining resources, energy efficiency data, and resource fragmentation of each cluster. The target resource occupation task carries the task priority, resource requirement parameters, execution time threshold, and task dependency identifier. S2. Calculate the cluster resource utilization rate based on the total resource allocation and the resource usage; calculate the remaining resource ratio of the cluster based on the remaining resource and resource requirement parameters; calculate the load sensitivity coefficient of the target resource occupation task based on the task priority, resource requirement parameters, execution time threshold and task dependency identifier. S3. Based on the value range of the load sensitivity coefficient of the target resource occupation task, the target resource occupation task is divided into different protection levels. Each cluster is screened according to the preset screening conditions corresponding to the different task protection levels, and the clusters that meet the preset screening conditions are determined as candidate clusters. S4. Based on the preset balancing strategy, the energy efficiency data of the candidate clusters and the energy consumption nonlinear correction model, determine the target cluster from the candidate clusters and allocate the resources of the target cluster to the target resource occupation task; The method for determining the target cluster from candidate clusters, Includes the following steps; Step S41: For each candidate cluster, collect the following data in real time: basic energy efficiency data, i.e., throughput per unit of energy consumption; The load baseline data includes processor utilization, memory utilization, and storage utilization; model fixed parameters, namely the fitting coefficients k1, k2, and k3 of the energy consumption nonlinear correction model; and baseline parameters, namely the cluster's rated energy consumption. Step S42: Calculate the cluster average resource utilization U as a unified load benchmark; then calculate the resource balance by calculating the variance between the cluster average resource utilization and processor utilization, memory utilization, and storage utilization, taking the square root of the variance and then taking the reciprocal as the resource balance. Step S43: Take the average resource utilization rate U of the cluster obtained in step S42 as input and substitute it into the trained and fixed energy consumption nonlinear correction model. The model expression is E=k1×U²+k2×U+k3, where E is the predicted energy consumption, and k1, k2, and k3 are the specific fitting coefficients of the cluster. If it is a storage-intensive cluster, the IO energy consumption impact term needs to be added to the model and adjusted to E=k1×U²+k2×U+k3+k4×D_io, where D_io is the IO throughput in GB / s and k4 is the IO energy consumption coefficient. Step S44: Multiply the unit energy consumption throughput collected in step S41 by the cluster's rated energy consumption, and then divide by the predicted energy consumption E obtained in step S43 to obtain the corrected energy efficiency ratio. Then calculate the comprehensive score using a weighted summation method. The formula is: Comprehensive score Score = ω1 × resource balance degree + ω2 × corrected energy efficiency ratio, where ω1 is the weight of load balancing and ω2 is the weight of energy efficiency. Step S45: Sort all candidate clusters from high to low according to their comprehensive scores. After sorting, take the cluster ranked first as the target cluster. If multiple candidate clusters have the same overall score, the resource balance of each cluster is further compared, and the cluster with higher resource balance is selected as the target cluster.

2. The resource allocation method as described in claim 1, characterized in that: In step S3, the protection level of the target resource occupation task includes elastic protection level, regular protection level and core protection level; The preset screening conditions for elastic guarantee level tasks are that the remaining resources of the cluster fully meet the resource requirements of the target resource occupation task, the resource fragmentation of the cluster is lower than the first preset fragmentation threshold, and the cluster resource utilization rate is within the first preset utilization rate range. The preset screening criteria for regular support-level tasks are that the remaining resources of the cluster meet the resource requirements of the target resource occupation task and have basic redundancy, the resource fragmentation of the cluster is lower than the second preset fragmentation threshold, and the cluster resource utilization rate is within the second preset utilization rate range. The preset screening criteria for core protection level tasks are that the remaining resources of the cluster meet the resource requirements of the target resource occupation task and have sufficient redundancy, the resource fragmentation of the cluster is lower than the third preset fragmentation threshold, and the cluster resource utilization rate is within the third preset utilization rate range.

3. The resource allocation method as described in claim 2, characterized in that: The first preset fragmentation threshold > the second preset fragmentation threshold > the third preset fragmentation threshold; the range of the first preset utilization rate range is wider than the second preset utilization rate range; the range of the second preset utilization rate range is wider than the third preset utilization rate range; the resource redundancy level corresponding to the core protection level is higher than that of the regular protection level; and the resource redundancy level corresponding to the regular protection level is higher than that of the elastic protection level.

4. The resource allocation method as described in claim 3, characterized in that: It also includes the following steps: S5. If no cluster simultaneously meets all the preset filtering conditions, the resource preemption negotiation mechanism is triggered. Based on the task resource dependency chain topology and load sensitivity coefficient priority, the low-sensitivity tasks are coordinated to release resources, and then steps S3 and S4 are re-executed. S6. Monitor the target cluster status and the execution progress of the target resource consuming task in real time. When the preset migration conditions are met, based on the updated load sensitivity coefficient of the target resource consuming task, reclassify the target resource consuming task into the corresponding task protection level and execute steps S3 and S4 to complete the resource migration.

5. The resource allocation method as described in claim 3, characterized in that: The cluster's resource fragmentation degree is lower than a preset fragmentation threshold, wherein the calculation of the cluster's resource fragmentation degree includes: Calculate the percentage of continuously available resources F for cluster processors, memory, and storage respectively. d ; The calculation formula is: ,in , which serves as a resource dimension identifier, where 1 represents cluster processors, 2 represents cluster memory, and 3 represents cluster storage; n d The total number of contiguous resource blocks in dimension d; Let d be the length of the i-th contiguous resource block in dimension d; This is the availability status identifier for the i-th contiguous resource block in dimension d, with a value of 1 indicating availability and a value of 0 indicating occupancy. The total number of resource units in dimension d; Let be the capacity of the j-th resource unit in dimension d; The standard resource unit capacity of dimension d; The overall resource fragmentation degree F of the cluster is calculated based on the proportion of continuously available resources in the cluster's processors, memory, and storage. The calculation formula is as follows: ,in , , These represent the percentage of continuously available resources for cluster processors, the percentage of continuously available resources for cluster memory, and the percentage of continuously available resources for cluster storage, respectively.

6. The resource allocation method as described in claim 1, characterized in that: The calculation of the load sensitivity coefficient of the target resource occupancy task in step S2 includes: The load sensitivity coefficient S is calculated using the following formula: ,in α is the dynamic weight coefficient corresponding to the task type, β is the dynamic weight coefficient corresponding to the cluster resource utilization range, γ is the dynamic weight coefficient corresponding to the task execution stage, and δ is the fixed weight coefficient corresponding to the task dependency coefficient, and α+β+γ+δ=1; P is the task priority quantification value, R is the total resource requirement of the target resource occupation task; R1 is the average total resource requirement of the same type of task in the same period in history. T represents the execution time threshold for the target resource-consuming task; is the maximum allowed execution time threshold of the system; D is the task dependency coefficient.

7. The resource allocation method as described in claim 4, characterized in that: The preset migration conditions are one of three scenarios: the target cluster is overloaded, the execution progress of the target resource-consuming task has not reached the preset progress threshold, the incremental resource demand of the target resource-consuming task exceeds the preset proportion of the initial allocation, and the remaining resources of the target cluster cannot adapt to the incremental demand.

8. A resource allocation apparatus for implementing the resource allocation method according to any one of claims 1-7, characterized in that: Includes a data acquisition module: used to acquire real-time status data of the target resource occupancy task and multiple clusters, including total resource configuration, resource usage, remaining resource, energy efficiency data, resource fragmentation degree, and task-related identifiers; Metrics calculation module: used to calculate cluster resource utilization, remaining resource ratio, and load sensitivity coefficient of target resource occupancy tasks; The classification and filtering module is used to classify task protection levels based on load sensitivity coefficients, call corresponding preset filtering conditions, and determine candidate clusters. Model training and application module: used to train the energy consumption nonlinearity correction model, and calculate the corrected energy efficiency ratio and comprehensive score based on the model; Resource allocation module: used to determine the target cluster based on comprehensive scores and allocate resources to the target resource-consuming tasks; Dependency Topology Builder: Used to build the task resource dependency chain topology, providing dependency relationship data for resource preemption; The preemption negotiation module is used to implement the resource preemption negotiation mechanism, including preemptible task screening, release cost calculation, resource coordination, and verification feedback. Monitoring and migration module: Used to monitor the target cluster status and task execution progress in real time, and trigger resource migration when migration conditions are met.

9. An electronic device, characterized in that: It includes a memory and a processor; the memory stores a computer program, and when the processor executes the computer program, it implements the resource allocation method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the resource allocation method according to any one of claims 1-7.

Images