Container resource scheduling method and device, storage medium and program product

Abstract

The embodiment of the invention provides a container resource scheduling method and device, a storage medium and a program product. The method comprises the steps of determining task feature analysis information; based on the task feature analysis information, allocating a container queue to each task type in a to-be-processed task; traversing the sub-containers in the container queue, and allocating a target sub-container to the current task of the to-be-processed task; the residual resource quantity in the target sub-container is greater than or equal to the task demand of the current task. According to the scheme, the task feature analysis information is determined, resource use features and prediction errors of different task types are deeply understood, the container queue is allocated to each task type in the to-be-processed task based on the task feature analysis information, and resource allocation is performed on the sub-containers in the current task traversal container queue. And flexible scheduling and efficient utilization of resources are realized.

Description

Technical Field

[0001] The present application relates to the technical field of container resource scheduling, and in particular to a container resource scheduling method, device, storage medium, and program product. Background Art

[0002] In container resource scheduling, existing technologies typically adopt a load balancing-based strategy. By monitoring the container's central processing unit (CPU) usage, memory usage, and other load conditions, new tasks are assigned to containers with lower loads. When the load is high, container resources are increased through horizontal scaling. However, this method lacks in-depth analysis of task characteristics and the diversity of resource requirements, making it difficult to accurately match tasks with container resources. For example, as a large-scale service platform, a certain project faces highly complex and dynamically changing task requests. Task types are diverse, some tasks require large amounts of memory but less CPU usage, and some have special requirements for network bandwidth. Conventional scheduling methods cannot effectively distinguish these differences, resulting in uneven resource allocation and low execution efficiency. There are also problems such as a lack of detailed analysis of task characteristics, insufficient flexibility and scalability, and the tendency to waste resources and uneven load balancing. Summary of the Invention

[0003] At least one embodiment of the present application provides a container resource scheduling method, device, storage medium and program product for solving problems such as uneven resource allocation, low execution efficiency, and resource waste caused by the lack of in-depth analysis of task characteristics and the diversity of resource requirements in existing container resource scheduling technologies, making it difficult to accurately match tasks with container resources.

[0004] In order to solve the above technical problems, this application is implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides a container resource scheduling method, including:

[0006] Determine task feature analysis information;

[0007] Allocating a container queue to each task type in the to-be-processed tasks based on the task feature analysis information;

[0008] The sub-containers in the container queue are traversed, and a target sub-container is allocated to the current task of the to-be-processed task; the remaining resource amount in the target sub-container is greater than or equal to the task requirement of the current task.

[0009] Optionally, determine task feature analysis information, including:

[0010] Collect historical mission data;

[0011] determining a variance ratio for each task type based on the historical task data;

[0012] The task feature analysis information is determined according to the probability distribution of the difference ratio and the probability distribution of the actually required resources corresponding to each task type.

[0013] Optionally, determining a difference ratio for each task type based on the historical task data includes:

[0014] Determine each task instance in the historical task data;

[0015] Calculating the difference between the actual resource usage and the predicted resource usage in each of the task instances;

[0016] determining a ratio between the difference and the actual resource usage in the task instance as a difference ratio for each task instance;

[0017] The difference ratio of each task type is determined according to the difference ratios of all task instances in the historical task data.

[0018] Optionally, allocating a container queue to each task type in the to-be-processed task based on the task feature analysis information includes:

[0019] Based on the task feature analysis information, resource quantity levels are divided; each level covers a preset percentage interval of the target probability distribution in the task feature analysis information;

[0020] According to the resource level, a corresponding container queue is allocated to each task type in the tasks to be processed; each of the container queues includes a plurality of sub-containers, and the number of the sub-containers matches the density distribution of the target probability distribution.

[0021] Optionally, traversing the sub-containers in the container queue and allocating a target sub-container to the current task of the pending task; the remaining amount of resources in the target sub-container is greater than or equal to the task requirement of the current task, including:

[0022] Obtaining task feature information of the current task;

[0023] Inputting the task characteristic information into a preset prediction model to obtain the predicted resource amount of the current task;

[0024] Integrating the predicted resource amount and the safety margin determined according to the task feature analysis information to determine the total resource amount of the current task;

[0025] A target sub-container is allocated to the current task using the target container queue corresponding to the current task and the total amount of resources.

[0026] Optionally, before integrating the predicted resource amount and the safety margin determined based on the task feature analysis information to determine the total resource amount of the current task, the method further includes:

[0027] The standard deviation and mean of the task characteristic analysis information in the current resource level are used, and a preset standard normal distribution formula is adopted to calculate the safety redundancy of the current resource level.

[0028] Optionally, allocating a target sub-container to the current task by using the target container queue corresponding to the current task and the total amount of resources includes:

[0029] Traversing the target container corresponding to the current task, and taking the first subcontainer that meets the total resource amount as the target subcontainer;

[0030] If no sub-container that satisfies the total resource amount is traversed, any one of waiting for resource release, requesting resources from the data center, or adjusting the task execution plan is performed.

[0031] In a second aspect, an embodiment of the present application provides a container resource scheduling device, including:

[0032] A first determining module is used to determine task feature analysis information;

[0033] A first processing module is configured to allocate a container queue to each task type in the to-be-processed task based on the task feature analysis information;

[0034] The second processing module is configured to traverse the sub-containers in the container queue and allocate a target sub-container to the current task of the task to be processed; the remaining resources in the target sub-container are greater than or equal to the task requirement of the current task.

[0035] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed by a processor, the steps of the method described in any one of the first aspects are implemented.

[0036] In a fourth aspect, an embodiment of the present application provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps of the method described in any one of the first aspects.

[0037] Compared with the prior art, the container resource scheduling method, device, storage medium and program product provided in the embodiments of the present application determine task feature analysis information; based on the task feature analysis information, a container queue is allocated to each task type in the to-be-processed task, the sub-containers in the container queue are traversed, and a target sub-container is allocated to the current task of the to-be-processed task; the remaining amount of resources in the target sub-container is greater than or equal to the task requirement of the current task, that is, a container queue is allocated to each task type in the to-be-processed task through task feature analysis information, and then the sub-containers with sufficient remaining resources are allocated by traversing the queue according to the task requirement, thereby achieving accurate matching of task characteristics and container resources, thereby solving the problem of uneven resource allocation. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present application. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0039] Figure 1 A flowchart of a container resource scheduling method provided in an embodiment of the present application;

[0040] Figure 2 This is a structural diagram of the container resource scheduling device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same type, and do not limit the number of objects, for example, the first object can be one or more. In addition, "or" in this application represents at least one of the connected objects. For example, "A or B" covers three options, namely, Option 1: including A but not including B; Option 2: including B but not including A; Option 3: including both A and B. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0042] The term "indication" in this application can be either a direct indication (or explicit indication) or an indirect indication (or implicit indication). A direct indication can be understood as the sender explicitly informing the receiver of specific information, the operation to be performed, or the requested result, etc. in the instruction sent; an indirect indication can be understood as the receiver determining the corresponding information based on the instruction sent by the sender, or making a judgment and determining the operation to be performed or the requested result, etc. based on the judgment result.

[0043] As described in the background technology, the existing technology has the following major problems when processing container resource scheduling: First, there is a lack of ability to conduct detailed analysis of task characteristics and resource requirements, and it is impossible to provide customized resource allocation solutions for different types of tasks; second, it lacks flexibility and scalability, and it is difficult to cope with large-scale and highly concurrent service requests; third, it is easy to cause resource waste and uneven load balancing. Some containers may not be able to effectively process tasks due to resource overload, while other containers may be idle for a long time. In order to solve at least one of the above problems, the embodiments of the present application provide a container resource scheduling method, device, storage medium and program product, which can reduce or avoid the occurrence of the above situations and improve resource utilization and task execution efficiency.

[0044] Please refer to Figure 1 , an embodiment of the present application provides a container resource scheduling method, including:

[0045] Step 11: Determine task feature analysis information.

[0046] Step 11 in this application is the basis of the entire scheduling process, and its main purpose is to provide a basis for subsequent resource allocation by analyzing the characteristics of the task. Specifically, step 11 can extract key features from historical task execution data, such as task type, resource requirements (CPU, memory, etc.), execution time, priority, IO intensive or compute intensive, etc.; analyze the characteristic relationship of historical task execution data, and identify the correlation and pattern between different tasks, such as whether certain task types require specific resources, or whether certain tasks are time-sensitive. Historical task execution data is classified or clustered by features to form structured task feature analysis information, such as a mapping relationship from task type to resource requirements. Here, step 11 provides data support for subsequent container queue allocation to ensure that resource allocation is more in line with the actual needs of the task.

[0047] Step 12: Allocate a container queue to each task type in the to-be-processed task based on the task feature analysis information.

[0048] This step allocates special container queues for different types of tasks based on the task feature analysis information in step 11, thereby realizing pre-allocation and isolation of resources. Specifically, this application can divide the available containers into different queues according to the task type or resource requirements. For example, create a queue for IO-intensive tasks and another queue for compute-intensive tasks. Based on the task feature analysis information, each task type is associated with one or more container queues to ensure that the task can be allocated to the appropriate resource pool. For example, real-time tasks are assigned to high-performance container queues. This application achieves a preliminary matching of task types and resources by pre-allocating container queues, thereby improving subsequent allocation efficiency and resource utilization.

[0049] Optionally, you can set resource limits or weights for each queue to prevent certain task types from overusing resources. For example, you can allocate more resources to queues for mission-critical tasks.

[0050] Step 13: traverse the sub-containers in the container queue and allocate a target sub-container to the current task of the task to be processed; the remaining resources in the target sub-container are greater than or equal to the task requirement of the current task.

[0051] In the embodiment of the present application, step 13 is the final resource allocation step, which finds a suitable container instance for a specific task in the allocated container queue. The current task can be to select a task from the pending tasks in order or priority for processing. For the container queue of the type to which the current task belongs, the sub-containers in the container queue are checked one by one to check whether the remaining resources of the sub-container meet the needs of the current task. If they do, they are allocated; otherwise, the traversal continues. Once a suitable sub-container is found, the task is assigned to it and the remaining resource amount of the container is updated. The present application implements precise matching in the pre-allocated resource pool (queue), ensuring that each task can obtain sufficient resources while avoiding resource waste.

[0052] This application first analyzes task characteristics, then performs macro-level resource planning (queue allocation), and finally micro-level resource allocation (container matching), forming a resource scheduling process from macro to micro, from coarse to fine. This layered design improves scheduling efficiency and resource utilization.

[0053] Optionally, the above step 11 includes:

[0054] Collect historical mission data;

[0055] determining a variance ratio for each task type based on the historical task data;

[0056] The task feature analysis information is determined according to the probability distribution of the difference ratio and the probability distribution of the actually required resources corresponding to each task type.

[0057] In an embodiment of the present application, a large amount of historical task data is collected, and the historical task data is historical task execution data, which includes the actual resource usage RTi and the predicted resource usage R^Ti of each task type Ti. According to each task type Ti in the historical task data, the ratio of its resource difference to the actual usage is calculated, and the difference ratio rTi of each task type is determined, rTi = (RTi-R^Ti) / RTi; the probability distribution PrTi(r) of the difference ratio rTi is analyzed using statistical methods (such as histograms, kernel density estimation, etc.); similarly, the actual required resources RTi are statistically analyzed to determine its probability distribution PRTi(R), and the probability distribution of the difference ratio and the probability distribution of the actual required resources corresponding to each task type are determined as the task feature analysis information.

[0058] Specifically, during the collection of historical task data, it is necessary to refine the data. Data collection is crucial for ensuring the accuracy of subsequent analysis and resource scheduling. To comprehensively and accurately understand resource usage across various task types, the following detailed data collection measures were implemented: First, data sources were identified to determine from which data would be collected. This includes, but is not limited to, logging systems, monitoring systems, and task management systems, ensuring comprehensive and real-time data. Next, data filtering was performed to select data directly related to task execution and resource usage from the data sources, including, but not limited to, each task type's unique identifier, execution time, execution duration, actual CPU time used, memory usage, disk I / O volume, and network bandwidth usage. Data cleaning was then performed on the collected raw data to remove duplicate, erroneous, and anomaly data to ensure data accuracy and usability. The cleaned data was then stored in a dedicated data warehouse or database to facilitate subsequent data analysis and processing. Through the above detailed data collection process, a complete, accurate and reliable historical task execution database can be established. The collected historical task data can be directly obtained from the historical task execution database. The historical task execution database provides a solid data foundation for subsequent task characteristic analysis and resource scheduling.

[0059] Optionally, determining a difference ratio for each task type based on the historical task data includes:

[0060] Determine each task instance in the historical task data;

[0061] Calculating the difference between the actual resource usage and the predicted resource usage in each of the task instances;

[0062] determining a ratio between the difference and the actual resource usage in the task instance as a difference ratio for each task instance;

[0063] The difference ratio of each task type is determined according to the difference ratios of all task instances in the historical task data.

[0064] In the embodiment of the present application, the difference ratio calculation is an important step to deeply understand the resource usage characteristics of the task, and its detailed process is as follows: define the calculation formula. For each task type Ti, first calculate the difference DTi between its actual resource usage RTi and the predicted resource usage R^Ti, that is, DTi = RTi-R^Ti. Next, calculate the difference ratio rTi between the difference DTi and the actual resource usage. This ratio reflects the relative size of the prediction error, and its calculation formula is rTi = DTi / RTi. Calculate the ratio of each task instance: for each task instance in the historical data, calculate its corresponding difference ratio (rTi, j) according to the above formula, where j represents the number of the task instance, that is, based on the difference ratios of all task instances in the historical task data, determine the difference ratio of each task type, and then perform summary statistics. Here, the difference ratios (rTi, j) of all instances of the same task type Ti are summarized for subsequent statistical analysis.

[0065] Optionally, this application will subsequently perform error analysis and adjustments. The calculation process is checked for outliers or extreme cases, such as when the actual resource usage of certain task instances is much lower than the predicted value, resulting in an abnormally large or small ratio rTi,j. For outliers, further analysis is performed to determine whether adjustments to the prediction model or data cleaning strategy are needed.

[0066] Optionally, determining the probability distribution is an important step in understanding the characteristics of the task type and its resource utilization deviation. In order to describe these characteristics more accurately, the following is a further explanation of the step of "determining the task characteristic analysis information based on the probability distribution of the difference ratio and the probability distribution of the actual required resources corresponding to each task type": 1) Select a statistical method. According to the characteristics of the data and the analysis requirements, select an appropriate statistical method to determine the probability distribution. Common methods include histograms, kernel density estimation, and parameterized distribution fitting (such as normal distribution, lognormal distribution, exponential distribution, etc.).

[0067] 2) Draw a distribution graph. First, use a histogram to perform a preliminary visualization of the difference ratio rTi (or adjusted statistic) for each task type Ti to observe its general distribution. Next, consider using a smoother density estimation method, such as kernel density estimation, to draw a continuous probability density function graph of the difference ratio to observe its distribution characteristics in more detail.

[0068] 3) Distribution Fitting. Try fitting the difference ratio data using different parameterized distributions, such as least squares and maximum likelihood estimation, to estimate the distribution parameters. Goodness of fit is assessed using common metrics such as the chi-square test and the Kolmogorov-Smirnov test to select the best-fitting distribution model.

[0069] 4) Parameter determination: For the selected distribution model, determine its parameters (such as mean, variance, skewness, kurtosis, etc.). These parameters will be used in subsequent resource scheduling decisions.

[0070] This application determines the probability distribution of the actually required resources. Similarly, for the actually required resources RTi, the above method is also used to determine the probability distribution. The distribution graph is drawn, and an appropriate distribution model is selected for fitting, and the distribution parameters are determined.

[0071] Step 11 of this application is crucial for understanding the resource requirements of task types and will directly impact the subsequent container resource partitioning and allocation strategies. Step 11 of this application outputs the determined difference ratio probability distribution PrTi(r) and the actual required resource probability distribution PRTi(R) as the results of the task type characteristic analysis for use in subsequent steps. Through this detailed probability distribution determination process, we can more accurately grasp the resource usage characteristics and demand distribution of task types, providing strong support for subsequent container resource scheduling.

[0072] Step 11 plays a crucial role in container resource scheduling, serving as the foundation and prerequisite for the entire scheduling strategy. Step 1 provides a comprehensive and in-depth understanding of the resource usage characteristics and demand distribution of different task types, providing a scientific basis for subsequent resource allocation and scheduling.

[0073] Specifically, step 11 first collects a large amount of historical task execution data through data collection. This data covers the actual and predicted resource usage for each task type, providing rich material for subsequent variance ratio calculation and probability distribution determination. Next, the variance ratio calculation reveals the relative size of the prediction error, reflecting the resource usage characteristics of the task type. This is important for evaluating the accuracy of the prediction model and adjusting resource allocation strategies. Finally, the probability distribution determination further refines the understanding of the characteristics of the task type. Using statistical methods, the variance ratio and the probability distribution of the actual required resources are analyzed, providing a reliable mathematical model for subsequent container resource partitioning and allocation.

[0074] Step 11 of this application builds a complete task type characteristic analysis framework through three sub-steps: data collection, difference ratio calculation, and probability distribution determination. This provides solid data support and theoretical basis for subsequent resource scheduling decisions. Successful implementation of this step will help improve the accuracy and efficiency of resource scheduling and ensure the rational and efficient use of container resources.

[0075] Optionally, the above step 12 includes:

[0076] Based on the task feature analysis information, resource quantity levels are divided; each level covers a preset percentage interval of the target probability distribution in the task feature analysis information;

[0077] According to the resource level, a corresponding container queue is allocated to each task type in the tasks to be processed; each of the container queues includes a plurality of sub-containers, and the number of the sub-containers matches the density distribution of the target probability distribution.

[0078] In the embodiment of the present application, resource quantity levels are divided based on the task feature analysis information, that is, according to the probability distribution PRTi(R) of the actually required resources RTi. For example, k levels are set, where each level covers a specific percentage interval of PRTi(R). Based on these resource quantity levels, a corresponding number of sub-containers is determined for the container queue QTi of each task type Ti. The resource quantities of these sub-containers are arranged in ascending order to match the density distribution of PRTi(R).

[0079] The purpose of categorizing resource levels here is to better manage and allocate container resources, ensuring that each task type receives the appropriate amount of resources based on its actual resource requirements. Specifically, we analyze the probability distribution of actual resource requirements: We use the probability distribution PRTi(R) of the actual required resources RTi, determined from task feature analysis information, to understand the distribution of resource requirements for different task types.

[0080] Next, determine the number of resource levels. Based on actual needs and the complexity of resource management, set an appropriate number of resource levels, k. This number should balance the sophistication of resource division with the ease of management.

[0081] Furthermore, the resource range is divided into k levels based on the distribution of PRTi(R). Each level covers a specific resource range. These ranges can be of equal width or vary based on the probability density function to ensure that each level contains similar resource requirements.

[0082] Furthermore, set the grade boundaries. Determine the upper and lower limits for each grade. These limits can be specific resource quantities or percentages relative to the overall resource distribution.

[0083] When allocating sub-container resources to each task type, for each task type Ti, a corresponding number of sub-containers are allocated to its container queue QTi based on the distribution of its historical resource usage RTi and the resource level classification. The resource amounts of these sub-containers should be arranged in ascending order, matching the density distribution of PRTi(R) to ensure that tasks are allocated to appropriate sub-containers based on their actual resource requirements.

[0084] The steps of dividing resource levels and allocating corresponding container queues to each task type in the processing tasks in this application can divide resource levels more scientifically and reasonably, and allocate appropriate sub-container resources to different task types, thereby improving the efficiency and accuracy of resource scheduling.

[0085] Optionally, after step 12 and before step 13, the method further includes:

[0086] The standard deviation and mean of the task characteristic analysis information in the current resource level are used, and a preset standard normal distribution formula is adopted to calculate the safety redundancy of the current resource level.

[0087] In an embodiment of the present application, for the resource amount of each resource amount level, the safety margin is calculated based on the difference ratio probability distribution PrTi(r) determined above. The determination of the safety margin is to ensure that the prediction error and resource fluctuation can be coped with during the resource allocation process, thereby ensuring the smooth execution of the task. The safety margin is to select a suitable confidence threshold (such as 95%, 99%, etc.) according to the stability and reliability requirements of the system. This threshold will be used to determine the upper limit of resource demand. For example, a high confidence threshold (such as 95%) can be selected to determine the upper limit of resource demand, that is, the safety margin. The safety margin determined in this application can be calculated by a statistical formula such as ΔR,Ti(j)=RTi,j×(1+1.96×σrTi,j / μrTi,j) (assuming that rTi is approximately normally distributed at a certain level j). Here, ΔR,Ti(j) represents the risk adjustment coefficient or resource requirement upper limit at the j-th level of task type Ti, which is used to determine the upper limit of resources that need to be reserved for tasks of this level to ensure that there are sufficient resources to handle tasks of this level under a certain confidence level; RTi,j represents the basic resource requirement (such as basic CPU usage, memory usage, etc.) of task type Ti at the j-th level, which is the baseline value for calculation, and subsequent adjustments are made based on this value; 1.96 represents the Z value corresponding to the 95% two-sided confidence interval of the standard normal distribution (the Z value is the critical value of the standard normal distribution corresponding to the selected confidence threshold, such as the z value corresponding to 95% confidence is approximately 1.96). In a normal distribution, approximately 95% of the data will fall within the range of ±1.96 times the standard deviation of the mean. Using 1.96 means we consider a 95% probability that resource requirements will not exceed the calculated upper limit. σrTi,j represents the standard deviation of the resource requirements for task type Ti at level j, which measures the dispersion of the data. A larger standard deviation indicates greater fluctuation in resource requirements. μrTi,j represents the mean resource requirements for task type Ti at level j, representing the average resource requirements for tasks at that level. σrTi,j / μrTi,j can be expressed as the coefficient of variation, which measures the relative dispersion of data. When comparing data with different means, standard deviations cannot be directly compared, while the coefficient of variation eliminates the influence of the mean and is more suitable for comparing relative fluctuations.

[0088] The process of determining the safety redundancy value in this application provides a strong guarantee for resource scheduling, ensuring that tasks can cope with various uncertainties and fluctuations during the resource allocation process.

[0089] The specific steps of step 12 of this application and the process of determining the safety redundancy value play the core organizational and management functions in the container resource scheduling method, and are the bridge from data insight to actual resource configuration. At this stage, the main task is to carry out refined container queue construction and resource division based on the task type characteristics and resource demand distribution obtained by analysis in step 11. Specifically, the specific steps of step 12 realize the hierarchical management of the resources required for the task through the resource level division, so that each task type can be allocated to a sub-container of the appropriate level according to its actual resource demand, which not only improves the efficiency of resource use, but also enhances the flexibility of management. The process of determining the safety redundancy value further strengthens the robustness of resource allocation. By determining the safety redundancy, a buffer zone is added to resource allocation to cope with the uncertainty and fluctuation of resource demand, ensuring that the task can be successfully completed even in the face of resource constraints during execution.

[0090] In summary, this application, through scientific resource grading and reasonable safety redundancy determination, provides a solid foundation for subsequent task allocation and resource adjustment, and is a key step in achieving efficient and stable container resource scheduling. Its overall purpose is to optimize resource allocation, improve resource utilization efficiency, and ensure smooth task execution.

[0091] Optionally, the above step 13 includes:

[0092] Obtaining task feature information of the current task;

[0093] Inputting the task characteristic information into a preset prediction model to obtain the predicted resource amount of the current task;

[0094] Integrating the predicted resource amount and the safety margin determined according to the task feature analysis information to determine the total resource amount of the current task;

[0095] A target sub-container is allocated to the current task using the target container queue corresponding to the current task and the total amount of resources.

[0096] In an embodiment of the present application, for a newly arrived current task, resource demand prediction is first performed based on its task type and task feature information to obtain a predicted resource amount. Here, the task feature information of the current task is first obtained; the task feature information is input into a preset prediction model to obtain the predicted resource amount of the current task. The above safety redundancy function is used to determine the safety redundancy, that is, to select a suitable resource level for the current task and calculate the required safety redundancy. The predicted resource amount and the safety redundancy determined based on the task feature analysis information are integrated to determine the total resource amount of the current task; the target container queue corresponding to the task type of the current task is traversed to find the first sub-container whose resource amount is greater than or equal to the total resource amount for allocation, thus completing the allocation of the target sub-container to the current task.

[0097] Optionally, allocating a target sub-container to the current task by using the target container queue corresponding to the current task and the total amount of resources includes:

[0098] Traversing the target container corresponding to the current task, and taking the first subcontainer that meets the total resource amount as the target subcontainer;

[0099] If no sub-container that satisfies the total resource amount is traversed, any one of waiting for resource release, requesting resources from the data center, or adjusting the task execution plan is performed.

[0100] In the embodiment of the present application, sub-container allocation is a key link in the resource scheduling method, which determines which specific sub-container the task will be assigned to for execution. First, according to the task type (Ti) of the newly arrived task, the corresponding container queue (QTi) is determined. This is based on the establishment of the mapping relationship between task type and container resources in the previous step. The total resource amount (Rtotal, Ti) required for task execution calculated based on the above steps is the sum of the predicted resource amount and the safety redundancy. Start traversing from the first sub-container of the container queue (QTi) corresponding to the task type (Ti). For each sub-container, check whether its remaining resource amount is greater than or equal to the total resource amount (Rtotal, Ti) required for task execution. Select sub-container: During the traversal process, find the first sub-container that meets the resource amount requirements, that is, the sub-container whose remaining resource amount is greater than or equal to (Rtotal, Ti). If no sub-container that meets the conditions is found, further processing is required according to the resource scheduling strategy, such as waiting for resource release, requesting more resources from the data center, or adjusting the task execution plan. Once a sub-container that meets the conditions is found, it is assigned to the current task and the remaining resource amount information of the sub-container is updated. This step ensures that tasks are accurately assigned to sub-containers with sufficient resources, thereby improving resource scheduling efficiency and task execution success rate. Furthermore, through resource monitoring and dynamic adjustment mechanisms, system stability and reliability can be further guaranteed.

[0101] Specifically, task resource demand prediction is a key step in ensuring that tasks can be allocated sufficient resources for successful execution. For newly arrived tasks, we first need to collect information about their task type (Ti) and other characteristics that may affect resource demand, such as task size, complexity, and expected execution time.

[0102] Select a prediction model. Based on historical data and the task feature information of the current task, select a suitable prediction model to predict resource requirements. The prediction model can be based on statistics (such as linear regression, multivariate regression), machine learning (such as decision tree, random forest, neural network) or a hybrid model. Train the prediction model. Use historical task execution data (including task type, characteristics and actual resource usage) as a training set to train the selected prediction model. Optimize the prediction accuracy by continuously adjusting the model parameters. Perform resource demand prediction. Input the characteristic information of the new task into the trained prediction model to obtain the predicted resource volume (RTi^) of the task. The prediction results should include the demand for key resources such as CPU, memory, and storage. This can more accurately predict the resource requirements of new tasks and provide strong data support for resource scheduling, thereby improving the rationality of resource allocation and the success rate of task execution. Furthermore, safety redundancy is added. The addition of safety redundancy is to ensure that there is sufficient resource buffer during the execution of the task to cope with possible resource demand fluctuations or prediction errors.

[0103] Review the task type and resource level: First, review the task type of the current task and the resource level determined above. This will help determine from which level of the container queue to select the child container for allocation. Further calculations are made to predict resource amounts and safety margins. Based on the predicted resource amounts (RTi^), combined with the safety margin function determined for the task type and resource level of the current task, the required safety margin is calculated. (The calculation of safety margins needs to take into account different resource types.) Consider the differences between task types (e.g., CPU, different types of memory redundancy, storage calculations, etc.), historical fluctuations in resource usage, and system stability requirements.

[0104] The predicted resource amount and safety margin are combined, and the predicted resource amount (RTi^) and the safety margin (ΔRTi) are added together to obtain the total resource amount required for task execution (Rtotal,Ti = RTi^ + ΔRTi). This total resource amount is used in subsequent child container allocation decisions. This allows for a more accurate determination of the total resource amount required for task execution and ensures that safety margin is fully considered during resource allocation, thereby improving the stability and success rate of task execution.

[0105] Step 13 of this application and its specific implementation steps are the core execution stage in the container resource scheduling method, which integrates three key links: resource demand prediction, safety redundancy calculation and sub-container allocation. First, through the task resource demand prediction, a preliminary resource demand assessment is provided for the task to be executed. Subsequently, according to the task characteristics and historical data, the appropriate safety redundancy is calculated and added to ensure that the task can still be executed smoothly under resource fluctuations or prediction errors. Finally, in the sub-container allocation stage, according to the total resource requirements of the task (including predicted resource volume and safety redundancy), the appropriate sub-container is selected from the corresponding container queue for allocation. This process not only improves the accuracy and efficiency of resource allocation, but also enhances the system's adaptability to changes in resource demand, providing a strong guarantee for the stable execution of tasks. Through the refined operation of step 13, the container resource scheduling method can more effectively manage and utilize resources, meet the needs of diversified tasks, and improve the performance and reliability of the overall system.

[0106] In summary, this application proposes a dynamic container resource scheduling method based on task characteristics and resource demand prediction. By collecting and analyzing historical task execution data, we have an in-depth understanding of the resource usage characteristics and prediction errors of different task types, and establish a probability distribution model to accurately characterize task resource requirements. On this basis, combined with the container queue and resource partitioning strategies, container resources that match different task types are provided. In addition, by real-time monitoring of task execution status and dynamic adjustment of resource allocation, resource utilization and task execution efficiency are further improved. This application aims to solve the problems of task diversity and resource demand specificity in larger projects, and to provide an efficient and reliable resource scheduling solution for large-scale service requests.

[0107] The container resource scheduling method described in this application aims to improve resource utilization, enhance system stability and reliability, and reduce operating costs. Upon receiving a service request, this method first retrieves pending tasks and traverses the corresponding sub-container resources, searching for a target sub-container whose current number of processing tasks has not reached the maximum limit for resource allocation. By intelligently matching task types with container resources and dynamically adjusting resource allocation strategies, flexible resource scheduling and efficient utilization are achieved.

[0108] In terms of technical implementation, this application first determines the task type of the pending task and allocates corresponding container resources based on the task type. If no resources are allocated for a specific task type, a resource allocation request is sent to the data center and newly allocated container resources are received. The new resources are divided and sorted in equal proportions and then serve as candidate sub-container resources for the pending task. Safety redundancy is also taken into account during the resource allocation process to ensure that tasks can continue to execute smoothly despite resource fluctuations or prediction errors.

[0109] In addition, this application also features a retry mechanism for task execution failures. This mechanism adjusts the maximum number of tasks that can be processed by the target subcontainer and rearranges resources until the task is successfully executed. This dynamic adjustment strategy enhances the adaptability and stability of the system and improves the success rate of task execution.

[0110] In summary, this application achieves efficient container resource utilization and stable system operation through flexible sub-container resource scheduling, intelligent task-resource matching, and robust resource management strategies. These technical advantages not only improve enterprise resource utilization efficiency and business responsiveness, but also reduce operating costs and risks, bringing significant commercial value to enterprises.

[0111] This application features a flexible sub-container resource scheduling mechanism. By traversing the sub-container resources corresponding to pending tasks and dynamically allocating resources to sub-containers whose current number of processing tasks has not reached the maximum limit, this achieves effective resource utilization and efficient scheduling. This can improve resource utilization and system responsiveness.

[0112] This application enables intelligent matching of task types with container resources. It implements a strategy for intelligently matching container resources based on task type. By pre-allocating corresponding container resources for different task types and requesting new resources from the data center when necessary, tasks can quickly find the appropriate execution environment. Furthermore, by equally dividing and sorting newly allocated resources, the resource allocation process is further optimized.

[0113] This application proposes a safety margin and dynamic adjustment mechanism. To address resource demand fluctuations and forecast errors, the concept of safety margin is introduced and factored into task allocation. Through statistical analysis and high-confidence threshold setting, a reasonable safety margin is calculated and dynamically adjusted during task execution, ensuring the stability and reliability of task execution. This mechanism is a key measure for ensuring system stability and improving task success rates.

[0114] By traversing sub-container resources and dynamically allocating them, this application ensures that each sub-container's resources are fully utilized, avoiding idle resources and waste. Compared to the existing technology, which may have problems such as uneven resource allocation or idle resources, this application significantly improves overall resource utilization and reduces operating costs. The intelligent matching mechanism between task types and container resources in this application can automatically allocate the most suitable execution environment for each task based on its characteristics and requirements. This intelligent matching method not only improves the efficiency and accuracy of task processing, but also reduces the need for manual intervention and operational complexity. Compared to the resource mismatch or resource waste that may occur in the existing technology, this application has greater flexibility and adaptability. By introducing safety redundancy and dynamic adjustment mechanisms, this application can effectively cope with fluctuations in resource demand and prediction errors, ensuring that tasks can continue to execute smoothly even under resource constraints or abnormal conditions. This robust resource management strategy improves system stability and reliability and reduces the risk of task failure due to insufficient resources. Compared to the resource bottlenecks or system crashes that may occur in the existing technology, this application provides a more solid guarantee for the stable operation of the system.

[0115] The above describes various methods of the embodiments of the present application. The following further provides apparatuses for implementing the above methods.

[0116] Please refer to Figure 2 , an embodiment of the present application further provides a container resource scheduling device, including:

[0117] A first determining module 21 is used to determine task feature analysis information;

[0118] A first processing module 22 is configured to allocate a container queue to each task type in the to-be-processed task based on the task feature analysis information;

[0119] The second processing module 23 is configured to traverse the sub-containers in the container queue and allocate a target sub-container to the current task of the task to be processed; the remaining resources in the target sub-container are greater than or equal to the task requirement of the current task.

[0120] Optionally, the first determining module 21 includes:

[0121] A collection unit, used to collect historical task data;

[0122] a first determining unit, configured to determine a difference ratio for each task type based on the historical task data;

[0123] The second determining unit is configured to determine the task feature analysis information according to the probability distribution of the difference ratio and the probability distribution of actually required resources corresponding to each task type.

[0124] Optionally, the first determining unit is specifically configured to:

[0125] Determine each task instance in the historical task data;

[0126] Calculating the difference between the actual resource usage and the predicted resource usage in each of the task instances;

[0127] determining a ratio between the difference and the actual resource usage in the task instance as a difference ratio for each task instance;

[0128] The difference ratio of each task type is determined according to the difference ratios of all task instances in the historical task data.

[0129] Optionally, the first processing module 22 includes:

[0130] A division unit, configured to divide resource quantity levels based on the task feature analysis information; each level covers a preset percentage interval of the target probability distribution in the task feature analysis information;

[0131] The first processing unit is configured to allocate a corresponding container queue to each task type in the to-be-processed tasks according to the resource level; each of the container queues includes a plurality of sub-containers, and the number of the sub-containers matches the density distribution of the target probability distribution.

[0132] Optionally, the second processing module 23 includes:

[0133] A first acquiring unit, configured to acquire task feature information of the current task;

[0134] a second acquiring unit, configured to input the task characteristic information into a preset prediction model to acquire the predicted resource amount of the current task;

[0135] A third determining unit is configured to integrate the predicted resource amount and the safety margin determined according to the task feature analysis information to determine the total resource amount of the current task;

[0136] The second processing unit is configured to allocate a target sub-container to the current task by using the target container queue corresponding to the current task and the total amount of resources.

[0137] Optionally, the container resource scheduling device further includes:

[0138] The third processing module is used to use the standard deviation and mean of the task feature analysis information in the current resource level and adopt a preset standard normal distribution formula to calculate the safety redundancy of the current resource level.

[0139] Optionally, the second processing unit is specifically configured to:

[0140] Traversing the target container corresponding to the current task, and taking the first subcontainer that meets the total resource amount as the target subcontainer;

[0141] If no sub-container that satisfies the total resource amount is traversed, any one of waiting for resource release, requesting resources from the data center, or adjusting the task execution plan is performed.

[0142] It should be noted that the device in this embodiment is a device corresponding to the above-mentioned method, and the implementation methods in the above-mentioned embodiments are all applicable to the embodiments of this device and can achieve the same technical effects. The above-mentioned device provided in the embodiment of this application can implement all the method steps implemented in the above-mentioned method embodiment and can achieve the same technical effects. The parts and beneficial effects of this embodiment that are the same as those in the method embodiment will not be specifically described here.

[0143] The present application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the container resource scheduling method embodiment described above and achieves the same technical effects. To avoid repetition, the details are omitted here. The computer-readable storage medium may be, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0144] An embodiment of the present application further provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the various processes of the above-mentioned container resource scheduling method embodiment are implemented, and the same technical effects can be achieved. To avoid repetition, they are not described here.

[0145] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0146] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0147] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A container resource scheduling method, characterized in that: include: Determine task feature analysis information; Allocating a container queue to each task type in the to-be-processed tasks based on the task feature analysis information; The sub-containers in the container queue are traversed, and a target sub-container is allocated to the current task of the to-be-processed task; the remaining resource amount in the target sub-container is greater than or equal to the task requirement of the current task.

2. The method according to claim 1, characterized in that Determine mission characteristic analysis information, including: Collect historical mission data; determining a variance ratio for each task type based on the historical task data; The task feature analysis information is determined according to the probability distribution of the difference ratio and the probability distribution of the actually required resources corresponding to each task type.

3. The method according to claim 2, characterized in that Based on the historical task data, determine the variance ratio for each task type, including: Determine each task instance in the historical task data; Calculating the difference between the actual resource usage and the predicted resource usage in each of the task instances; determining a ratio between the difference and the actual resource usage in the task instance as a difference ratio for each task instance; The difference ratio of each task type is determined according to the difference ratios of all task instances in the historical task data.

4. The method according to claim 1, wherein Based on the task feature analysis information, a container queue is allocated to each task type in the to-be-processed task, including: Based on the task feature analysis information, resource quantity levels are divided; each level covers a preset percentage interval of the target probability distribution in the task feature analysis information; According to the resource level, a corresponding container queue is allocated to each task type in the tasks to be processed; each of the container queues includes a plurality of sub-containers, and the number of the sub-containers matches the density distribution of the target probability distribution.

5. The method according to claim 1, wherein Traversing the sub-containers in the container queue and allocating a target sub-container to the current task of the pending task; the remaining amount of resources in the target sub-container is greater than or equal to the task requirement of the current task, including: Obtaining task feature information of the current task; Inputting the task characteristic information into a preset prediction model to obtain the predicted resource amount of the current task; Integrating the predicted resource amount and the safety margin determined according to the task feature analysis information to determine the total resource amount of the current task; A target sub-container is allocated to the current task using the target container queue corresponding to the current task and the total amount of resources.

6. The method according to claim 5, characterized in that Before determining the total resource amount of the current task by integrating the predicted resource amount and the safety margin determined based on the task feature analysis information, the method further includes: The standard deviation and mean of the task characteristic analysis information in the current resource level are used, and a preset standard normal distribution formula is adopted to calculate the safety redundancy of the current resource level.

7. The method according to claim 5, characterized in that Allocating a target sub-container to the current task using the target container queue corresponding to the current task and the total amount of resources includes: Traversing the target container corresponding to the current task, and taking the first subcontainer that meets the total resource amount as the target subcontainer; If no sub-container that satisfies the total resource amount is traversed, any one of waiting for resource release, requesting resources from the data center, or adjusting the task execution plan is performed.

8. A container resource scheduling device, characterized in that: include: A first determining module is used to determine task feature analysis information; A first processing module is configured to allocate a container queue to each task type in the to-be-processed task based on the task feature analysis information; The second processing module is configured to traverse the sub-containers in the container queue and allocate a target sub-container to the current task of the task to be processed; the remaining resources in the target sub-container are greater than or equal to the task requirement of the current task.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 7.

Images