A method for dynamically allocating computing resources based on retry count and time series

Abstract

The application provides a kind of computing resource dynamic allocation method based on retry number and time sequence, belong to computer task scheduling technical field.The method aims at solving the problem that the priority setting of existing task scheduling strategy is rigid, the time and retry history of task failure are not considered comprehensively, resulting in low scheduling efficiency.The method comprises: obtaining the timestamp and retry number of the task to be processed;Based on a preset nonlinear mathematical model, calculate the first score representing the time factor according to the timestamp;Based on the nonlinear mathematical model, calculate the second score representing the retry number factor according to the retry number;Through a preset combination function, the final priority value is calculated by combining the first score and the second score.The application unifies the two key factors of time and retry number of the task into the calculation, so that the priority can dynamically and scientifically reflect the real urgency of the task, and more reasonable and efficient task scheduling is realized.

Description

Technical Field

[0001] This application relates to the field of computer task scheduling technology, and in particular to a method for dynamic allocation of computing resources based on the number of retries and time series. Background Technology

[0002] In modern distributed computing systems and microservice architectures, task execution failure is a common phenomenon, which may be caused by momentary network interruptions, temporary service unavailability, or brief resource contention. To ensure system robustness and eventual consistency, task retry mechanisms are typically introduced. However, when there are a large number of tasks awaiting retries in the system, determining the execution order of these tasks, i.e., how to assign them reasonable priorities, becomes a key technical challenge.

[0003] Existing task scheduling strategies, such as simple first-in-first-out (FIFO) or fixed priority allocation, often fail to effectively handle complex retry scenarios. These strategies are too rigid and cannot dynamically reflect the urgency of tasks. For example, a task that has just failed and a task that has failed multiple times and for a long time might be treated the same under a simple strategy, which is clearly unreasonable. This can easily lead to important tasks remaining unprocessed for extended periods or system resources being occupied by invalid retry tasks, thus reducing overall operational efficiency.

[0004] To achieve more scientific scheduling, some technical solutions attempt to incorporate mathematical models to handle the time factor, such as using Gaussian functions to simulate the decay of information confidence over time. However, these methods typically focus only on the time dimension and do not include another crucial attribute of the task: the number of retries. The number of retries reflects the persistence of a task's execution or the severity of its failures, and is an important basis for determining its priority. Therefore, scheduling models based solely on the time factor are incomplete and cannot achieve optimal resource allocation and system efficiency. Summary of the Invention

[0005] The purpose of this application is to provide a method for dynamic allocation of computing resources based on the number of retries and time series, so as to solve the technical problems of rigid priority setting of task scheduling strategy in the prior art, failure to comprehensively consider the time factor of task failure and retry history, resulting in low scheduling efficiency.

[0006] To achieve the above objectives, the present invention provides a method for dynamic allocation of computing resources based on retry counts and time series, comprising: obtaining the timestamp and retry count of the task to be processed; calculating a first score representing the time factor based on the timestamp according to a preset nonlinear mathematical model; calculating a second score representing the retry count factor based on the retry count according to the preset nonlinear mathematical model; and calculating the final priority value of the task to be processed by combining the first score and the second score through a preset combination function.

[0007] Optionally, the preset nonlinear mathematical model is a Gaussian distribution model.

[0008] Optionally, the step of calculating the first score based on the timestamp includes: determining the preset time interval to which the timestamp belongs based on the timestamp, and obtaining the time weight coefficient corresponding to the interval; normalizing the timestamp to obtain a normalized timestamp; multiplying the normalized timestamp by the time weight coefficient to obtain a weighted timestamp; and using the weighted timestamp as input to calculate the first score through the preset nonlinear mathematical model.

[0009] Optionally, the preset time interval is divided based on a preset reference date, including the time period before the reference date and the time period on and after the reference date; and, for the time period before the reference date, a time weight coefficient of less than 1 is applied; for the time period on and after the reference date, a time weight coefficient of greater than 1 is applied.

[0010] Optionally, the step of calculating the second score based on the number of retries includes: multiplying the number of retries by a preset retry weighting coefficient to obtain a weighted number of retries; and using the weighted number of retries as input to calculate the second score through the preset nonlinear mathematical model.

[0011] Optionally, the preset combination function is: Where P is the final priority value, C is a preset constant, S1 is the first score, and S2 is the second score.

[0012] Furthermore, the constant C is 1.

[0013] In another embodiment of this application, the preset combination function is: Where P is the final priority value, C is a preset constant, S1 is the first score, and S2 is the second score. and This is the preset positive adjustment coefficient.

[0014] Compared with existing technologies, this application has the following advantages: By using nonlinear mathematical models such as Gaussian distribution, the two key factors of task time and retry count are uniformly incorporated into the calculation, enabling the priority to dynamically and scientifically reflect the true urgency of the task, thus achieving more reasonable and efficient task scheduling; the weight coefficients and model parameters involved in the method are all configurable items, which can flexibly adapt the scheduling strategy to different business scenarios, improving the applicability of the method; by rationally planning and dynamically adjusting the priority of retry tasks, the backlog of retry tasks can be effectively managed, preventing system overload or avalanche phenomena that may be caused by disordered task accumulation, thus enhancing the robustness of the system. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A flowchart illustrating a method for dynamically allocating computing resources based on retry counts and time series, provided in an embodiment of this application; Figure 2 This is a schematic diagram of the framework of a dynamic allocation system for computing resources based on the number of retries and time series provided in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following will provide a more detailed description of this application in conjunction with the accompanying drawings and specific embodiments. It should be noted that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0018] Example 1 In one embodiment of this application, a method for dynamically allocating computing resources based on the number of retries and time series is provided. This method aims to comprehensively evaluate the time and retry attributes of tasks to be processed using a unified mathematical model, in order to generate a priority value that dynamically and scientifically reflects the urgency of the tasks.

[0019] Please see Figure 1 This is a flowchart illustrating a method for dynamically allocating computing resources based on retry counts and time series, provided in an embodiment of this application. This method can be applied to a computer task scheduling system, which may include hardware devices such as processors and memory, and implements the method by executing computer program instructions stored in the memory. The specific calculation process includes the following steps: Step S101: Obtain the timestamp t and retry count n of the task to be processed. In a distributed computing environment, when a task fails, the system marks it as pending retry. At this time, the system needs to record or obtain two key metadata: the precise timestamp t of the task's most recent failure, and the cumulative number of retries n so far. The timestamp t can be a standard Unix timestamp, such as the number of milliseconds since the epoch; the retry count n is a non-negative integer, initially set to 1 after the initial task failure.

[0020] Step S102: Perform time weighting. To quantify the impact of time factors on task priority, the acquired timestamp t needs to be processed. As an optional implementation, the timestamp t can be normalized to eliminate the influence of its absolute value. Specifically, a preset reference day can be selected, such as calculating the midnight timestamp of that day. Then, by calculating the difference (t - ) and divide it by the total number of milliseconds in a day (i.e. This yields a relative time value, re. This value represents the number of days since the task failed, relative to midnight of the current day. For example, -1.5 indicates that the task failed a day and a half ago.

[0021] Furthermore, to assign differentiated processing preferences to tasks in different time periods, this embodiment introduces a time-interval weighting mechanism. The preset time interval to which a task belongs is determined based on the timestamp t, and the corresponding time weight coefficient for that interval is obtained. Specifically, the preset time interval can be divided based on a reference date, for example, including periods before the reference date and periods on and after the reference date. For failure times earlier than the reference date (i.e., ... For tasks that fail on or after the reference date, a time weighting factor less than 1 (e.g., 0.95) is applied. This aims to moderately reduce the weighting and prevent the priority from growing indefinitely, thus allowing time for recently failed tasks to be processed. Correspondingly, for tasks that fail on or after the reference date (i.e.,...) For tasks involving time weighting, a time weighting factor greater than 1 (e.g., 1.1) is applied to increase the time weight and make it more readily noticed. The weighted time value is obtained by multiplying the normalized time value *re* by the selected time weighting factor. .

[0022] Step S103: Calculate the first score representing the time factor based on a preset nonlinear mathematical model. To make the change in priority over time smoother and conform to a specific pattern, this embodiment uses a nonlinear mathematical model for mapping. A preferred nonlinear mathematical model is the Gaussian distribution model, because it has the characteristic of reaching a maximum at the center point and smoothly decaying to both sides, which can well simulate the change in the importance of a specific event over time. The Gaussian function used... It can be defined as:

[0023] in, The standard deviation of the Gaussian distribution is used to control the shape of the curve, and in this embodiment it can be set to 1; Let π be the mathematical constant pi; e is the natural exponential function. The weighted time value calculated in step S102 is... Substituting x into the Gaussian function, the first score can be calculated. ,Right now .

[0024] Step S104: Perform retry count weighting. To quantify the impact of the retry count on priority, the obtained retry count n also needs to be processed. In this embodiment, the retry count n can be multiplied by a preset retry count weighting coefficient (e.g., 0.149) to obtain the weighted retry count. The retry count weighting coefficient can be used to scale the number of retry counts n to a numerical range suitable for the input Gaussian function, making it comparable in magnitude to the time factor.

[0025] Step S105: Calculate the second score representing the retry count factor based on the preset nonlinear mathematical model. Similar to step S103, this step also uses the Gaussian distribution model defined above. The weighted trial number calculated in step S104 is then used... Substituting x into the Gaussian function The second score can then be calculated. ,Right now In this way, both the time factor and the number of retry factors are quantified uniformly through the same nonlinear model, ensuring the inherent mathematical consistency between the two scores.

[0026] Step S106: Using a preset combination function, combined with the first score Second score The final priority value P was calculated. First scores representing the time factor were then obtained. And the second score representing the number of retries Next, these values ​​need to be combined to form a final overall priority value. In this embodiment, a preferred combination function form is product followed by subtraction, specifically defined as: Where P is the final priority value and C is a preset constant. The first score , The second score As a more specific implementation, the constant C can be set to 1, in which case the final priority calculation formula is: .

[0027] Understandably, according to this combination function, when the task is a recent failure (t is close to the current time, after normalization)... (close to 0) and the number of retries is small (n is small, When it is close to 0, its and All of these result in larger values, leading to a smaller final priority P. Conversely, when the task failure time is long (t represents the distant past), (The absolute value is very large) or the number of retries is extremely large (n is very large). When it is very large, its or (or both) will become so small that their product becomes too small. The value of P approaches 0, causing the final priority P value to approach 1. This achieves the scheduling goal of "the older or more stubborn the task, the higher its priority value".

[0028] Step S107: Output the final calculated priority value P. This value P can be used by the task scheduling system to sort all tasks in the "retry pool". The scheduler can select the task with the highest P value and allocate computing resources to it first for the next retry.

[0029] In summary, the method provided in this embodiment nonlinearly maps the information of the task's time and retry dimensions through a Gaussian model and calculates the final priority in a product combination manner, so that the priority can dynamically and smoothly reflect the changes in the task state, thereby achieving a more scientific and efficient dynamic allocation of computing resources.

[0030] Example 2 As an optional implementation, this embodiment modifies Embodiment 1 to demonstrate the flexibility and configurability of the method provided in this application. In some business scenarios, it may be desirable for the scheduling strategy to be more sensitive to tasks that repeatedly fail, i.e., to prioritize "stubborn" tasks with a high number of retries, as these tasks may indicate deeper system problems or lock up critical business resources. This embodiment demonstrates how to tilt the scheduling strategy towards the goal of "prioritizing tasks with a high number of retries" by adjusting the parameters in the method.

[0031] The overall process of this embodiment is basically the same as that of Embodiment 1, and follows the steps shown in Figure 1. The core difference lies in the parameter settings in step S104.

[0032] Specifically, in Example 1, the retry count weighting coefficient is set to... This is an empirical equilibrium value. However, in the scenario addressed in this embodiment, the number of retries is increased. For final priority The impact of this necessitates a significant increase in the weighting coefficient. For example, the weighting coefficient for the number of retries could be increased from... Adjust to a larger value, such as .

[0033] To illustrate the technical effects of this adjustment, a specific example will be used for comparative analysis below. Assume there are two tasks in the system awaiting retries, Task A and Task B: Task A: Failure timestamp is similar to Task B's, retries... Task B: Failure timestamp is similar to Task A, number of retries... .

[0034] Since the timestamps of their failures are close, after calculations in steps S102 and S103, their first scores are... and It will be very close, that is .

[0035] The second score is calculated below using the parameters from Examples 1 and 2, respectively. .

[0036] Under the parameters of Example 1 (retry count weighting coefficient) ): Weighted number of retries Weighted number of retries .

[0037] Second score Second score .

[0038] because Compare Further away from the center point of the Gaussian function Therefore Less than .according to Priority of Task B Prioritize tasks above task A. However, the difference between the two may not be significant.

[0039] Under the parameters of Example 2 (retry count weighting coefficient) ): Weighted number of retries Weighted number of retries Second score Second score .

[0040] At this point, the weighted input value of task B Much larger than the weighted input value of task A ,lead to The value ( It will be drastically and exponentially smaller than The value ( ), making Very close to Therefore, when calculating the final priority, Very close ,and This results in a relatively small value. In this way, the priority advantage of task B over task A is greatly amplified.

[0041] During operation, the task scheduler configured with the parameters in this embodiment will prioritize scheduling the task with the highest number of retries when faced with multiple tasks that have similar failure times but significantly different retries. This strategy helps to quickly identify and attempt to resolve "stubborn" problems in the system, preventing them from occupying resources for extended periods or causing cascading failures. It is particularly suitable for scenarios with extremely high requirements for the stability of core system functions and rapid problem response.

[0042] It is evident that by adjusting the weighting coefficients in the method (such as the retry count weighting coefficient, or the equally adjustable time weighting coefficient), the bias of the scheduling strategy can be flexibly controlled to adapt to diverse business needs and operational goals, thus demonstrating the applicability and scalability of the proposed solution.

[0043] Example 3 This embodiment aims to illustrate the core idea protected by this application, namely, using a nonlinear mathematical model (such as a Gaussian model) to quantify and combine time and retry factors, which has good scalability, and the final combination function used to combine the first score and the second score is not unique and fixed.

[0044] In this embodiment, the preliminary steps are exactly the same as in Embodiment 1. That is, the system also performs step S101 to obtain task parameters. and Steps S102 and S103 are executed to calculate the first score representing the time factor. And perform steps S104 and S105 to calculate the second score representing the retry count factor. These two scores and As a nonlinear mapping result of the original task attributes, it is the core intermediate product of the method in this application.

[0045] The innovation of this embodiment lies in step S106, namely the priority synthesis step. Compared to the method used in Embodiment 1... In this embodiment, a new combination function based on weighted sums is used to combine the product forms.

[0046] Specifically, the alternative combination function can be defined as:

[0047] in, This is the final priority value; It is a preset, relatively large constant used to set the priority benchmark and range; for example, it can take the value of... ; It is the first score ; It is the second score ;and and These are preset positive adjustment coefficients, used to adjust the relative importance of the time score and the number of retries score, respectively.

[0048] For example, in a specific scenario, it can be set , , At this point, the final priority calculation formula is:

[0049] The working logic of this combined function is as follows: Similar to Example 1, the more outdated the task or the more retries are performed, the more... or The smaller the value (the closer it is to) ).

[0050] 1. When a task has recently failed and has few retries, and Both are relatively large, making the weighted sum... The value is also relatively large, which leads to a higher final priority. The value is relatively small.

[0051] 2. When a task has failed for a long time or has been retried multiple times, or (or both) are very small, making the weighted sum The value also becomes very small, thus affecting the final priority. The value is relatively large (close to) ).

[0052] Therefore, although the specific mathematical forms and priority numerical ranges have changed, the core scheduling logic trend remains the same: "the older and more stubborn the task, the higher the priority."

[0053] Furthermore, this weighted sum format provides a more intuitive way to adjust the relative weights of the time factor and the retry factor. In the example, due to the retry score coefficient Greater than the time score coefficient This indicates that under this configuration, the scheduling strategy prioritizes the number of retries. System administrators can adjust this directly. and The proportion is used to precisely control the tendency of the scheduling strategy, making the adjustment of the scheduling strategy tendency more intuitive and convenient.

[0054] In this embodiment, the priority calculation module in the task scheduling system is configured to use a combination function of weighted summation and subtraction to replace the product combination function in Embodiment 1, while the rest of the system interaction and scheduling logic remain unchanged.

[0055] Therefore, it is understandable that the core contribution of this application lies in providing a method to map raw, linear task attributes (timestamps, number of retries) into scores with specific mathematical characteristics (e.g., high in the center, low on both sides) through a nonlinear model (such as a Gaussian model). and The mechanism. How to combine these two scores subsequently can be done in various ways, such as product combination (as in Example 1), weighted sum combination (as in this example), or any other method that can utilize... and The numerical characteristics are presented as a function to achieve the expected scheduling goals, which demonstrates the flexibility and scalability of the proposed solution.

[0056] Example 4 This embodiment describes in detail, from the perspective of system architecture and engineering implementation, how to integrate and deploy the dynamic allocation method of computing resources based on retry count and time series proposed in any of the foregoing embodiments into a complete and runnable automated task scheduling system.

[0057] Please see Figure 2 This is a schematic diagram illustrating the framework of a dynamic computing resource allocation system based on retry counts and time series, provided in an embodiment of this application. The system can be deployed on a single server or a distributed cluster, and its core functionality consists of a series of collaborative software modules. These modules, as computer programs, are stored in the system's memory and executed by the processor. The system mainly includes the following core components: Task receiving module: As the system's entry point, it is responsible for receiving task submission requests from external systems or users. Received tasks are typically in their initial state and do not contain retry information. The task receiving module formats these new tasks and places them in the task queue.

[0058] Task queue: This acts as a buffer to store tasks awaiting execution. It can be a simple first-in, first-out queue or a priority queue that supports priority sorting. The execution engine will retrieve tasks from this queue.

[0059] Execution Engine: Responsible for actually executing tasks. It retrieves tasks from the task queue and invokes the corresponding business logic for processing. The execution result may be success or failure.

[0060] Failure and Retry Management Module: As the key to implementing the retry mechanism, this module closely monitors the execution status of the execution engine. When a task fails in the execution engine (e.g., due to network timeout, unavailability of dependent services, or internal logic errors), the execution engine notifies the Failure and Retry Management module. Upon capturing the failure event, this module updates or creates metadata for the task, including recording the current timestamp 't' as the failure time and incrementing its retry count 'n'. Subsequently, the module stores this failed task with updated metadata in the retry pool.

[0061] Retry Pool: A dedicated storage area for all tasks that failed and are waiting for a retry opportunity. It can be an in-memory data structure or a persistent table in a database to prevent task loss during system restarts. Each task in the pool contains its latest failure timestamp t and the number of retries n.

[0062] Priority Calculation Module: This is the core algorithm implementation module of the technical solution in this application. It encapsulates the priority calculation logic described in embodiments 1, 2, or 3 and provides an interface to the outside world. This interface receives the task's t and n as input, and after internal steps such as normalization, weighting, Gaussian function calculation, and combination function calculation, outputs a final priority value P. This module can be designed as a stateless computational unit.

[0063] Scheduling decision module: As the core control unit of the system, it is responsible for coordinating the entire retry process. Its operation can be triggered in two ways: periodic triggering (e.g., the system timer is activated every 5 seconds) or event triggering (e.g., it is activated whenever a new failed task is added to the retry pool).

[0064] The system's operation can be described as follows: 1. A task executes and fails in the execution engine. The execution engine sends a task failure notification to the failure and retry management module, along with task information. 2. The failure and retry management module is activated, calls its internal metadata update logic, obtains the current system time as the new failure timestamp t, and increments the retry count n stored in the task object by 1. 3. Next, the failure and retry management module stores the updated task object in the retry pool. 4. At a certain moment (periodic arrival or event triggering), the scheduling decision module is activated and begins its scheduling cycle. 5. The scheduling decision module first accesses the retry pool, obtains a list of all or part of the tasks to be processed, and extracts the metadata t and n for each task in the list. 6. Subsequently, the scheduling decision module calls the priority calculation interface provided by the priority calculation module for each task. 7. The priority calculation module internally executes the complete calculation process described in the aforementioned embodiment and returns the calculated priority value P to the scheduling decision module. 8. After obtaining the latest priority P of the task, the scheduling decision module re-inserts the task into the task queue. Specifically, if the task queue is a priority queue, the task along with its priority P can be directly inserted, and the queue will automatically place it in the correct position based on the P value. If the task queue is a regular queue, the scheduling decision module can first prioritize all tasks to be retried, and then insert them into the task queue in order. 9. At this point, the rescheduling cycle of a failed task is complete. The task, carrying a dynamically calculated priority reflecting its urgency, is returned to the main task queue, where it awaits the next scheduling execution by the execution engine along with other new tasks and other retried tasks.

[0065] This embodiment provides a complete, automated closed-loop system. Through the precise collaboration of its modules, this system seamlessly integrates the dynamic priority calculation method proposed in this application into engineering practice, achieving intelligent and automated management of the retry process for failed tasks without manual intervention. This optimizes system resource utilization, improves overall task processing efficiency, and enhances system robustness and stability. The system itself, as well as the computer-readable storage medium carrying the program code of these modules, fall within the protection scope of this application.

[0066] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for dynamic allocation of computing resources based on retry count and time series, characterized in that, include: Get the timestamp and retry count of the pending task; Based on a pre-defined nonlinear mathematical model, the first score representing the time factor is calculated according to the timestamp. Based on the preset nonlinear mathematical model, a second score representing the retry count factor is calculated according to the number of retry counts; The final priority value of the task to be processed is calculated by combining the first score and the second score using a preset combination function.

2. The method according to claim 1, characterized in that, The preset nonlinear mathematical model is a Gaussian distribution model.

3. The method according to claim 1 or 2, characterized in that, The step of calculating the first score based on the timestamp includes: The preset time interval to which it belongs is determined based on the timestamp, and the time weight coefficient corresponding to the interval is obtained; The timestamp is normalized to obtain a normalized timestamp; Multiply the normalized timestamp by the time weight coefficient to obtain the weighted timestamp; And by taking the weighted timestamp as input, the first score is calculated using the preset nonlinear mathematical model.

4. The method according to claim 3, characterized in that, The preset time interval is divided based on a preset reference date, including the time period before the reference date and the time period on and after the reference date; and, for the time period before the reference date, a time weight coefficient of less than 1 is applied; for the time period on and after the reference date, a time weight coefficient of greater than 1 is applied.

5. The method according to claim 1 or 2, characterized in that, The step of calculating the second score based on the number of retries includes: The number of retries is multiplied by a preset retry weighting coefficient to obtain the weighted number of retries; And using the weighted number of retries as input, the second score is calculated through the preset nonlinear mathematical model.

6. The method according to claim 1, characterized in that, The preset combination function is: Where P is the final priority value, C is a preset constant, S1 is the first score, and S2 is the second score.

7. The method according to claim 6, characterized in that, The constant C is 1.

8. The method according to claim 1, characterized in that, The preset combination function is: Where P is the final priority value, C is a preset constant, S1 is the first score, and S2 is the second score. and This is the preset positive adjustment coefficient.

Images