Intelligent scheduling method and system based on computing power pooling technology

Abstract

The application belongs to the technical field of computers, and particularly relates to an intelligent scheduling method and system based on a computing power pooling technology. The method comprises the following steps: acquiring a resource lock waiting time cumulative value of a computing power pool; in response to the cumulative value exceeding a preset time threshold, extracting threads with abnormal computing power resource preemption frequencies from active threads to form a first thread set; allocating time slice lengths according to weight proportions of each thread in the first thread set to form a second thread set and a corresponding state transition path; finding a transaction commit sequence conflict position in the state transition path, determining a lock granularity level of the conflict thread and adjusting a transaction execution time sequence to obtain an updated state transition path; predicting a deadlock probability of each thread according to the updated state transition path and an instruction queue depth; and in response to the deadlock probability exceeding a risk threshold, generating a resource allocation instruction according to the lock granularity level and the time slice length of the corresponding thread, which is used to represent a priority preemption right of a short-period task to a computing power resource.

Description

Technical Field

[0001] This application belongs to the field of computer technology, specifically relating to an intelligent scheduling method and system based on computing power pooling technology. Background Technology

[0002] Computing pooling technology plays a key role in improving resource utilization by integrating scattered computing resources into a unified resource pool.

[0003] Current mainstream scheduling schemes employ a flat, unified processing mechanism, handling all types of tasks with a fixed time interval, ignoring the significant spans in the runtime cycles of different services. When long-cycle data migration tasks (day-level) and short-cycle high-frequency transaction tasks (millisecond-level) run intertwined in the same computing pool, the system cannot quickly respond to the preemption demands of short-cycle tasks amidst long-cycle resource changes, leading to delays or even failures of high-frequency tasks. This fragmentation of state information caused by the difference in time scales can easily trigger resource allocation chaos.

[0004] Therefore, how to construct a scheduling mechanism that is compatible with differences in multiple time scales, and how to ensure the smooth operation of long-cycle tasks while achieving precise preemption of short-cycle tasks, has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] To address the above issues, this application provides an intelligent scheduling method and system based on computing power pooling technology to achieve collaborative scheduling of tasks across multiple time scales, ensuring that short-cycle tasks accurately seize computing resources and synchronize their states during the execution of long-cycle tasks.

[0006] To achieve the above objectives, this application provides an intelligent scheduling method based on computing power pooling technology, including: Obtain the cumulative value of resource lock waiting time in the computing power pool. The cumulative value of resource lock waiting time is used to characterize the degree of resource contention among heterogeneous tasks. In response to the resource lock waiting time accumulation value exceeding the preset time accumulation threshold, threads with abnormal computing resource preemption frequency are extracted from active threads to form a first thread set; Based on the weight ratio of each thread in the first thread set, the time slice length is determined for each thread, and the execution is divided in a time slice round-robin manner to form a second thread set and the corresponding state transition path. The weight ratio is used to represent the priority of the thread in the allocation of computing resources, and the time slice length is used to represent the maximum execution time quota that the thread is allowed to occupy continuously. Find the transaction commit order conflict position in the state transition path, determine the lock granularity level of the conflicting thread at the transaction commit order conflict position, and adjust the transaction execution sequence to eliminate the conflict, thus obtaining the updated state transition path. Based on the updated state transition path and the instruction queue depth of each thread in the second thread set, the deadlock probability of each thread is predicted, whereby the instruction queue depth is used to represent the number of instructions to be executed by the thread. In response to the deadlock probability exceeding a preset risk threshold, a resource allocation instruction is generated based on the lock granularity level and time slice length of the corresponding thread. The resource allocation instruction is used to indicate the priority preemption right of short-cycle tasks for computing resources in heterogeneous tasks.

[0007] On the one hand, this application provides an intelligent scheduling system based on computing power pooling technology, including: The conflict index acquisition module is used to acquire the cumulative value of resource lock waiting time in the computing power pool. The cumulative value of resource lock waiting time is used to characterize the degree of resource contention between heterogeneous tasks. The abnormal thread extraction module is used to extract threads with abnormal computing resource preemption frequency from active threads in response to the resource lock waiting time accumulation value exceeding a preset time accumulation threshold, forming a first thread set. The differentiated scheduling module is used to determine the time slice length allocated to each thread based on the weight ratio of each thread in the first thread set, and to divide the execution in a time slice round-robin manner to form a second thread set and the corresponding state transition path. The weight ratio is used to represent the priority of the thread in the allocation of computing resources, and the time slice length is used to represent the maximum execution time quota that the thread is allowed to occupy continuously. The conflict resolution module is used to find the transaction commit order conflict position in the state transition path, determine the lock granularity level of the conflicting thread at the transaction commit order conflict position, and adjust the transaction execution sequence to eliminate the conflict and obtain the updated state transition path. The deadlock prediction module is used to predict the deadlock probability of each thread based on the updated state transition path and the instruction queue depth of each thread in the second thread set. The instruction queue depth is used to represent the number of instructions to be executed by the thread. The resource allocation module is used to generate a resource allocation instruction based on the lock granularity level and time slice length of the corresponding thread in response to the deadlock probability exceeding a preset risk threshold. The resource allocation instruction is used to indicate the priority preemption right of short-cycle tasks for computing resources in heterogeneous tasks.

[0008] The embodiments of this application have the following beneficial effects: The embodiments of this application acquire the accumulated value of resource lock waiting time and trigger subsequent scheduling operations when the accumulated value exceeds a preset threshold. This ensures that the scheduling mechanism only intervenes when resource contention reaches a certain level, avoiding frequent interference with long-cycle tasks and thus guaranteeing the smooth operation of long-cycle data migration tasks. Compared with related technologies that use a fixed time tick to handle all tasks and cannot identify differences in task time scales, the embodiments of this application achieve differentiated responses to heterogeneous tasks through adaptive threshold triggering of the accumulated resource lock waiting time. Furthermore, the embodiments of this application extract threads with abnormally high frequencies of computing power resource contention to form a first thread set, focusing scheduling operations on threads that truly have resource contention issues, thus preventing threads in long-cycle tasks that do not participate in resource contention from being subject to scheduling intervention.

[0009] In this application, embodiments allocate differentiated time slice lengths based on the weight ratio of each thread. Short-cycle tasks, due to their dynamic priority weight and transaction isolation level identifier, receive a higher weight ratio, thus obtaining a longer time slice length, enabling them to prioritize the use of computing resources. Furthermore, this application's embodiments also divide execution using a time-slice round-robin method, allowing short-cycle tasks to obtain more execution opportunities per unit time, while ensuring that long-cycle tasks still receive execution time, achieving state synchronization between the two.

[0010] The embodiments of this application locate the conflicting transaction commit order and dynamically refine the lock granularity, reducing it from table-level locks to row-level or field-level locks. This prevents the cascading cache invalidation caused by lock conflicts and avoids long-cycle tasks waiting for extended periods due to lock occupancy by short-cycle tasks. Compared to related technologies that use global locks or transaction rollback to handle transaction conflicts, the embodiments of this application completely eliminate lock wait conflicts and cache invalidation cascading reactions by dynamically refining lock granularity and adjusting timing. Furthermore, the embodiments of this application adjust the transaction execution timing, shifting the commit timestamps of conflicting transactions backward to within the refined lock granularity release time window. This causes the lock occupancy periods of conflicting transactions to be staggered across the timeline, eliminating lock wait conflicts between short-cycle and long-cycle tasks.

[0011] The embodiments of this application predict the deadlock probability of each thread based on the updated state transition path and instruction queue depth. Before the execution of short-cycle tasks, high-risk threads that may cause deadlocks are proactively identified and resource-locked to prevent short-cycle tasks from being blocked for extended periods due to deadlocks, while also preventing deadlocks from affecting long-cycle tasks. Compared to related technologies that use post-event detection and rollback mechanisms to handle deadlocks, the embodiments of this application achieve proactive prediction and early intervention of deadlocks. Subsequently, the embodiments of this application generate resource allocation instructions containing exclusive disk bandwidth access and computing core priority scheduling flags. This allows short-cycle tasks to exclusively access disk bandwidth and computing cores during execution, while long-cycle tasks are paused during the execution of short-cycle tasks and resumed after the completion of short-cycle tasks, thus granting short-cycle tasks a defined resource access window. Compared to related technologies where scheduling decisions remain at the software level, the embodiments of this application generate hardware-executable resource allocation instructions.

[0012] In summary, the embodiments of this application, through the synergistic effect of the aforementioned technical features, achieve advanced control over computing resources, ensuring the smooth operation of long-cycle data migration tasks while simultaneously enabling precise preemption and state synchronization of computing resources for short-cycle high-frequency trading tasks. Attached Figure Description

[0013] Figure 1 This is a flowchart illustrating the intelligent scheduling method based on computing power pooling technology provided in the first embodiment of this application; Figure 2 This is a flowchart illustrating the intelligent scheduling method based on computing power pooling technology provided in the second embodiment of this application; Figure 3 This is a flowchart illustrating the intelligent scheduling method based on computing power pooling technology provided in the third embodiment of this application; Figure 4 This is a flowchart illustrating the intelligent scheduling method based on computing power pooling technology provided in the fourth embodiment of this application; Figure 5 This is a flowchart illustrating the intelligent scheduling method based on computing power pooling technology provided in the fifth embodiment of this application; Figure 6 This is a flowchart illustrating the intelligent scheduling method based on computing power pooling technology provided in the sixth embodiment of this application; Figure 7 This is a flowchart illustrating the intelligent scheduling method based on computing power pooling technology provided in the seventh embodiment of this application; Figure 8 This is a flowchart illustrating the intelligent scheduling method based on computing power pooling technology provided in the eighth embodiment of this application; Figure 9This is a flowchart illustrating the intelligent scheduling method based on computing power pooling technology provided in the ninth embodiment of this application; Figure 10 This is a schematic diagram of the modules of the intelligent scheduling system based on computing power pooling technology provided in the embodiments of this application. Detailed Implementation

[0014] To enable those skilled in the art to better understand the technical solution, the present application will be described in detail below with reference to the embodiments. The description in this section is only exemplary and explanatory, and should not be used to limit the scope of protection of the present application.

[0015] Current mainstream computing power pool scheduling solutions generally adopt a flat, unified processing mechanism, often handling all types of tasks with a fixed time tick, ignoring the significant spans in the runtime of different businesses. When long-cycle data migration tasks and short-cycle high-frequency transaction tasks are intertwined in the same computing power pool, the lack of ability to identify the time sensitivity of tasks makes it impossible to quickly respond to the preemption demands of short-cycle tasks within the resource change cycle of long-cycle tasks, resulting in delays or even failures of high-frequency tasks.

[0016] Meanwhile, related technologies typically rely solely on queue length or lock wait count to detect conflicts, which cannot quantify the degree of concurrent execution overlap of tasks on the timeline, nor do they incorporate the concurrent access conflict status of memory page tables, resulting in the system being unable to distinguish the severity level of conflicts.

[0017] Specifically, when locating problematic threads, coarse-grained metrics such as global CPU utilization or average wait time are insufficient to accurately identify specific threads blocked due to abnormal preemption frequency. The frequency of involuntary context switches, a key indicator reflecting the degree of preemption, is not collected or utilized. Thread scheduling priorities are also statically configured and cannot be dynamically adjusted. Regarding time slice allocation, all threads are assigned the same time slice length or a simple priority queue, making it impossible to dynamically allocate execution quotas based on real-time weight ratios. Furthermore, thread state transition paths are not recorded, resulting in a lack of data foundation for subsequent conflict detection and deadlock prediction.

[0018] Furthermore, when transaction order conflicts are detected, related technologies typically employ heavy-handed operations such as global locks or transaction rollbacks. These methods fail to precisely pinpoint the conflict location or dynamically refine lock granularity based on the conflict severity. The cascading cache invalidation caused by lock conflicts cannot be identified or prevented. Regarding deadlock prevention, the response is delayed; the probability of resource request-to-resource-wait transitions in the state transition path is not utilized, and the thread instruction queue depth is not integrated with deadlock risk assessment. In addition, scheduling decisions typically remain at the software level and cannot be translated into executable resource allocation instructions for the underlying hardware. Disk bandwidth lacks exclusive usage rights mechanisms, and the scheduling of computing cores lacks priority scheduling markers, preventing the effective preemption of resources for short-cycle tasks.

[0019] Therefore, how to achieve collaborative scheduling of tasks across multiple time scales, ensuring the smooth operation of long-cycle tasks while enabling precise preemption and state synchronization of short-cycle tasks, has become a pressing technical problem to be solved in this field.

[0020] To address the aforementioned technical problems, one or more embodiments and combinations thereof in this specification provide an intelligent scheduling method and system based on computing power pooling technology, which solves the resource allocation conflict problem caused by the mixed operation of tasks with multiple time scales in related technologies, and realizes the accurate preemption and state synchronization of computing power resources for short-cycle tasks.

[0021] In one embodiment, see Figure 1 This provides an intelligent scheduling method based on computing power pooling technology, including the following steps: Step S100: Obtain the cumulative value of resource lock waiting time in the computing power pool. The cumulative value of resource lock waiting time is used to characterize the degree of resource contention between heterogeneous tasks.

[0022] Step S120: In response to the cumulative value of resource lock waiting time exceeding the preset cumulative time threshold, extract threads with abnormal computing resource preemption frequency from active threads to form the first thread set.

[0023] Step S140: Based on the weight ratio of each thread in the first thread set, determine the time slice length allocated to each thread, and divide the execution in a time slice round-robin manner to form the second thread set and the corresponding state transition path. The weight ratio is used to represent the priority of the thread in the allocation of computing resources, and the time slice length is used to represent the maximum execution time quota that the thread is allowed to occupy continuously.

[0024] Step S160: Locate the transaction commit order conflict position in the state transition path, determine the lock granularity level of the conflicting thread at the transaction commit order conflict position, and adjust the transaction execution sequence to eliminate the conflict, thus obtaining the updated state transition path.

[0025] Step S180: Based on the updated state transition path and the instruction queue depth of each thread in the second thread set, predict the deadlock probability of each thread. The instruction queue depth is used to represent the number of instructions to be executed by the thread.

[0026] Step S200: In response to the deadlock probability exceeding a preset risk threshold, a resource allocation instruction is generated based on the lock granularity level and time slice length of the corresponding thread. The resource allocation instruction is used to indicate the priority preemption right of short-cycle tasks for computing resources in heterogeneous tasks.

[0027] In step S100, the cumulative resource lock wait time can be used to characterize the degree of resource contention among heterogeneous tasks. For example, heterogeneous tasks include long-cycle tasks and short-cycle tasks. Long-cycle tasks refer to cross-cluster cold and hot data migration executed on a daily basis, while short-cycle tasks refer to high-frequency transaction order matching requiring millisecond-level responses.

[0028] The cumulative resource lock wait time can be used to measure the overlap between short-cycle and long-cycle tasks within a time window, as well as the contention for memory access, quantifying the intensity of the conflict between them. It can be understood that a larger cumulative resource lock wait time indicates more severe resource contention between heterogeneous tasks, and a higher necessity for subsequent scheduling intervention. The specific calculation method for the cumulative resource lock wait time can be implemented using an exponential backoff algorithm. Those skilled in the art can set relevant parameters according to the actual application scenario, which will not be elaborated here.

[0029] Alternatively, other algorithms such as random forests can be used to classify historical transaction logs to obtain a conflict classification set, which can be used to assist in tuning the parameters of the resource lock wait time accumulation value. Those skilled in the art can choose to use these algorithms according to actual needs, and no limitations are imposed here.

[0030] In step S120, when the accumulated resource lock waiting time exceeds a preset accumulated time threshold, an abnormal thread extraction operation is triggered.

[0031] The preset time accumulation threshold can be adjusted based on the historical load characteristics of the computing power pool. For example, the statistical distribution of the cumulative resource lock wait time value of the computing power pool under normal operating conditions can be collected, and the 95th percentile can be used as the basis for setting the preset time accumulation threshold. As an example, in this embodiment, the preset time accumulation threshold can be set to 20 milliseconds.

[0032] An active thread is a thread that is currently in the ready or running state and is participating in the competition of computing cores.

[0033] Threads with abnormal computing resource preemption frequency refer to threads whose number of times they are forcibly switched out of computing cores deviates from the normal level per unit time. Threads with abnormal computing resource preemption frequency are the main factor leading to an increase in the cumulative value of resource lock waiting time.

[0034] Threads with abnormally high frequency of preemption of extracted computing resources are designated as the first thread set.

[0035] In step S140, the weight percentage of each thread within the first thread set is first determined. The weight percentage is used to represent the priority of a thread in the allocation of computing resources. Threads with higher weight percentages will receive longer time slices in subsequent time slice allocations.

[0036] Each thread is allocated a corresponding time slice based on its weight. The time slice length represents the maximum execution time allowed for a thread to continuously occupy. For example, threads with a higher weight can obtain a longer time slice, while threads with a lower weight can obtain a shorter time slice.

[0037] Then, the threads within the first thread set can be divided for execution using a time-slice rotation method. Time-slice rotation means that each thread takes turns occupying a computing core according to its allocated time slice. When the execution time of the current thread reaches its allocated time slice length, the thread is forcibly switched out of the computing core, and the next thread begins execution. By using time-slice rotation, a single thread can be prevented from monopolizing a computing core for an extended period, ensuring that all threads within the first thread set have an opportunity to execute.

[0038] After the aforementioned time slice allocation and segmented execution, a second set of threads and corresponding state transition paths are formed. The second set of threads refers to the set of threads after differentiated time slice allocation. The state transition path refers to the temporal trajectory of each thread transitioning between different states (e.g., ready, running, waiting) during execution. This state transition path is used for conflict detection and deadlock prediction in subsequent steps.

[0039] In step S160, firstly, the transaction commit order conflict location in the state transition path is located. A transaction commit order conflict refers to a situation where, during concurrent execution, the order in which at least two transactions operate on the same data object is inconsistent with the order required by the serializable scheduling. The transaction commit order conflict location is a node in the state transition path that marks the occurrence of such a conflict.

[0040] Based on the conflict position in the transaction commit order, the lock granularity level of the conflicting thread is determined. The lock granularity level represents the scope of the locking operation; it includes table-level locks, row-level locks, and field-level locks, with higher granularity values ​​indicating finer lock granularity. Optionally, other algorithms such as K-means clustering can be used to cluster the marker bits, obtaining the alignment of the marker bits to assist in locating conflict nodes requiring finer lock granularity. Those skilled in the art can choose to use these algorithms according to actual needs; no limitations are imposed here.

[0041] Adjusting the execution order of transactions can eliminate conflicts. As an example, adjusting the execution order of transactions involves fine-tuning the commit timestamps of conflicting transactions to fit within a more refined lock-granularity release time window. This causes the lock holding periods of conflicting transactions to be staggered across the timeline, thereby avoiding a chain reaction of cache invalidation caused by lock waiting.

[0042] After the above adjustments, the updated state transition path is obtained. The updated state transition path is one that has eliminated transaction commit order conflicts and cache invalidation cascading effects, and is used for deadlock probability prediction in subsequent steps.

[0043] In step S180, the updated state transition path and the instruction queue depth of each thread in the second thread set are first obtained. The instruction queue depth is used to represent the number of instructions to be executed by the thread.

[0044] Deadlock probability refers to the likelihood that a thread will deadlock during future execution. The deadlock probability ranges from 0 to 1, with a higher value indicating a higher risk of deadlock.

[0045] Optionally, the transition probability from the resource request state to the resource wait state for each thread can be extracted from the updated state transition path. The transition probabilities and the normalized instruction queue depth are then input into the Hidden Markov Model, and the deadlock probability for each thread is output after matrix iterative calculation.

[0046] Alternatively, the normalization formula for the instruction queue depth can be:

[0047] in, This represents the normalized instruction queue depth. This represents the instruction queue depth of the current thread. This represents the preset maximum instruction queue depth. For example, if the current instruction queue depth is 120 and the preset maximum instruction queue depth is 200, then after normalization, we get 0.6.

[0048] Alternatively, the Hidden Markov Model can be computed as follows: Let the observation sequence be The state sequence is Deadlock probability Calculated recursively using the Viterbi algorithm:

[0049] in, This represents the maximum probability of being in state j at time t. It is the transition probability from state i to state j. It was observed in state j. The probability of deadlock. In this embodiment, the states include resource request state and resource wait state, and the observed value is the normalized value of the thread's instruction queue depth. After recursive calculation, the final output deadlock probability is... .

[0050] It is understood that the above formulas and calculation processes are for illustrative purposes only and do not limit the scope of protection of this embodiment.

[0051] In step S200, threads with a deadlock probability exceeding a preset risk threshold are marked as high-risk threads. The preset risk threshold can be adjusted based on the historical deadlock frequency of the computing power pool. For example, the deadlock probability statistics of each thread when deadlocks occur in the computing power pool during historical operations can be collected, and the 90th percentile can be used as the basis for setting the preset risk threshold. In this embodiment, the preset risk threshold is exemplarily set to 0.8.

[0052] Resource allocation instructions are generated based on the lock granularity level and time slice length of the high-risk thread. The lock granularity level indicates the scope of the lock held by the high-risk thread. The time slice length indicates the maximum execution time quota that the high-risk thread is allowed to hold continuously.

[0053] Resource allocation instructions are used to indicate the priority preemption right of short-cycle tasks for computing resources in heterogeneous tasks. These instructions include exclusive disk bandwidth access and a compute core priority scheduling flag. Exclusive disk bandwidth access grants short-cycle tasks temporary exclusive access to disk I / O bandwidth. The compute core priority scheduling flag elevates the scheduling priority of the short-cycle task to the highest level.

[0054] Upon receiving a resource allocation instruction, the underlying hardware driver or kernel scheduling module allocates disk bandwidth to short-cycle tasks for exclusive use based on the exclusive disk bandwidth permissions specified in the instruction. It also prioritizes the allocation of designated computing cores to short-cycle tasks based on the computing core priority scheduling flag in the instruction. During execution, the short-cycle task receives a defined resource access window, releases the resources after completing the disk access operation, and the data migration task resumes execution.

[0055] Optionally, a token bucket algorithm is used to allocate computing power to short-cycle tasks. The size of the token bucket can be adjusted according to the maximum computing power requirement of the short-cycle task; in this embodiment, the bucket size is exemplarily set to 2000 tokens. The token generation rate is set to 200 tokens per millisecond, meaning 200 tokens are added to the bucket every millisecond. Each time a short-cycle task consumes a token, it receives a computing power quota to execute a certain number of instructions. If the token bucket is empty, the task must wait for the next token generation cycle. Simultaneously, the number of computing cores allocated to short-cycle tasks is dynamically increased from the usual 4 cores to 16 cores. The above is for illustrative purposes only.

[0056] As an example, a thread with a deadlock probability of 0.92 is marked as a high-risk thread. The lock granularity level of a high-risk thread is a field-level lock, and its time slice length is 2.5 milliseconds. The scheduler generates a resource allocation instruction based on the lock granularity level and time slice length of the high-risk thread and sends this instruction to the underlying hardware driver. The resource allocation instruction includes exclusive disk bandwidth (allocating a direct memory access channel with a rate of 500 megabytes per second) and a computing core priority scheduling flag (binding short-cycle tasks to physical cores 0 through 3 and raising their real-time scheduling priority to level 99). Upon receiving the instruction, the underlying hardware driver allocates disk bandwidth exclusively to the short-cycle task and prioritizes the execution of physical cores 0 through 3. The short-cycle task completes its disk access operation within its 2.5-millisecond time slice, releases the resources, and then the data migration task resumes execution.

[0057] Optionally, after a short-cycle task completes its disk access operation, a time window overlap metric is calculated in real time. This is done by subtracting the actual time taken by the current transaction task from the expected recovery time of the data migration task. When the difference changes from negative to positive, it indicates that resource contention between the two tasks on the same timeline has been successfully prevented. For example, the difference changes from -15.5 milliseconds to +5.2 milliseconds, resulting in a resource allocation outcome that avoids further increases in the time window overlap metric.

[0058] Optionally, other algorithms such as Support Vector Machines can be used to evaluate the cycle length of resource allocation instructions and determine the priority scheduling identifier of the target processor. Optionally, other models such as Multilayer Perceptrons can be used, taking the lock granularity level and time slice length as input feature vectors, and generating resource allocation instructions after forward propagation. Optionally, other algorithms such as Exponential Smoothing can be used to predict the execution cycle of the current task and combine it with historical average execution time to calculate the computing power allocation time slice length. Those skilled in the art can choose to use the above algorithms according to actual needs; no limitations are imposed here.

[0059] This embodiment acquires the accumulated resource lock waiting time value and triggers subsequent scheduling operations when the accumulated value exceeds a preset threshold. This ensures that the scheduling mechanism only intervenes when resource contention reaches a certain level, avoiding frequent interference with long-cycle tasks and thus guaranteeing the smooth operation of long-cycle data migration tasks. Compared with related technologies that use a fixed time tick to handle all tasks and cannot identify differences in task time scales, this embodiment achieves differentiated responses to heterogeneous tasks through adaptive threshold triggering of the accumulated resource lock waiting time value. Simultaneously, this embodiment extracts threads with abnormally high computing power resource contention frequency to form a first thread set, focusing scheduling operations on threads that truly experience resource contention issues, ensuring that threads in long-cycle tasks that do not participate in resource contention are not subject to scheduling intervention.

[0060] This embodiment allocates differentiated time slice lengths based on the weight ratio of each thread. Short-cycle tasks, due to their dynamic priority weight and transaction isolation level identifier, receive a higher weight ratio, thus obtaining a longer time slice length, enabling short-cycle tasks to prioritize the use of computing resources. Furthermore, this embodiment also uses a time-slice round-robin method to divide execution, allowing short-cycle tasks to obtain more execution opportunities per unit time, while ensuring that long-cycle tasks still receive execution time, achieving state synchronization between the two.

[0061] This embodiment addresses the conflict in transaction commit order by locating the conflicting points and dynamically refining the lock granularity. The lock granularity is reduced from table-level locks to row-level or field-level locks, preventing a chain reaction of cache invalidation caused by lock conflicts and avoiding long-cycle tasks waiting for extended periods due to lock occupancy by short-cycle tasks. Compared to related technologies that use global locks or transaction rollback to handle transaction conflicts, this embodiment completely eliminates lock wait conflicts and cache invalidation chain reactions through dynamic refinement of lock granularity and timing adjustments. Furthermore, this embodiment adjusts the transaction execution timing, shifting the commit timestamps of conflicting transactions backward to within the refined lock granularity release time window. This causes the lock occupancy periods of conflicting transactions to be staggered across the timeline, eliminating lock wait conflicts between short-cycle and long-cycle tasks.

[0062] This embodiment predicts the deadlock probability of each thread based on the updated state transition path and instruction queue depth. Before the execution of short-cycle tasks, it proactively identifies high-risk threads that may cause deadlock and intervenes by locking resources, preventing short-cycle tasks from being blocked for extended periods due to deadlock and preventing deadlock from affecting long-cycle tasks. Compared to related technologies that use post-event detection and rollback mechanisms to handle deadlock, this embodiment achieves proactive prediction and early intervention of deadlock. Subsequently, this embodiment generates resource allocation instructions containing exclusive disk bandwidth access and computing core priority scheduling flags, allowing short-cycle tasks to exclusively access disk bandwidth and computing cores during execution, while long-cycle tasks are paused during the execution of short-cycle tasks and resumed after the completion of short-cycle tasks, thus granting short-cycle tasks a defined resource access window. Compared to related technologies where scheduling decisions remain at the software level, this embodiment generates hardware-executable resource allocation instructions.

[0063] In summary, this embodiment achieves advanced control over computing resources through the synergistic effect of the aforementioned technical features. While ensuring the smooth operation of long-cycle data migration tasks, it also enables precise acquisition and state synchronization of computing resources for short-cycle high-frequency trading tasks.

[0064] In one embodiment, see Figure 2 Step S100 includes: Step S101: Collect the bandwidth utilization rate of disk input and output at a preset sampling frequency, and read the memory access contention flag. The memory access contention flag is used to indicate whether the memory controller has detected a concurrent access conflict.

[0065] Step S102: Use the sliding window algorithm to perform time-series analysis on the collected bandwidth utilization rates, and identify the time window overlap metric of the transaction task and the migration task. The time window overlap metric is used to represent the proportion of time that the transaction task and the migration task are executed concurrently within the same time period.

[0066] Step S103: In response to the time window overlap metric being greater than a preset threshold and memory access contention being identified as conflicting, calculate the single conflict wait time according to an exponentially increasing rule.

[0067] Step S104: Accumulate the conflict waiting time to obtain the cumulative value of resource lock waiting time.

[0068] In step S101, the preset sampling frequency can be adjusted according to the disk response speed and task time sensitivity. For example, the average response time of the disk under typical load is collected, and half of that time is used as the basis for adjusting the sampling frequency. In this embodiment, the preset sampling frequency is exemplarily set to 5 milliseconds.

[0069] Disk I / O bandwidth utilization rate refers to the ratio of the bandwidth used by disk read / write operations per unit time to the maximum limit of the total disk bandwidth.

[0070] Optionally, the formula for calculating disk input / output bandwidth utilization is:

[0071] in, This refers to disk input / output bandwidth utilization. For disk read speed, This refers to the disk write speed. This represents the maximum total disk bandwidth.

[0072] As an example, with a read rate of 850 megabytes per second, a write rate of 1200 megabytes per second, and a total bandwidth limit of 2500 megabytes per second, the bandwidth utilization rate is 82%.

[0073] The memory access contention flag indicates whether the memory controller has detected a concurrent access conflict. It can be determined based on the concurrent access error codes recorded by the memory controller. The flag is true when a concurrent access conflict exists.

[0074] In step S102, optionally, when the bandwidth utilization rate of three consecutive time windows all exceeds 80%, the calculation of the time window overlap metric is triggered. 80% is only used as an example of the bandwidth utilization rate threshold, which can be adjusted according to the redundancy capacity of disk bandwidth.

[0075] The sliding window algorithm refers to a method that sets a fixed-length time window, scans the data sequence window by window, and performs statistical calculations within each window. The time window length can be adjusted according to the response time requirements of the high-frequency trading task. For example, the maximum allowable delay of the high-frequency trading task can be used as the basis for adjusting the time window length. In this embodiment, the time window length is set to 50 milliseconds.

[0076] Transaction tasks refer to short-cycle, high-frequency transaction tasks, such as order matching transactions requiring millisecond-level response times. Migration tasks refer to long-cycle data migration tasks, such as cross-cluster cold and hot data migration executed on a daily basis.

[0077] The time window overlap metric is the ratio of the time during which transaction tasks and migration tasks are executed concurrently within the same time period to the total lifecycle of the transaction task, used to indicate the degree of overlap between the two on the time axis.

[0078] In step S103, the preset threshold value can be adjusted according to the requirements of the computing power pool for conflict sensitivity. For example, the statistical distribution of time window overlap measurement values ​​under normal operating conditions can be collected, and the 90th percentile can be used as the basis for setting the preset threshold value. In this embodiment, the preset threshold value is set to 0.5.

[0079] An exponentially increasing rule is used to calculate the single conflict waiting time, so that the conflict waiting time increases rapidly with the number of conflicts, avoiding repeated invalid retries for short-cycle tasks when resource contention is intense, and reducing continuous interference to long-cycle tasks.

[0080] The formula for calculating the exponentially increasing rule can be: the waiting time for the nth conflict is tn = t0 × 2. n Where t0 is the initial waiting time and n is the number of collisions. The initial waiting time can be adjusted based on the average disk I / O response time, for example, twice the average disk I / O response time can be used as the initial waiting time. In this embodiment, the initial waiting time is exemplarily set to 2 milliseconds.

[0081] Optionally, the exponential increment rule can be triggered after four consecutive memory page table access conflicts. The number of consecutive conflicts can be tuned according to the system's tolerance for conflicts; in this embodiment, it is exemplarily set to four.

[0082] In step S104, the formula for calculating the cumulative value of resource lock waiting time is:

[0083] in, For the first The waiting time for the next conflict This represents the total number of conflicts.

[0084] The cumulative resource lock wait time value is used to represent the sum of historical conflict wait times, reflecting the overall resource contention among heterogeneous tasks in the computing power pool.

[0085] Optionally, the accumulated resource lock waiting time is substituted into a linear mapping function and multiplied by a baseline conflict coefficient to obtain an absolute conflict intensity index. The baseline conflict coefficient can be tuned according to the system's conflict sensitivity; in this embodiment, it is set to 1.5. The absolute conflict intensity index is then determined to fall within a preset numerical range according to dynamic interval division rules, generating a state change interval. For example, if the accumulated resource lock waiting time is 30 milliseconds, multiplying it by the baseline conflict coefficient of 1.5 yields an absolute conflict intensity index of 45, falling within the range of 30 to 50, thus generating a state change interval. The state change interval can be used to adjust the aggressiveness of time slice allocation in step S140; for example, a conservative time slice allocation can be used when the state change interval is low, and an aggressive time slice allocation can be used when it is high. Those skilled in the art can choose whether to use a state change interval based on the actual application scenario; this is not limited here.

[0086] This embodiment collects disk input / output bandwidth utilization at a preset sampling frequency and reads memory access contention flags. It employs a sliding window algorithm to identify time window overlap metrics. When the time window overlap metric exceeds a preset threshold and the memory access contention flag indicates a conflict, the single-conflict waiting time is calculated exponentially and accumulated to obtain the cumulative resource lock waiting time. This achieves precise quantification of the degree of resource contention among heterogeneous tasks. Compared to related technologies that rely on only a single indicator and cannot distinguish conflict levels, this embodiment comprehensively quantifies conflict intensity from both temporal and spatial dimensions, providing precise triggering conditions for subsequent scheduling operations and achieving advanced control over computing resources.

[0087] In one embodiment, see Figure 3 Step S101 includes: Step S1011: Read the concurrent access error codes recorded in the memory controller.

[0088] Step S1012: Convert the concurrent access error code to a Boolean identifier.

[0089] Step S1013: In response to the Boolean flag being true, determine that the memory access contention flag indicates a conflict exists.

[0090] In step S1011, the concurrent access error code recorded in the memory controller refers to the error status code recorded by the memory controller hardware when multiple threads simultaneously access the same memory page table or memory address. The concurrent access error code can be obtained by reading specific status registers in the memory controller, such as reading register bits related to page table conflict detection.

[0091] In step S1012, the Boolean identifier refers to the identifier that converts the concurrent access error code into a logical true or logical false identifier.

[0092] Optionally, the Boolean flag is true when the concurrent access error code is non-zero, and false when the concurrent access error code is zero. For example, if the concurrent access error code is 0x0001 (indicating a page table conflict), the Boolean flag is true; if the concurrent access error code is 0x0000 (indicating no conflict), the Boolean flag is false.

[0093] In step S1013, when the Boolean flag is true, the memory access contention flag is determined to be conflicted, indicating that there is currently concurrent memory access contention; when the Boolean flag is false, the memory access contention flag is determined to be conflict-free.

[0094] This embodiment reads the concurrent access error codes recorded in the memory controller, converts these codes into Boolean flags, and determines that a memory access contention flag exists when the Boolean flag is true, thus achieving accurate detection of memory-level concurrent access conflicts. Compared with related technologies that cannot obtain the memory access contention status at the hardware level, this embodiment utilizes the concurrent access error codes recorded by the memory controller hardware to convert hardware-level error states into memory access contention flags recognizable by the scheduling mechanism. This provides a precise spatial dimension input for quantifying the conflict intensity, enabling advanced control of computing resources.

[0095] In one embodiment, step S100 may further include: obtaining historical transaction logs within the state change interval, and classifying the historical transaction logs using a random forest algorithm to obtain a conflict classification set.

[0096] The Random Forest algorithm improves classification accuracy by constructing multiple decision trees and performing a voting ensemble. The classification result of a Random Forest can be represented as:

[0097] in, For the final classification result, For the first The prediction results of the decision tree The total number of decision trees.

[0098] Different resource lock waiting time accumulation parameters are assigned to different conflict types (mild, moderate, and severe) within the conflict classification set. For example, a smaller initial waiting time is used for mild conflicts, while a larger initial waiting time is used for severe conflicts. By pre-classifying conflict types using a random forest algorithm, the calculation of resource lock waiting time accumulation is made more accurate, avoiding the use of the same exponential backoff parameter for all conflicts, thereby improving the quantitative accuracy of resource contention between heterogeneous tasks.

[0099] In one embodiment, see Figure 4 Step S120 includes: Step S121: Collect the number of context switches for each thread.

[0100] Step S122: Extract the involuntary context switch count from the context switch count. The involuntary context switch count represents the number of times a thread is forced to switch due to time slice exhaustion or preemption by a higher priority thread.

[0101] Step S123: Compare the number of involuntary context switches of each thread with the preset baseline number, and filter out the threads whose number of involuntary context switches exceeds the preset baseline number as threads with abnormal computing resource preemption frequency.

[0102] In step S121, the context switch count refers to the total number of times the operating system kernel switches the CPU from the current thread to another thread per unit time, which can be obtained by reading the kernel-mode context switch counter.

[0103] In step S122, the involuntary context switch count refers to the number of times a thread is forced out due to time slice exhaustion or preemption by a higher-priority thread. A voluntary context switch refers to a switch caused by a thread actively relinquishing the CPU (e.g., waiting for I / O or actively sleeping). Involuntary and voluntary context switches together constitute the total number of context switches. The involuntary context switch count can be extracted from the context switch counter, for example, by reading relevant fields from the scheduling statistics maintained by the kernel for each thread.

[0104] In step S123, the preset baseline number of times can be adjusted based on the historical statistical values ​​of threads under normal load. For example, the statistical distribution of the number of involuntary context switches of each thread in the computing power pool under normal operation is collected, and the 75th percentile is used as the basis for adjusting the preset baseline number of times. In this embodiment, the preset baseline number of times is set to 45 times.

[0105] The number of involuntary context switches for each thread is compared with a preset baseline. If a thread's involuntary context switch count exceeds the preset baseline, it is considered a thread with an involuntary context switch count exceeding the preset baseline; otherwise, it is considered a normal thread. All threads with an involuntary context switch count exceeding the preset baseline are filtered out and recorded as threads with abnormal computing resource preemption frequency.

[0106] This embodiment collects the context switch count of each thread, extracts the involuntary context switch count, and compares the involuntary context switch count of each thread with a preset benchmark count. Threads with an involuntary context switch count exceeding the preset benchmark count are identified as threads with abnormal computing resource preemption frequency. Compared with related technologies that use coarse-grained indicators such as global CPU utilization or average waiting time to locate problematic threads, this embodiment accurately identifies threads that are forced out due to time slice exhaustion or preemption by higher-priority threads by using the involuntary context switch count. This focuses scheduling operations on threads that truly have resource contention issues, avoiding indiscriminate scheduling that interferes with normal threads and achieving advanced control over computing resources.

[0107] In one embodiment, see Figure 5 Step S140 includes: Step S141: Obtain the dynamic priority weight and transaction isolation level identifier of each thread in the first thread set. The dynamic priority weight is used to represent the dynamic priority of the thread in the computing power competition, and the transaction isolation level identifier is used to represent the strictness of concurrency control of the corresponding transaction of the thread.

[0108] Step S142: Calculate the weight value of each thread based on the priority dynamic weight and the transaction isolation level identifier.

[0109] Step S143: Determine the weight percentage of each thread based on the proportion of each thread's weight value to the total weight of the first thread set.

[0110] In step S141, the priority dynamic weight refers to the dynamic priority of a thread in the computing power competition, which can be calculated based on the thread's virtual runtime parameters using an exponential moving average model.

[0111] For details, please refer to Figure 6 Step S141 includes: Step S1411: Read the virtual runtime parameter of each thread. The virtual runtime parameter is used to represent the cumulative time that the thread occupies the processor.

[0112] Step S1412: Substitute the virtual running time parameter into the preset exponential moving average model to calculate the priority dynamic weight.

[0113] In step S1411, the virtual runtime parameter refers to the cumulative metric used by the operating system to record the processor time occupied by a thread. The larger the value, the more execution time the thread has gained.

[0114] In step S1412, exemplarily, the exponential moving average model is a time series forecasting method that smooths data fluctuations by assigning higher weights to recent data. In an embodiment, the exponential moving average model is used to calculate priority dynamic weights based on the thread's virtual runtime parameter.

[0115] In the exponential moving average model, the input is the virtual running time parameter at each time step. This parameter represents the cumulative time a thread occupies the processor, and is maintained by the operating system kernel for each thread. The parameters of the exponential moving average model include the decay factor. The value ranges from 0 to 1 and is used to control the degree of influence of historical data on the current weight. The larger the value, the greater the impact of the current observation on the weight, and the faster the weight responds to changes in load; The smaller the value, the greater the influence of historical data on the weights, and the smoother the weighting. (Attenuation factor) The priority weight can be adjusted based on the system's sensitivity to changes in thread priority; in this embodiment, it is set to 0.8. The initial dynamic priority weight... It can be set to the thread's initial priority value or a preset initial weight value.

[0116] Furthermore, the calculation formula for the exponential moving average model is as follows:

[0117] in, The current priority is dynamically weighted. This is the current virtual runtime parameter. The priority dynamic weight of the previous time step. This is the attenuation factor.

[0118] As an example, in the preset exponential moving average model, the priority dynamic weights of the previous time step... Current virtual runtime parameters Attenuation factor The current priority dynamic weight .

[0119] The transaction isolation level identifier is used to indicate the strictness of concurrency control for the corresponding transaction of a thread. It includes three isolation levels: Read Committed, Repeatable Read, and Serializable. Optionally, the three isolation levels correspond to the values ​​2, 3, and 4, respectively.

[0120] Alternatively, other algorithms such as Isolation Forest can be used to perform anomaly detection on the dynamic weights of thread priorities and transaction isolation level identifiers, resulting in an abnormal thread classification matrix to assist in filtering threads with abnormal computing resource preemption frequencies. Those skilled in the art can choose to use these algorithms according to actual needs; no limitations are imposed here.

[0121] In step S142, the weight value of each thread can be obtained by multiplying the dynamic priority weight by the value of the transaction isolation level identifier. As an example, if a thread has a dynamic priority weight of 116 and a transaction isolation level identifier of 3, then the weight value of the thread is 116 × 3 = 348.

[0122] In step S143, the total weight of the first thread set refers to the sum of the weight values ​​of all threads within the first thread set. The weight percentage of each thread is equal to the weight value of that thread divided by the total weight of the first thread set.

[0123] The formula for calculating the weighting percentage is:

[0124] in, For the first The weight percentage of each thread For the first The weight value of each thread. This represents the total number of threads in the first thread set.

[0125] As an example, the weights of the two threads in the first thread set are 348 and 75, respectively, with a total weight of 423. Therefore, the weight ratios of the two threads are approximately 0.823 (348 / 423) and 0.177 (75 / 423).

[0126] This embodiment obtains the dynamic priority weights and transaction isolation level identifiers of each thread in the first thread set. It calculates the weight value of each thread based on these factors and determines the weight percentage of each thread based on its proportion to the total weights of the first thread set. Compared to related technologies that use static priorities or simple priority queues for weight determination, this embodiment integrates the dynamic priority of threads with the strictness of transaction concurrency control. This allows the weight percentage to reflect the actual priority of threads in computing power competition in real time, providing a precise weight basis for subsequent differentiated time slice allocation and achieving advanced control over computing resources.

[0127] In one embodiment, see Figure 7 Step S140 includes: Step S144: Calculate the time slice length of each thread based on the weight ratio of each thread and the preset total time slice length, and schedule each thread to execute sequentially according to the time slice length.

[0128] Step S145: In response to the current thread's execution time exceeding the corresponding time slice length, switch the current thread to the next thread.

[0129] In step S144, the preset total time slice length refers to the sum of the execution times of all threads in a single round of scheduling. This can be adjusted according to the total throughput requirements of the computing power pool. In this embodiment, the preset total time slice length is exemplarily set to 100 milliseconds. The time slice length of each thread is equal to the preset total time slice length multiplied by the weight ratio of that thread. The calculation formula is as follows:

[0130] in, For the first The time slice length of each thread The preset total time slice length, For the first The weight percentage of each thread.

[0131] As an example, the preset total time slice length In milliseconds, the weights of the two threads are 0.823 and 0.177 respectively, so the time slice lengths of the two threads are respectively... milliseconds and Milliseconds. Each thread is scheduled to execute sequentially according to its time slice length; that is, the first thread is scheduled to execute for 82.3 milliseconds, and then the second thread is scheduled to execute for 17.7 milliseconds.

[0132] In step S145, the cumulative execution time of the current thread is monitored. When the cumulative execution time of the current thread exceeds the time slice allocated to the thread, the thread is forcibly switched out of the computing core, and the next thread is scheduled to start execution. As an example, the time slice length of the current thread is 82.3 milliseconds. When the cumulative execution time of the thread reaches 82.3 milliseconds, a context switch is triggered, the current thread is moved to the tail of the ready queue, and the next thread is selected from the ready queue for execution.

[0133] This embodiment calculates the time slice length for each thread based on its weight and a preset total time slice length, and schedules each thread to execute sequentially according to its time slice length. When the execution time of the current thread exceeds its corresponding time slice length, the current thread is switched to the next thread. Compared to related technologies that use the same time slice length for all threads or simple priority queues, this embodiment makes the time slice length proportional to the thread's weight. Short-cycle tasks with higher weights receive longer time slices, thus gaining more execution opportunities per unit time. Simultaneously, time slice rotation ensures that long-cycle tasks still receive execution time, achieving advanced control over computing resources.

[0134] In one embodiment, see Figure 8 Step S160 includes: Step S161: Use the sliding window algorithm to scan the state transition path window by window.

[0135] Step S162: In response to the situation where at least two threads have conflicting operation order on the same data object, mark the position of the corresponding window as the transaction commit order conflict position.

[0136] In step S161, the sliding window algorithm refers to a method that sets a fixed-length time window, scans the state transition path sequence window by window, and performs conflict detection within each window. The window length can be adjusted according to the average execution time of the transaction. For example, twice the average execution time of the transaction can be used as the basis for adjusting the window length. In this embodiment, the window length is exemplarily set to 50 state change records.

[0137] In step S162, an operation order conflict refers to a situation where the order in which at least two threads perform read and write operations on the same data object is inconsistent with the order required by the serializable scheduling. For example, if thread A performs a write operation on the account balance field at timestamp 120.5, and thread B performs a write operation on the same account balance field at timestamp 118.2, and the execution order of the two operations is inconsistent with the commit order, then an operation order conflict is determined to exist. The location of the window where the operation order conflict is detected is marked as the transaction commit order conflict location, which is used for subsequent lock granularity refinement and timing adjustment. Specifically, timestamp 120.5 can be extracted as the transaction commit order conflict marker location.

[0138] This embodiment employs a sliding window algorithm to scan the state transition path window by window. In response to situations where at least two threads have conflicting operation sequences on the same data object, the corresponding window location is marked as the transaction commit order conflict location. Compared to related technologies that cannot accurately locate transaction conflict locations and can only use heavy-duty operations such as global locks or transaction rollbacks, this embodiment can accurately mark the node location where the conflict occurs in the state transition path. This provides precise operation objects for subsequent lock granularity refinement and timing adjustment, achieving advanced control over computing resources.

[0139] In one embodiment, see Figure 9 Step S160 includes: Step S163: Locate the conflicting data row based on the conflict position in the transaction commit order.

[0140] Step S164: Calculate the hash value of the conflicting data row.

[0141] Step S165: Calculate the hash collision rate of the conflicting data rows based on their hash values.

[0142] Step S166: In response to the hash collision rate exceeding the preset hash threshold, refine the lock granularity of the conflicting data row from the current level to the next level.

[0143] In step S163, a conflicting data row refers to a data row in the database corresponding to the position where the transaction commit order conflicts. Based on the transaction commit order conflict positions marked in step S162, the transaction identifier and operation object identifier associated with the conflict position are extracted. By querying the database's metadata information, the specific data row where the conflict occurred is located.

[0144] As an example, the transaction commit order conflict location record records the identifiers of the two conflicting transactions, TxnA and TxnB, and the operation object is "account balance table". The specific row record can be located by querying the database metadata, such as the data row in the account balance table with primary key 1001.

[0145] In step S164, the hash value refers to the numerical value obtained by hashing the primary key or unique identifier of the conflicting data row. The hash algorithm can use common hash functions, such as mapping the primary key value of a data row to a fixed-length hash code. For example, a modulo hash function:

[0146] in, For the primary key value of the conflicting data row, Modulus, representing the remainder operation. The modulus can be adjusted based on the bucket size of the hash table. For example, the bucket size can be used as the basis for setting the modulus. In this embodiment, the modulus is set to 1024. As an example, the primary key value of a conflicting data row is 1001, and the modulus... The hash value is This means that the remainder when 1001 is divided by 1024 is 1001.

[0147] In step S165, the hash collision rate refers to the proportion of multiple different data rows that are mapped to the same hash value.

[0148] The calculation formula is:

[0149] in, For hash collision rate, The number of data rows that have hash collisions. This represents the total number of data rows. As an example, the total number of data rows... The number of data rows that have hash collisions Then the hash collision rate .

[0150] In step S166, the preset hash threshold can be adjusted according to the system's tolerance for lock conflicts. For example, the statistical distribution of hash conflict rate under normal operating conditions can be collected, and the 95th percentile can be used as the basis for setting the preset hash threshold. In this embodiment, the preset hash threshold is set to 5% for example.

[0151] When the hash collision rate exceeds a preset hash threshold, the lock granularity for conflicting data rows is refined from the current level to the next level. Lock granularity levels include table-level locks, row-level locks, and field-level locks.

[0152] Optionally, the refinement method is as follows: If the current lock granularity is a table-level lock, then refine the lock granularity to a row-level lock, that is, narrow the locking scope from the entire table to only locking conflicting data rows; if the current lock granularity is a row-level lock, then further refine the lock granularity to a field-level lock, that is, narrow the locking scope from the entire row of data to only locking the conflicting field column. As an example, if the current lock granularity is a table-level lock and the hash collision rate is 7.5%, exceeding the preset hash threshold of 5%, then refine the lock granularity from a table-level lock to a row-level lock, narrowing the locking scope from the entire "Account Balance Table" to only locking the data row with primary key 1001. If the hash collision rate still exceeds the preset hash threshold after refinement, then continue to refine the lock granularity from a row-level lock to a field-level lock, narrowing the locking scope from the entire row of data to only locking the "Balance" field.

[0153] Optionally, after the lock granularity is refined, a directed acyclic graph (DAG) analysis algorithm is invoked to traverse the underlying cache nodes and identify the propagation path of cache failure. A DAG is a graph structure where no cyclic paths exist, often used to represent a sequence of nodes with dependencies. In this embodiment, the dependencies between cache nodes form a DAG, where each cache node is a vertex. If cache node A fails, causing cache node B to fail, then there exists a directed edge from A to B.

[0154] The steps of the directed acyclic graph (DAG) analysis algorithm are as follows: Starting from a conflicting cache node, traverse all outgoing edges using either depth-first or breadth-first search, record all reachable downstream nodes, and output the propagation path and the number of downstream nodes. For example, if the failure of cache node A causes B to fail, B's failure causes C to fail, and C's failure causes D to fail, then the propagation path is A→B→C→D, and the number of downstream nodes is 3. When the lock granularity is refined, the propagation path of downstream cache node failures that would originally be triggered by transaction rollbacks is blocked. For example, originally 3 downstream cache nodes would fail due to transaction rollbacks; after lock granularity refinement, the failure propagation path of these 3 nodes is blocked, and the cache failure chain reaction count decreases from 3 to 0.

[0155] This embodiment locates conflicting data rows based on the conflict position in the transaction commit order, calculates the hash value of the conflicting data rows, and calculates the hash collision rate. In response to a hash collision rate exceeding a preset hash threshold, the lock granularity of the conflicting data rows is refined from the current level to the next level. Compared to related technologies where the lock granularity is fixed (such as table-level locks) and cannot be dynamically adjusted according to the degree of conflict, this embodiment quantifies the degree of conflict of data rows through the hash collision rate and refines the lock granularity level by level according to the degree of conflict, from table-level locks to row-level locks or field-level locks. This allows the lock granularity to adaptively match the degree of conflict, thereby preventing the chain reaction of cache invalidation caused by lock conflicts and achieving advanced control over computing resources.

[0156] In one embodiment, see Figure 10 It provides an intelligent scheduling system based on computing power pooling technology, including: The conflict index acquisition module is used to obtain the cumulative value of resource lock waiting time in the computing power pool. The cumulative value of resource lock waiting time is used to characterize the degree of resource contention between heterogeneous tasks.

[0157] The abnormal thread extraction module is used to extract threads with abnormal computing resource preemption frequency from active threads in response to the cumulative value of resource lock waiting time exceeding a preset cumulative time threshold, forming a first thread set.

[0158] The differentiated scheduling module is used to determine the time slice length allocated to each thread based on the weight ratio of each thread in the first thread set, and to divide the execution in a round-robin manner to form the second thread set and the corresponding state transition path. The weight ratio is used to represent the priority of the thread in the allocation of computing resources, and the time slice length is used to represent the maximum execution time quota that the thread is allowed to occupy continuously.

[0159] The conflict resolution module is used to locate the conflicting transaction commit order in the state transition path, determine the lock granularity level of the conflicting thread, and adjust the transaction execution sequence to eliminate the conflict and obtain the updated state transition path.

[0160] The deadlock prediction module is used to predict the deadlock probability of each thread based on the updated state transition path and the instruction queue depth of each thread in the second thread set. The instruction queue depth is used to represent the number of instructions to be executed by the thread.

[0161] The resource allocation module is used to generate resource allocation instructions in response to a deadlock probability exceeding a preset risk threshold, based on the lock granularity level and time slice length of the corresponding thread. The resource allocation instructions are used to indicate the priority preemption right of short-cycle tasks for computing resources in heterogeneous tasks.

[0162] The specific implementation of each module can be found in the previous description of the method, and will not be repeated here.

[0163] This application also provides an electronic device, which includes one or more processors and memory resources represented by memory for storing instructions executable by the processor, such as application programs. The application programs stored in the memory may include one or more modules, each corresponding to a set of instructions.

[0164] The electronic device may also include a power supply component configured to perform power management of the electronic device, a wired or wireless network interface configured to connect the electronic device to a network, and an input / output (I / O) interface. The electronic device can be operated based on operating devices stored in memory, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or similar.

[0165] In one embodiment, a computer device, which may be a server, is also provided. The computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is connected to the system bus via the I / O interfaces. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device stores data. The I / O interfaces of the computer device are used for exchanging information between the processor and external devices. The communication interface of the computer device is used for communicating with external terminals via a network connection.

[0166] In one embodiment, a computer device is provided, which may be a terminal. The computer device includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface of the computer device is used for exchanging information between the processor and external devices. The communication interface of the computer device is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. The display unit of the computer device is used to form a visually visible image and may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0167] This application also provides a non-transitory computer-readable storage medium, which, when the instructions in the storage medium are executed by the processor of the electronic device, enables the electronic device to perform one of the above embodiments.

[0168] This application may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-readable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information may be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to: phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0169] It should be noted that although the steps of the method in this application are described in a specific order in the accompanying drawings, this does not require or imply that these steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps, such as omitting certain steps, combining multiple steps into one step, and / or breaking down one step into multiple steps, should all be considered part of this application.

[0170] It should be understood that this application is not limited to the detailed data of the methods presented in this specification. This application is capable of other implementations and can be implemented and performed in various ways. The foregoing variations and modifications fall within the scope of this application. It should be understood that the disclosure and definition of this application extend to all alternative combinations of two or more individual features mentioned or apparent in the text and / or drawings. All these different combinations constitute multiple alternative aspects of this application. The embodiments described in this specification illustrate the best known mode for implementing this application and will enable those skilled in the art to utilize this application.

Claims

1. An intelligent scheduling method based on computing power pooling technology, characterized in that, include: Obtain the cumulative value of resource lock waiting time in the computing power pool. The cumulative value of resource lock waiting time is used to characterize the degree of resource contention among heterogeneous tasks. In response to the resource lock waiting time accumulation value exceeding a preset time accumulation threshold, threads with abnormal computing resource preemption frequency are extracted from active threads to form a first thread set; Based on the weight ratio of each thread in the first thread set, the time slice length is determined for each thread, and the execution is divided in a time slice round-robin manner to form a second thread set and the corresponding state transition path. The weight ratio is used to represent the priority of the thread in the allocation of computing resources, and the time slice length is used to represent the maximum execution time quota that the thread is allowed to occupy continuously. Find the transaction commit order conflict position in the state transition path, determine the lock granularity level of the conflicting thread at the transaction commit order conflict position, and adjust the transaction execution sequence to eliminate the conflict, thus obtaining the updated state transition path. Based on the updated state transition path and the instruction queue depth of each thread in the second thread set, the deadlock probability of each thread is predicted, whereby the instruction queue depth is used to represent the number of instructions to be executed by the thread. In response to the deadlock probability exceeding a preset risk threshold, a resource allocation instruction is generated based on the lock granularity level and time slice length of the corresponding thread. The resource allocation instruction is used to indicate the priority preemption right of short-cycle tasks for computing resources in heterogeneous tasks.

2. The method according to claim 1, characterized in that, The process of obtaining the accumulated resource lock waiting time value of the computing power pool includes: The bandwidth utilization rate of disk input and output is collected at a preset sampling frequency, and the memory access contention flag is read. The memory access contention flag is used to indicate whether the memory controller has detected a concurrent access conflict. A sliding window algorithm is used to perform time-series analysis on the collected bandwidth utilization rates to identify the time window overlap metric of transaction tasks and migration tasks. The time window overlap metric is used to represent the proportion of time that transaction tasks and migration tasks are executed concurrently within the same time period. In response to the time window overlap metric value being greater than a preset threshold, and the memory access contention being identified as conflicting, the single conflict wait time is calculated according to an exponentially increasing rule. The conflict waiting time is accumulated to obtain the cumulative value of the resource lock waiting time.

3. The method according to claim 2, characterized in that, The memory access contention flag includes: Read the concurrent access error codes recorded in the memory controller; Convert the concurrent access error code to a Boolean identifier; In response to the Boolean flag being true, the memory access contention flag is determined to be in conflict.

4. The method according to claim 1, characterized in that, The extraction of threads with abnormally high computing resource preemption frequency from active threads includes: Collect the number of context switches for each thread; Extract the number of involuntary context switches from the context switch count, where the involuntary context switch is used to represent the number of times a thread is forced to switch due to time slice exhaustion or preemption by a higher priority thread; The number of involuntary context switches for each thread is compared with a preset baseline number, and threads whose number of involuntary context switches exceeds the preset baseline number are selected as threads with abnormal computing resource preemption frequency.

5. The method according to claim 1, characterized in that, Before determining the time slice length for each thread based on the weight ratio of each thread in the first thread set, the following steps are included: Obtain the dynamic priority weight and transaction isolation level identifier of each thread in the first thread set. The dynamic priority weight is used to represent the dynamic priority of the thread in the computing power competition, and the transaction isolation level identifier is used to represent the strictness of concurrency control of the corresponding transaction of the thread. Calculate the weight value for each thread based on the priority dynamic weight and the transaction isolation level identifier; The weight percentage of each thread is determined based on the proportion of each thread's weight value to the total weight of the first thread set.

6. The method according to claim 5, characterized in that, The step of obtaining the dynamic priority weights of each thread in the first thread set includes: Read the virtual runtime parameter of each thread, which represents the cumulative time the thread occupies the processor; The virtual runtime parameters are substituted into a preset exponential moving average model to calculate the priority dynamic weight.

7. The method according to claim 1, characterized in that, The execution is divided into time-slice rotations, including: Based on the weight ratio of each thread and the preset total time slice length, the time slice length of each thread is calculated, and each thread is scheduled to execute sequentially according to the time slice length. In response to the current thread's execution time exceeding the corresponding time slice length, the current thread is switched to the next thread.

8. The method according to claim 1, characterized in that, The step of finding the transaction commit order conflict location in the state transition path includes: The state transition path is scanned window by window using a sliding window algorithm; In response to a situation where at least two threads have conflicting operation orders on the same data object, the corresponding window location is marked as the transaction commit order conflict location.

9. The method according to claim 8, characterized in that, The lock granularity level for determining the conflicting threads at the transaction commit order conflict positions includes: Locate the conflicting data row based on the conflict position in the transaction commit order; Calculate the hash value of the conflicting data row; Based on the hash value of the conflicting data row, calculate the hash collision rate of the conflicting data row; In response to the hash collision rate exceeding a preset hash threshold, the lock granularity of the conflicting data row is refined from the current level to the next level.

10. An intelligent scheduling system based on computing power pooling technology, characterized in that, include: The conflict index acquisition module is used to acquire the cumulative value of resource lock waiting time in the computing power pool. The cumulative value of resource lock waiting time is used to characterize the degree of resource contention between heterogeneous tasks. The abnormal thread extraction module is used to extract threads with abnormal computing resource preemption frequency from active threads in response to the resource lock waiting time accumulation value exceeding a preset time accumulation threshold, forming a first thread set. The differentiated scheduling module is used to determine the time slice length allocated to each thread based on the weight ratio of each thread in the first thread set, and to divide the execution in a time slice round-robin manner to form a second thread set and the corresponding state transition path. The weight ratio is used to represent the priority of the thread in the allocation of computing resources, and the time slice length is used to represent the maximum execution time quota that the thread is allowed to occupy continuously. The conflict resolution module is used to find the transaction commit order conflict position in the state transition path, determine the lock granularity level of the conflicting thread at the transaction commit order conflict position, and adjust the transaction execution sequence to eliminate the conflict and obtain the updated state transition path. The deadlock prediction module is used to predict the deadlock probability of each thread based on the updated state transition path and the instruction queue depth of each thread in the second thread set. The instruction queue depth is used to represent the number of instructions to be executed by the thread. The resource allocation module is used to generate a resource allocation instruction in response to the deadlock probability exceeding a preset risk threshold, based on the lock granularity level and time slice length of the corresponding thread. The resource allocation instruction is used to indicate the priority preemption right of short-cycle tasks for computing resources in heterogeneous tasks.

Images