A method and system for optimizing joint resource allocation and task offloading

Abstract

This application relates to the technical field of computing resource optimization, and discloses a joint resource allocation and task offloading optimization method and system. The method includes: constructing a global resource view to perceive the real-time residency status of cache lines on each computing node in a distributed system; establishing a cache-aware joint optimization model for task offloading and data prefetching, whose decision variables simultaneously include task offloading variables and data prefetching variables; and endogenously constructing cache capacity constraints using a task offloading scheme to predict the set of cache lines to be evicted due to task execution; solving the joint optimization model with the objective of minimizing the total task completion time, and simultaneously generating task offloading decisions and collaborative prefetching decisions, wherein the prefetching decision is determined based on the predicted eviction set and task data dependencies. This invention solves the problem of mutual interference caused by independent optimization of task offloading and data prefetching within a unified optimization framework, and achieves global proactive orchestration of computing and cache resources.

Description

Technical Field

[0001] This application relates to the technical field of computing resource optimization, and discloses a method and system for optimizing joint resource allocation and task offloading. Background Technology

[0002] As the scale and complexity of distributed computing systems continue to grow, the decoupling of computing tasks from data has become the norm. To minimize task execution time, task offloading techniques are used to schedule computing tasks to nodes where the data resides or where access costs are lower. To reduce data access latency, cache prefetching techniques are widely used to preload data that will be needed in the future into the cache.

[0003] In existing technologies, task offloading optimization and cache prefetching optimization are typically designed as two independent and sequential processes. This separate optimization model has the following problems: When adjusting computational and data locality, task offloading decisions dynamically change the cache content of each node, thereby interfering with cache prefetching strategies based on historical statistics or static task graphs, leading to decreased prefetching accuracy and even cache pollution. Simultaneously, independent prefetching decisions cannot anticipate cache state changes caused by task offloading, making it difficult to effectively compensate for the negative impact of cache line eviction on the performance of subsequent tasks. The task offloader and cache prefetcher have conflicting optimization objectives; their independent operation creates performance bottlenecks due to localized optimization conflicts, restricting overall system performance. Summary of the Invention

[0004] The purpose of this section is to outline some aspects of the embodiments of this application and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents, and such simplifications or omissions should not be construed as limiting the scope of this application.

[0005] To address the aforementioned technical problems, this application provides a method and system for optimizing joint resource allocation and task unloading.

[0006] On the one hand, this application provides a joint resource allocation and task offloading optimization method, including S1 constructing a global resource view to perceive the real-time residence status of cache lines of each node in the distributed system;

[0007] S2 establishes a cache-aware joint optimization model for task offloading and data prefetching. The decision variables of the joint optimization model include a first type of variable representing task offloading to each node and a second type of variable representing data block prefetching to each node's cache. The cache capacity constraint of the joint optimization model is generated by the task offloading scheme determined by the first type of variable, and the set of cache lines to be evicted in each node's cache is intrinsically calculated and predicted.

[0008] S3 solves the joint optimization model with the goal of minimizing the total task completion time, and simultaneously generates task unloading decisions and collaborative prefetching decisions. The collaborative prefetching decisions are determined based on the predicted set of cache lines to be evicted and the data dependencies between tasks.

[0009] As a preferred embodiment of the joint resource allocation and task unloading optimization method of this application, wherein:

[0010] The construction of the global resource view includes: deploying a monitoring probe on each computing node, wherein the monitoring probe periodically collects and reports the hardware performance counter values ​​of the node through the system network, and the hardware performance counters include counters reflecting cache hit and miss events at all levels;

[0011] Based on the counter value sequence reported by each node, the real-time residence probability or recent access time tag of each node's cache line is calculated and maintained through predefined mapping rules.

[0012] As a preferred embodiment of the joint resource allocation and task unloading optimization method of this application, wherein:

[0013] The construction of the global resource view also includes: deploying a listening agent on the interconnection network of the distributed system to capture messages transmitted between nodes that conform to the cache consistency protocol;

[0014] Parse the message's type and address fields, where the message type includes read request, write-back, and invalidation notification;

[0015] Based on the predefined protocol state machine logic and the parsing result of the message, the state of the corresponding cache line in the global resource view is inferred and updated. The state includes exclusive, shared, modified, or invalid.

[0016] As a preferred embodiment of the joint resource allocation and task unloading optimization method of this application, wherein:

[0017] The cache capacity constraint of the joint optimization model is constructed through dynamic judgment and prediction within the joint optimization model.

[0018] Based on the task unloading scheme determined by the first type of variables, calculate the set of new data blocks that must be loaded into the cache of each node due to the execution of these tasks and their total amount;

[0019] The total amount is compared with the current free capacity of each node cache obtained through the global resource view. If the total amount of new data exceeds the free capacity, the joint optimization model will automatically identify and determine a set of data blocks to be removed from the currently resident data according to the preset cache replacement strategy during the solution process.

[0020] The set of data blocks to be removed is the set of cache lines that are predicted to be evicted.

[0021] As a preferred embodiment of the joint resource allocation and task unloading optimization method of this application, wherein:

[0022] The preset cache replacement strategy is a strategy based on access history.

[0023] Information reflecting the historical access status of cache lines is derived from the global resource view, and the type of information is the most recent access timestamp or historical access frequency.

[0024] The set of cache lines to be evicted is determined by comparing the historical access information corresponding to each data block in the currently resident data block set, and sorting and filtering the historical access information according to the principle of least recently used.

[0025] As a preferred embodiment of the joint resource allocation and task unloading optimization method of this application, the joint optimization model takes minimizing the total task completion time of the system as the optimization objective;

[0026] The optimization objective outputs a target value, the calculation of which includes: the theoretical execution time of the task on the selected computing node; the calculation of the target value also includes the expected latency caused by the task accessing its dependent data;

[0027] The expected latency is calculated as follows: under the combined effect of the task unloading decision and the data prefetching decision, the final expected physical location of each data block is quantified. The location is divided into local cache hit, local memory hit, and cross-node remote memory access, and is weighted and accumulated differently.

[0028] As a preferred embodiment of the joint resource allocation and task unloading optimization method of this application, wherein:

[0029] The joint optimization model is solved using a decomposition and coordination algorithm.

[0030] The decomposition and coordination algorithm first decomposes the original problem, which couples task unloading variables and data prefetching variables, into task unloading sub-problems and data prefetching sub-problems along the resource dimension. By introducing coordination variables to reflect the tension of shared resources such as cache capacity, the algorithm correlates the sub-problems with the tension of shared resources through coordination variables. Through iterative calculation, each sub-problem is solved independently with the current coordination variable value.

[0031] The main problem updates the coordination variables and the estimate of the predicted cache line eviction set based on the solutions to the subproblems, thereby guiding the next iteration until convergence to a consistent joint optimization solution.

[0032] As a preferred embodiment of the joint resource allocation and task unloading optimization method of this application, wherein:

[0033] The task scheduler and the data prefetcher share the same solution vector output by the joint optimization model;

[0034] The task scheduler generates the task unloading decision based on the assignment results of the first type of variables in the solution vector; the data prefetcher generates the collaborative prefetch decision based on the assignment results of the second type of variables in the solution vector and in combination with the cache vacancy positions corresponding to the predicted set of cache lines to be evicted.

[0035] The assignment of the second type of variable implicitly includes the data dependency relationship between tasks and the dual constraint of the cache vacancy position.

[0036] As a preferred embodiment of the joint resource allocation and task unloading optimization method of this application, wherein:

[0037] Based on a preset task dependency graph, identify subsequent tasks on the critical path and the data blocks they depend on; compare the set of cache lines predicted to be evicted with the set of data blocks depended on by the subsequent tasks on the critical path;

[0038] Data blocks identified during the comparison as being invalid due to eviction and simultaneously dependent on the critical path task are prefetched instructions.

[0039] This application provides a joint resource allocation and task offloading optimization system, including:

[0040] The global resource view senses the real-time residency status of cache lines on each computing node in the distributed computing system.

[0041] The joint optimization module, which is communicatively connected to the global resource view construction module, includes a computing unit, a first unit, and a decision unit. The computing unit establishes a cache-aware joint optimization model for task offloading and data prefetching. The decision variables of the joint optimization model include both first-type variables representing task offloading to each node and second-type variables representing data block prefetching to the cache of each node.

[0042] The first unit generates the cache capacity constraint of the joint optimization model based on the task unloading scheme determined by the first type of variables, and intrinsically calculates and predicts the set of cache lines that will be evicted in the cache of each node.

[0043] The decision-making unit solves the joint optimization model with the objective of minimizing the total task completion time, and simultaneously outputs task unloading decisions and collaborative prefetching decisions.

[0044] The task coordination module is communicatively connected to the joint optimization module. Based on the task unloading decision, it schedules the computational tasks to the corresponding nodes for execution. Based on the collaborative prefetch decision, the predicted set of cache lines to be evicted and the data dependencies between tasks, it initiates a data prefetch operation on the corresponding nodes.

[0045] The beneficial effects of this application are as follows:

[0046] This application constructs a joint optimization model for task offloading and data prefetching. In this model, the impact of offloading schemes on cache state is intrinsically predicted, merging the two processes of task offloading and data prefetching into a unified global optimization problem. This scheme avoids local optima and mutual interference problems caused by independent optimization, and achieves coordinated optimization of offloading and prefetching decisions.

[0047] This application enables prefetching decisions to directly respond to cache state changes caused by the current task unloading decision through joint optimization of intra-model prediction and synchronous generation, thereby filling cache gaps that will be generated due to task unloading and preventing critical data from being evicted in advance.

[0048] This application unifies the decision-making of computing task scheduling and data cache management within the same optimization cycle, realizing the coordinated scheduling of multi-dimensional resources such as computing units, memory bandwidth, cache space and network resources. By using a joint optimization model to jointly schedule and prefetch the execution location of tasks and the flow trajectory of data, the overall efficiency and determinism of resource utilization are improved.

[0049] This application enables rapid decision-making based on a real-time global resource view through a joint optimization model, allowing the system to quickly adjust joint strategies when facing dynamically changing task loads and node states, and always maintain the ability to accurately predict and compensate for cache states. Attached Figure Description

[0050] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained through these drawings without creative effort. Wherein:

[0051] Figure 1 A schematic diagram of a joint resource allocation and task unloading optimization method provided in this application;

[0052] Figure 2 A flowchart illustrating the global resource view construction process in a joint resource allocation and task unloading optimization method provided in this application;

[0053] Figure 3A flowchart illustrating the construction and solution of the joint optimization model in the joint resource allocation and task unloading optimization method provided in this application;

[0054] Figure 4 A flowchart illustrating the collaborative execution of task scheduling and data prefetching in a joint resource allocation and task offloading optimization method provided in this application;

[0055] Figure 5 This application provides a module interaction and data flow diagram for a joint resource allocation and task unloading optimization system. Detailed Implementation

[0056] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the specific embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0057] Many specific details are set forth in the following description in order to provide a full understanding of this application. However, this application may also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0058] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of this application. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single embodiment or an embodiment selectively excluded from other embodiments.

[0059] Example 1

[0060] like Figure 1 As shown, a joint resource allocation and task offloading optimization method aims to improve system resource utilization and reduce task latency, including:

[0061] S1 constructs a global resource view to perceive the real-time residency status of cache lines on each node in the distributed system;

[0062] like Figure 2 As shown, the construction of the global resource view includes: deploying a monitoring probe on each computing node, the monitoring probe periodically collecting and reporting the hardware performance counter values ​​of the node through the system network, the hardware performance counters including counters reflecting cache hit and miss events at all levels;

[0063] Based on the counter value sequence reported by each node, the real-time residence probability or recent access time tag of each node's cache line is calculated and maintained through predefined mapping rules.

[0064] A preferred example: The global resource view is implemented through the collaboration of fine-grained monitoring units deployed on each computing node and protocol listening units deployed on the interconnection network. On each computing node in the distributed system, a lightweight performance monitoring probe is deployed. The monitoring probe runs in the form of a kernel module or a user space background process and is used to periodically collect the hardware performance counter values ​​of the computing node.

[0065] The monitoring probe collects counter values ​​of access behavior events that directly or indirectly reflect the cache levels, such as L1, L2, and LastLevelCache. These events include at least cache read hits, cache read misses, cache write hits, and cache write misses. By monitoring these events, the cache access patterns and efficiency can be inferred.

[0066] After collecting the counter values, the monitoring probe reports the collected data sequence to a centralized or logically centralized resource view manager through the existing internal network of the distributed system.

[0067] The resource view manager receives a sequence of performance counter values ​​from all compute nodes. The manager uses a set of predefined mapping rules to calculate and dynamically maintain the real-time status information of cache lines on each compute node.

[0068] The construction of the global resource view also includes: deploying a listening agent on the interconnection network of the distributed system to capture messages transmitted between nodes that conform to the cache consistency protocol;

[0069] Parse the message's type and address fields, where the message type includes read request, write-back, and invalidation notification;

[0070] Based on the predefined protocol state machine logic and the parsing result of the message, the state of the corresponding cache line in the global resource view is inferred and updated. The state belongs to the sub-state of the real-time resident state of the cache line, and the state includes exclusive, shared, modified or invalid.

[0071] Specifically, in the interconnection network between nodes of the distributed system, for example, a high-speed interconnection network employing a consistent directory or listening protocol, a hardware-based listening agent is deployed. The listening agent is configured to capture messages flowing through the network in a bypass listening mode and implement selective sampling and probabilistic inference mechanisms to reduce system overhead and improve scalability.

[0072] A preferred selective sampling and probabilistic inference mechanism includes: adaptive sampling, multi-dimensional message parsing, probabilistic state inference, differential pattern, confidence assessment, and hardware acceleration.

[0073] Adaptive sampling samples messages with an adaptive probability, which is dynamically adjusted according to network load. When the network message rate exceeds a preset threshold, the sampling rate is automatically reduced to control processing overhead.

[0074] Multi-dimensional message parsing not only parses the type and address of the sampled message, but also extracts the message's timestamp, source and destination node topology information, and message size, forming a five-tuple information packet (type, address, timestamp, source / destination node, size).

[0075] Probabilistic state inference is based on historical sampling data. A low-dimensional cache line state transition probability model is constructed and represented by a hidden Markov model. The hidden state is the hidden Markov model state of the cache line, and the observed value is the sampled protocol message type. The state transition probability of other cache lines in the unsampled time period is inferred by forward and backward algorithms.

[0076] Differential mode switching: When the system detects that the network message load continues to exceed the high threshold, the listening agent automatically switches to differential mode. In this mode, only messages related to cached lines that have changed state compared to the previous sampling period are captured, and only the state records of these cached lines are updated, further reducing processing overhead.

[0077] The confidence assessment assigns a confidence score to each inferred cache line state, and the sampling density and model consistency are calculated. When the confidence score falls below a threshold, a full sampling cycle is triggered to recalibrate the model.

[0078] The monitoring agent also includes hardware acceleration. When deployed on a smart network interface card (NIC) that supports a programmable data plane, the NIC hardware performs message filtering, sampling, and preliminary parsing, uploading only concise metadata to the host CPU, further reducing host-side overhead.

[0079] After the listening agent captures a network message, it parses the message. The parsing process is used to extract two key fields of the message: message type and memory address field. The message type includes at least read request, write-back, invalidation notification, etc.

[0080] By using predefined protocol state machine logic and the parsing result of the current message, the state of the corresponding cache line in the global resource view is inferred and updated. The predefined protocol state machine logic is the actual hardware cache consistency protocol behavior adopted by the distributed system.

[0081] In this application, a preferred example of monitoring includes: when the monitoring agent captures a read request message for a certain address from node A, and according to the history of the global resource view, the data of the address is currently only in a modified state in the cache of node B, then according to the state transition rules of the MESI protocol, it is inferred that node B will perform a write-back operation to update the memory data, and subsequently the data will be shared in the caches of node A and node B. Therefore, the state of the address in node B in the view needs to be updated to shared, and a new shared state record needs to be added for node A.

[0082] The cache line status inferred through the above process is used to define the permissions and freshness of the i-cache line replica, including:

[0083] Exclusive is used to indicate that the data exists only in the current node's cache and is consistent with memory.

[0084] Shared means that the data exists in the caches of multiple nodes, and is consistent with memory.

[0085] The "modified" flag indicates that the data exists only in the current node's cache and has been modified, and is inconsistent with memory.

[0086] Invalid is used to indicate that the copy of the data in the current node's cache is invalid and cannot be used.

[0087] After integrating this mechanism, the global resource view not only includes cache line residency and popularity information, but also permission status under the consistency protocol.

[0088] A preferred example of the predefined mapping rule is as follows: Taking an LLC cache miss event as an example, when the monitoring probe reports an LLC read failure counter event for an address, the resource view manager infers that the cache line corresponding to the address does not reside in the LLC of that node; combined with the reported memory read response event, its status is updated to reside in local memory, and status tracking is achieved by maintaining a global mapping table with memory address as the key and (node ​​ID, status, last access timestamp) as the value.

[0089] The predefined protocol state machine logic is a simplified MESI protocol logic: when the listening agent captures an invalidation notification message for address Addr, it updates the state of the Addr cache line on all nodes in the global view to invalid; when it captures a read request for Addr from node A and the current state is exclusive to node B, it updates the state of node B to shared and adds a shared state record for node A.

[0090] S2 establishes a cache-aware joint optimization model for task offloading and data prefetching. The decision variables of the joint optimization model include a first type of variable representing task offloading to each node and a second type of variable representing data block prefetching to each node's cache. The cache capacity constraint of the joint optimization model is generated by the task offloading scheme determined by the first type of variable, and the set of cache lines to be evicted in each node's cache is intrinsically calculated and predicted.

[0091] It should be noted that the first type of variable can represent a task unloading indicator variable, while the second type of variable can represent a data prefetching indicator variable.

[0092] like Figure 3 As shown, the cache capacity constraint of the joint optimization model is constructed through dynamic judgment and prediction within the joint optimization model;

[0093] Based on the task unloading scheme determined by the first type of variables, calculate the set of new data blocks that must be loaded into the cache of each node due to the execution of these tasks and their total amount;

[0094] The total amount is compared with the current free capacity of each node cache obtained through the global resource view. If the total amount of new data exceeds the free capacity, the joint optimization model will automatically identify and determine a set of data blocks to be removed from the currently resident data according to the preset cache replacement strategy during the solution process.

[0095] The set of data blocks to be removed is the set of cache lines that are predicted to be evicted.

[0096] Specifically, the cache capacity constraint of the joint optimization model is not a static, simple capacity limit, but is constructed as a dynamic judgment and prediction process within the joint optimization model, which is used to incorporate the active impact of task unloading on the cache state into the evaluation and decision-making loop within the model.

[0097] Furthermore, the method for constructing and executing cache capacity constraints includes: S201, based on the candidate task unloading scheme determined by the first type of decision variables, namely task unloading variables, calculating the set of data that needs to be accessed and is not currently residing in the cache of the current node in order to execute the tasks scheduled to each node, and calculating the total cache space size required for this new set of data blocks.

[0098] The joint optimization model described in S202 dynamically compares the estimated total amount of new data with the current free capacity of each node's cache obtained in real time through the global resource view.

[0099] If the comparison results show that, for a certain node, the total amount of data to be newly loaded does not exceed its cache free capacity, then the cache capacity constraint is automatically satisfied, and the joint optimization model does not need to handle the cache replacement problem. If the total amount of new data exceeds the free capacity, it indicates that directly loading new data will cause a cache overflow. The joint optimization model does not regard this as an unsolvable hard constraint, but in its solution process, according to the preset cache replacement strategy, for example, simulating the Least Recently Used algorithm, it identifies and selects a set of data blocks to be removed from the set of data blocks currently residing in the node. The solution algorithm of the joint optimization model includes the determination of the set of data blocks, which varies with different unloading schemes.

[0100] S203 is the set of data blocks to be removed determined by the internal logic of the model, which is defined as the set of cache lines that are predicted to be evicted.

[0101] The preset cache replacement strategy is a strategy based on access history.

[0102] Information reflecting the historical access status of cache lines is derived from the global resource view, and the type of information is the most recent access timestamp or historical access frequency.

[0103] The set of cache lines to be evicted is determined by comparing the historical access information corresponding to each data block in the currently resident data block set, and sorting and filtering the historical access information according to the principle of least recently used.

[0104] The preset cache replacement strategy is constructed as a strategy based on access history.

[0105] In this application, a preferred method for constructing a preset cache replacement strategy includes:

[0106] Assuming that the future value of cached data is related to the patterns of its past accesses, and quantifying the future value based on historical access information, the system prioritizes removing data with the lowest estimated future value when space needs to be freed up.

[0107] Furthermore, from the global resource view, key information reflecting the historical access status of each cache line is derived. The types of this information are mainly the most recent access timestamp or the historical access frequency. The most recent access timestamp records the moment when the cache line copy was last accessed by the processor core; the historical access frequency counts the number of times the cache line was accessed within a specific time window, capturing the temporal locality and frequency locality characteristics of data access, respectively.

[0108] The process of determining the set of cache lines to be evicted involves several steps. First, for a specific node, the set of currently residing data blocks is obtained. Then, the historical access information for each data block in this set is compared. Based on the Least Recently Used (LRU) principle, the historical access information is sorted and filtered: if based on timestamps, the earliest accessed data block with the oldest timestamp is considered the lowest value; if based on frequency, the data block with the fewest accesses is considered the lowest value. Through this sorting, a candidate eviction list is identified and determined in ascending order of value. When space needs to be freed up for new data, the corresponding number of cache lines are selected sequentially from this list to form the final predicted eviction set.

[0109] The joint optimization model aims to minimize the total system task completion time.

[0110] The optimization objective outputs a target value, the calculation of which includes: the theoretical execution time of the task on the selected computing node; the calculation of the target value also includes the expected latency caused by the task accessing its dependent data;

[0111] The expected latency is calculated as follows: under the combined effect of the task unloading decision and the data prefetching decision, the final expected physical location of each data block is quantified. The location is divided into local cache hit, local memory hit, and cross-node remote memory access, and is weighted and accumulated differently.

[0112] The joint optimization model establishes a clear optimization objective: to minimize the total system task completion time;

[0113] The specific optimization objective is a core indicator used to measure the overall efficiency of the distributed system. It drives the joint optimization model to always evolve in the direction of improving system throughput and reducing job latency when exploring possible decision combinations. The joint optimization model guides the search by evaluating the objective function value under different decision combinations, i.e., the objective value. The calculation of the objective value consists of two core parts.

[0114] Furthermore, the first part is the theoretical execution time of the task on the selected computing node. This time depends on the instruction complexity of the task itself, the required computing resources such as CPU cycles, and the current computing power and load of the target node.

[0115] The second part is the expected latency caused by the task accessing its dependent data, which is improved by using a differentiated data access cost modeling method to improve its accuracy.

[0116] Specifically, the method for calculating the expected latency includes: the system needs to quantify the final expected physical location of each data block that each task depends on, under the combined effect of a specific set of task offloading decisions and data prefetching decisions.

[0117] In this application, a preferred implementation method of the joint optimization model includes:

[0118] A cache-aware joint optimization model for task offloading and data prefetching is established. The joint optimization model is formalized as a mixed integer linear programming problem. Its decision variables include both first-class variables representing task offloading to each node and second-class variables representing data block prefetching to each node's cache.

[0119] Furthermore, cache capacity constraints are intrinsically constructed through the task offloading scheme to predict the eviction set; and the set-associative properties of the cache are further modeled through prefetch eviction interlock constraints to ensure that the optimized solution is consistent with the actual hardware behavior. The cache capacity constraints of the joint optimization model are generated through the task offloading scheme determined by the first type of variables, and the set of cache lines to be evicted in each node's cache is intrinsically calculated and predicted.

[0120] Specifically, the joint optimization model includes modeling of the objective function, cache state continuity constraints, prefetch eviction interlock constraints, and cache replacement strategy;

[0121] The objective function is used to minimize the total task completion time, and its calculation expression is:

[0122]

[0123] in, Let be the theoretical execution time of task i on the selected computing node; n represents the expected latency caused by task i accessing its dependent data, where i is the task identifier and n is a positive integer.

[0124]

[0125] This indicates whether data block k is located in the local cache, and its value is a 0-1 indicator. Data block k is located in local memory and has a value of 0-1 as an indicator. Local cache latency weight; Local memory latency weight; Delay weights for other locations; The size of data block k, The set of data blocks that task i depends on;

[0126] Furthermore, key innovation constraints include cache state continuity constraints and prefetch eviction interlock constraints;

[0127] The cache state continuity constraint is also an endogenous eviction prediction, and its calculation expression is as follows:

[0128]

[0129] in, This indicates whether data block k has been prefetched into the cache of node j, with 1 indicating yes and 0 indicating no. This indicates whether task i should be unloaded to node j for execution; 1 indicates yes, 0 indicates no. Is data block k predicted to be evicted from node j's cache? 1 for yes, 0 for no. j is the node identifier, and k is the identifier of the data block to be prefetched, used to identify data needed for future tasks. This refers to the set of cached data blocks currently residing on node j. The set of tasks scheduled to be executed on node j; Let j be the set of data blocks predicted to be evicted at node j. This represents the total amount of new data that needs to be loaded during the execution of task i, excluding cached data. The current free capacity of the cache for node j;

[0130] This represents the total amount of newly prefetched data and newly loaded data for the task. This is the sum of the cache free capacity and the total amount of data that the model allows to be evicted. The value of is constrained by the cache replacement strategy, LRU, and is linearized through auxiliary constraints. The calculation expression of the auxiliary constraints is as follows:

[0131]

[0132] in, The LRU rank of data block k in the cache of node j is provided by the global resource view, and the smaller the value, the more recently accessed it is. This represents the number of data blocks currently cached by node j.

[0133] The prefetch-eviction interlock constraint indicates that a data block can only be prefetched into a specific cache set. The prefetch-eviction interlock constraint ensures that a prefetch operation is allowed only when the prefetched data k and some evicted data k′ are mapped to the same cache set. The calculation expression is:

[0134]

[0135] in, This is an indicator function whose value is 1 if and only if the prefetch block k and the eviction candidate block k' are mapped to the same cache set according to their memory addresses. For the currently cached data, Data block k' that is already in the cache of node j is predicted to be evicted.

[0136] Specifically, in the prefetch eviction interlock constraint, For a data block k' already residing in the cache of node j that is predicted to be evicted, the index k' is used to traverse all data blocks in the current cache of node j. Only when at least one data block k' mapped to the same cache group as data block k is evicted, i.e., there exists a k' such that... =1 and When =1, data block k can be prefetched to node j.

[0137] In the joint optimization model, the prefetch-eviction interlock constraint is used to describe the physical limitations of set-associative caches. New data can only be stored in the corresponding cache line released by a specific eviction operation. This avoids the decision distortion caused by the traditional model treating the cache as a unified pool. The set-associative cache structure and the resulting physical constraints are modeled at the optimization level, making the collaborative decision-making of task offloading and data prefetching consistent with the underlying hardware behavior. This solves the problem of infeasible optimization results or inaccurate performance prediction caused by the oversimplification of cache behavior in the traditional resource scheduling model, and improves the accuracy of resource orchestration and the upper limit of system performance.

[0138] Furthermore, the intended locations of data blocks are explicitly categorized into three levels with significantly different access costs:

[0139] The local cache was hit, and the data block is located in the last-level cache of the node where the task is located.

[0140] A local memory hit means that the data block resides only in the system's main memory on the node where the task is located, and does not reside in the cache.

[0141] Cross-node remote memory access, where data blocks reside in the memory of other compute nodes.

[0142] After determining the expected location of each data block, the expected latency is calculated by differentially weighting and accumulating the access latency of all data blocks, assigning a baseline latency weight to local cache hit, local memory hit, and cross-node remote memory access respectively. For example, the weight coefficient of local cache hit is the lowest, and the weight coefficient of cross-node access is the highest. The weight is calibrated based on the actual hardware performance parameters of the distributed system, such as cache access cycle, memory bus latency, network bandwidth and latency.

[0143] For each data block, a corresponding baseline weight is selected based on its predicted position by the model, and then multiplied by its data size or number of accesses. Finally, the weighted delays of all data blocks are summed to obtain the total expected delay for the task to access the data.

[0144] This application, by constructing the aforementioned joint optimization model, can provide a quantitative assessment of the actual runtime performance of a candidate task unloading-data prefetching joint decision scheme, identify decision schemes with drastically different performance due to subtle differences in data layout, and find the globally optimal resource coordination configuration to minimize the total task completion time of the system.

[0145] The joint optimization model is solved using a decomposition and coordination algorithm.

[0146] The decomposition and coordination algorithm first decomposes the original problem, which couples task unloading variables and data prefetching variables, into task unloading sub-problems and data prefetching sub-problems along the resource dimension. By introducing coordination variables to reflect the tension of shared resources such as cache capacity, the algorithm correlates the sub-problems with the tension of shared resources through coordination variables. Through iterative calculation, each sub-problem is solved independently with the current coordination variable value.

[0147] The main problem updates the coordination variables and the estimate of the predicted cache line eviction set based on the solutions to the subproblems, thereby guiding the next iteration until convergence to a consistent joint optimization solution.

[0148] In this application, a preferred implementation method of the decomposition and coordination algorithm includes:

[0149] S311 decomposes the original problem along the resource dimension.

[0150] Specifically, based on the nature of the decision variables, a task offloading subproblem and a data prefetching subproblem are constructed. The task offloading subproblem is to select the best computing node for the task given a data access cost estimate. The data prefetching subproblem is to decide when and where to prefetch which data blocks given a data demand forecast for the task.

[0151] The decomposition and coordination algorithm uses coordination variables to pass the scarcity of shared resources to each subproblem: the scarcer the resource, the higher the price, and the more economical the subproblem will be in using the resource.

[0152] In each iteration of S312, the decomposition and coordination algorithm fixes the current value of the coordination variable, and the task unloading subproblem and the data prefetching subproblem are solved independently and in parallel.

[0153] Specifically, since resource constraints are internalized into their respective objective functions through pricing, solving the subproblems is simplified. After solving, each subproblem reports its solution to the main problem, such as task offloading scheme X and data prefetching scheme P, as well as the actual usage of shared resources such as cache capacity under that scheme.

[0154] The S313 master problem receives reports from subproblems. Based on the actual resource usage reported by the subproblems, it compares it with the total global resource supply. If a resource or a node's cache is over-demanded, the master problem will increase its corresponding coordination variable, i.e., change the price; otherwise, it will decrease it.

[0155] A preferred example: While updating the coordination variables, the main problem also updates the estimate of the predicted cache line eviction set based on the latest resource requirements implied by the solution X of the task offloading subproblem. This ensures that the cache vacancy information used by the data prefetching subproblem in the next iteration is up-to-date and consistent with the current offloading scheme, thus achieving synchronous decision-making between task offloading and data prefetching based on the same solution vector.

[0156] The decomposition and coordination algorithm described in S314 executes S312 and S313 in a loop. In each loop, the coordination variables, the solutions to subproblems, and the prediction of the cache state are closer to the global optimum. When the solutions to subproblems no longer change significantly in two consecutive iterations, and all shared resource constraints are approximately satisfied, the algorithm is considered to have converged. At this time, the outputs X and P are the coordinated joint optimization solutions obtained under the decomposition and coordination framework.

[0157] Based on the decomposition and coordination architecture, this scheme decomposes complex joint optimization problems into subproblems that can be solved in parallel, and achieves resource coordination among subproblems through coordination variables. While ensuring the quality of the solution, it improves the efficiency of generating optimization decisions and supports real-time scheduling of the system in dynamic environments.

[0158] S3 solves the joint optimization model with the goal of minimizing the total task completion time, and simultaneously generates task unloading decisions and collaborative prefetching decisions. The collaborative prefetching decisions are determined based on the predicted set of cache lines to be evicted and the data dependencies between tasks.

[0159] Specifically, the execution sequence of the collaborative prefetching decision is as follows: before the task scheduler issues task execution instructions to the nodes, the data prefetcher issues prefetch instructions to the target nodes based on the solution vector and the predicted eviction set. When each node's cache controller loads new task data, if it needs to evict old rows, it will prioritize overwriting non-prefetch target data blocks, thereby reserving space for the data to be prefetched. The prefetcher obtains the expected empty positions by querying the prediction cache mapping table maintained by the resource view manager.

[0160] The task scheduler and the data prefetcher share the same solution vector output by the joint optimization model;

[0161] The task scheduler generates the task unloading decision based on the assignment results of the first type of variables in the solution vector; the data prefetcher generates the collaborative prefetch decision based on the assignment results of the second type of variables in the solution vector and in combination with the cache vacancy positions corresponding to the predicted set of cache lines to be evicted.

[0162] The assignment of the second type of variable implicitly includes the data dependency relationship between tasks and the dual constraint of the cache vacancy position.

[0163] Specifically, after convergence, the joint optimization model outputs a solution vector, which is the set of optimal values ​​for all decision variables of the joint optimization model, namely the first type of variables and the second type of variables. It encapsulates the best resource allocation scheme after global trade-offs. The task scheduler and the data prefetcher, as two execution components of the system, share this unique solution vector as the common basis for their actions.

[0164] Furthermore, the task scheduler parses the solution vector and extracts the assignment results of the first type of variables. Each first type of variable is used to represent which computing node a specific task should be unloaded to. The task scheduler generates a task unloading decision, namely a series of task-target node mapping instructions, and distributes the mapping instructions to the corresponding computing nodes to start task execution.

[0165] Furthermore, the data prefetcher parses the same solution vector in parallel and extracts the assignment results of the second type of variables. Each second type of variable defines which node's cache a specific data block should be prefetched into within a specific time period. The prefetcher uses this as the basic driving force for the prefetching operation. After the task is unloaded and executed, the empty positions in each node's cache will be freed up.

[0166] Furthermore, the prefetcher matches and schedules the prefetch targets specified by the second type of variables with known cache vacancy locations to generate accurate collaborative prefetch decisions, ensuring that the prefetched data can be efficiently filled into the cache space generated by the unloading of this task, thus realizing the cyclical reuse of the cache space.

[0167] In this scheme, the task scheduler and data prefetcher work together based on the same optimization solution to achieve coordination between computing tasks and data flow in the spatiotemporal dimensions, thereby reducing the total task completion time.

[0168] Based on a preset task dependency graph, identify subsequent tasks on the critical path and the data blocks they depend on; compare the set of cache lines predicted to be evicted with the set of data blocks depended on by the subsequent tasks on the critical path;

[0169] Data blocks identified during the comparison as being invalid due to eviction and simultaneously dependent on the critical path task are prefetched instructions.

[0170] The collaborative prefetching decision generation is based on global optimization solutions and cache state prediction, and a task criticality analysis mechanism is introduced to determine priorities. This mechanism prioritizes the data required by tasks that have the greatest impact on the overall system completion time, optimizes cache resource allocation, and improves system performance.

[0171] Specifically, the pre-defined task dependency graph, in the form of a directed acyclic graph, describes the data dependencies and sequence relationships between all tasks to be executed.

[0172] By using the critical path through the task dependency graph, subsequent tasks located on the critical path are identified. The critical path is the path with the longest sum of the durations of all tasks from the beginning to the end of the task flow. Any delay on the path will directly lead to an extension of the total completion time. In summary, the tasks on the critical path and the data blocks they depend on have the highest scheduling and guarantee priority.

[0173] The collaborative prefetching decision generation process then introduces a comparison and filtering stage. Specifically, the set of cache lines predicted to be evicted by the joint optimization model is compared with the set of data blocks on which subsequent tasks on the critical path depend. This is to find the intersection between the cache line set and the data block set, that is, the data blocks that are currently residing in the cache but are predicted to be evicted due to the execution of this unloading task, and are also necessary for subsequent tasks on the critical path. This ensures that the data needs that have the greatest impact on the overall system performance bottleneck are met first, and actively prevents the chain of delays caused by critical path tasks waiting for data.

[0174] Example 2

[0175] A joint resource allocation and task offloading optimization system includes:

[0176] The global resource view senses the real-time residency status of cache lines on each computing node in the distributed computing system.

[0177] The joint optimization module, which is communicatively connected to the global resource view construction module, includes a computing unit, a first unit, and a decision unit. The computing unit establishes a cache-aware joint optimization model for task offloading and data prefetching. The decision variables of the joint optimization model include both first-type variables representing task offloading to each node and second-type variables representing data block prefetching to the cache of each node.

[0178] The first unit generates the cache capacity constraint of the joint optimization model based on the task unloading scheme determined by the first type of variables, and intrinsically calculates and predicts the set of cache lines that will be evicted in the cache of each node.

[0179] The decision-making unit solves the joint optimization model with the objective of minimizing the total task completion time, and simultaneously outputs task unloading decisions and collaborative prefetching decisions.

[0180] The task coordination module is communicatively connected to the joint optimization module. Based on the task unloading decision, it schedules the computational tasks to the corresponding nodes for execution. Based on the collaborative prefetch decision, the predicted set of cache lines to be evicted and the data dependencies between tasks, it initiates a data prefetch operation on the corresponding nodes.

[0181] Preferably, the global resource view construction module includes:

[0182] Monitoring probes deployed on each computing node are used to periodically collect and transmit hardware performance counter values ​​across the system network; and

[0183] The resource view manager is used to calculate and maintain the real-time resident probability or recent access time tag of each node's cache line based on the reported counter value sequence and through predefined mapping rules.

[0184] Preferably, the global resource view construction module further includes:

[0185] A listening agent deployed on the interconnection network of the distributed computing system is used to capture and parse messages transmitted between nodes that conform to the cache consistency protocol;

[0186] The resource view manager is also used to infer and update the state of the corresponding cache line in the global resource view based on the parsed message and its corresponding protocol state machine logic.

[0187] Preferably, the joint optimization module is configured as follows:

[0188] The dynamic judgment and joint optimization model prediction based on the cache capacity constraint includes: calculating the total amount of newly loaded data determined by the task unloading scheme, comparing it with the current free capacity of the node cache, and when the total amount exceeds the limit, identifying and determining the set of cache lines predicted to be evicted from the currently resident data according to the preset cache replacement strategy.

[0189] Preferably, the preset cache replacement strategy is a strategy based on access history; the joint optimization module obtains or derives the historical access information of cache lines by querying the global resource view, and sorts and filters the information according to the least recently used principle to determine the set of cache lines predicted to be evicted.

[0190] Preferably, the joint optimization module is further configured as follows:

[0191] The objective function of the joint optimization model is constructed to minimize the total task completion time of the system. The calculation of the total task completion time takes into account the computational overhead of the task at the node and the expected delay caused by accessing dependent data. The expected delay is evaluated differently based on the expected physical location of the data after the decision.

[0192] Preferably, the joint optimization module is configured to solve the joint optimization model by a decomposition coordination algorithm, wherein the algorithm decomposes the original problem into a task offloading subproblem and a data prefetching subproblem for iterative coordination and solution, and updates the estimate of the set of cache lines predicted to be evicted during the iteration process.

[0193] Preferably, the task and data coordination and execution module includes:

[0194] A task scheduler is used to allocate tasks to computing nodes based on the task offloading decision;

[0195] A data prefetcher, working in conjunction with the task scheduler, is used to perform data prefetching based on the collaborative prefetching decision;

[0196] The task scheduler and the data prefetcher share the same solution vector output by the joint optimization module.

[0197] Preferably, the data prefetcher is specifically configured as follows:

[0198] Identify subsequent tasks on the critical path and their dependent data blocks based on the task dependency graph;

[0199] The set of cache lines predicted to be evicted is compared with the set of data blocks on which the critical path task depends;

[0200] A prefetch operation is initiated for data blocks that are identified during the comparison as being invalid due to eviction and are also depended upon by the critical path task.

[0201] like Figure 5 As shown, the bottom-level global resource view module constructs the system resource status by collecting hardware performance data of each computing node and cache consistency messages in the network in real time; the middle-level linkage optimization engine performs joint optimization calculations based on this view, and synchronously generates collaborative decisions for task unloading and data prefetching; the upper-level task scheduler and data prefetcher execute these two types of decisions respectively, accurately distributing computing tasks and required data to the corresponding nodes, thus realizing the orchestration and scheduling of computing and cache resources.

[0202] It is important to note that the constructions and arrangements of this application shown in several different exemplary embodiments are merely illustrative. Although only two embodiments are described in detail in this disclosure, those who consult this disclosure will readily understand that many modifications are possible without substantially departing from the novel teachings and advantages of the subject matter described in this application. These modifications may include, for example, changes in the size, dimensions, structure, shape, and proportions of various elements, as well as parameter values ​​(e.g., temperature, pressure, etc.), installation arrangements, the use of materials, colors, orientations, etc. For example, an element shown as integrally formed may be composed of multiple parts or elements, the position of elements may be inverted or otherwise altered, and the nature or number or position of discrete elements may be changed or altered. Therefore, all such modifications are intended to be included within the scope of this application. The order or sequence of any process or method steps may be changed or rearranged by alternative embodiments. Any "apparatus plus function" clause is intended to cover, and not only structurally equivalent but also equivalent structures, the structures performing the functions described herein. Other substitutions, modifications, alterations, and omissions may be made in the design, operation, and arrangement of the exemplary embodiments without departing from the scope of this application. Therefore, this application is not limited to a particular embodiment, but extends to various modifications that still fall within the scope of the appended claims.

[0203] Furthermore, in order to provide a concise description of exemplary embodiments, not all features of actual embodiments (i.e., those features that are not relevant to the best mode of performing this application as currently considered, or those features that are not relevant to implementing this application) may be omitted.

[0204] It should be understood that numerous specific implementation decisions can be made during the development of any practical implementation, such as in any engineering or design project. Such development efforts may be complex and time-consuming, but for those of ordinary skill in the art who benefit from this disclosure, the development effort will be a routine task in design, manufacturing, and production without requiring extensive experimentation.

[0205] It should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application, and all such modifications and substitutions should be covered within the scope of the claims of this application.

Claims

1. A method for optimizing joint resource allocation and task offloading, characterized in that, include: S1 constructs a global resource view to perceive the real-time residency status of cache lines on each node in the distributed system; S2 establishes a cache-aware joint optimization model for task offloading and data prefetching. The decision variables of the joint optimization model include a first type of variable representing task offloading to each node and a second type of variable representing data block prefetching to each node's cache. The task offloading scheme determined by the first type of variable generates the cache capacity constraint of the joint optimization model. The set of cache lines to be evicted in each node's cache is intrinsically calculated and predicted. The cache capacity constraint of the joint optimization model is constructed through dynamic judgment and prediction within the joint optimization model. Based on the task unloading scheme determined by the first type of variables, calculate the set of new data blocks that must be loaded into the cache of each node due to the execution of these tasks and their total amount; The total amount is compared with the current free capacity of each node cache obtained through the global resource view. If the total amount of new data exceeds the free capacity, the joint optimization model will automatically identify and determine a set of data blocks to be removed from the currently resident data according to the preset cache replacement strategy during the solution process. The set of data blocks to be removed is the set of cache lines predicted to be evicted; S3 solves the joint optimization model with the goal of minimizing the total task completion time, and simultaneously generates task unloading decisions and cooperative prefetching decisions. The cooperative prefetching decision is determined by the set of cache lines to be evicted and the data dependencies between tasks.

2. The method for optimizing joint resource allocation and task offloading as described in claim 1, characterized in that: The construction of the global resource view includes: deploying a monitoring probe on each computing node, the monitoring probe periodically collecting and reporting the hardware performance counter values ​​of the node through the system network, the hardware performance counters including counters reflecting cache hit and miss events at all levels; Based on the counter value sequence reported by each node, the real-time residence probability or recent access time tag of each node's cache line is calculated and maintained through predefined mapping rules.

3. The method for optimizing joint resource allocation and task offloading as described in claim 2, characterized in that: The construction of the global resource view also includes: deploying a listening agent on the interconnection network of the distributed system to capture messages transmitted between nodes that conform to the cache consistency protocol; Parse the message's type and address fields, where the message type includes read request, write-back, and invalidation notification; Based on the predefined protocol state machine logic and the parsing result of the message, the state of the corresponding cache line in the global resource view is inferred and updated. The state includes exclusive, shared, modified, or invalid.

4. The method for optimizing joint resource allocation and task offloading as described in claim 3, characterized in that: The preset cache replacement strategy is a strategy based on access history. Information reflecting the historical access status of cache lines is derived from the global resource view, and the type of information is the most recent access timestamp or historical access frequency. The set of cache lines to be evicted is determined by comparing the historical access information corresponding to each data block in the currently resident data block set, and sorting and filtering the historical access information according to the principle of least recently used.

5. The method for optimizing joint resource allocation and task offloading as described in claim 1, characterized in that: The joint optimization model aims to minimize the total system task completion time. The optimization objective outputs a target value, the calculation of which includes: the theoretical execution time of the task on the selected computing node; the calculation of the target value also includes the expected latency caused by the task accessing its dependent data; The expected latency is calculated as follows: under the combined effect of the task unloading decision and the data prefetching decision, the final expected physical location of each data block is quantified. The location is divided into local cache hit, local memory hit, and cross-node remote memory access, and is weighted and accumulated differently.

6. The method for optimizing joint resource allocation and task offloading as described in claim 1, characterized in that: The joint optimization model is solved using a decomposition and coordination algorithm. The decomposition and coordination algorithm first decomposes the original problem, which couples task unloading variables and data prefetching variables, into task unloading sub-problems and data prefetching sub-problems along the resource dimension. By introducing coordination variables to reflect the tension of cache capacity, the sub-problems are correlated with the tension of shared resources through coordination variables. Through iterative calculation, each sub-problem is solved independently with the current coordination variable value. The main problem updates the coordination variables and the estimate of the predicted cache line eviction set based on the solutions to the subproblems, thereby guiding the next iteration until convergence to a consistent joint optimization solution.

7. The method for optimizing joint resource allocation and task offloading as described in claim 1, characterized in that: The task scheduler and the data prefetcher share the same solution vector output by the joint optimization model; The task scheduler generates the task unloading decision based on the assignment results of the first type of variables in the solution vector; The data prefetcher generates the collaborative prefetching decision by using the assignment results of the second type of variables in the solution vector and combining them with the cache vacancy positions corresponding to the predicted set of cache lines to be evicted. The assignment of the second type of variable implicitly includes the data dependency relationship between tasks and the dual constraint of the cache vacancy position.

8. The method for optimizing joint resource allocation and task offloading as described in claim 7, characterized in that: Based on a preset task dependency graph, identify subsequent tasks on the critical path and the data blocks they depend on; compare the set of cache lines predicted to be evicted with the set of data blocks depended on by the subsequent tasks on the critical path; Data blocks identified during the comparison as being invalid due to eviction and simultaneously dependent on the critical path task are prefetched instructions.

9. A joint resource allocation and task offloading optimization system, characterized in that... Includes a joint resource allocation and task offloading optimization method as described in any one of claims 1-8; wherein: The global resource view senses the real-time residency status of cache lines on each computing node in the distributed computing system; The joint optimization module, which is communicatively connected to the global resource view construction module, includes a computing unit, a first unit, and a decision unit. The computing unit establishes a cache-aware joint optimization model for task offloading and data prefetching. The decision variables of the joint optimization model include both first-type variables representing task offloading to each node and second-type variables representing data block prefetching to the cache of each node. The first unit generates the cache capacity constraint of the joint optimization model based on the task unloading scheme determined by the first type of variables, and intrinsically calculates and predicts the set of cache lines that will be evicted in the cache of each node. The decision-making unit solves the joint optimization model with the objective of minimizing the total task completion time, and simultaneously outputs task unloading decisions and collaborative prefetching decisions. The task coordination module is communicatively connected to the joint optimization module. Based on the task unloading decision, it schedules the computational tasks to the corresponding nodes for execution. Based on the collaborative prefetch decision, the predicted set of cache lines to be evicted and the data dependencies between tasks, it initiates a data prefetch operation on the corresponding nodes.

Images