Non-uniform memory access resource allocation method based on parallel adaptive auction algorithm
By applying parallel adaptive auction algorithms and DQN models in the NUMA architecture, the reference dependence and resource allocation efficiency of traditional algorithms in parallel computing are solved, efficient resource allocation is achieved, and system performance and energy efficiency are improved.
Patent Information
- Application Number
- CN202410987081.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-07-23
AI Technical Summary
In the non-unified memory access (NUMA) architecture, traditional dynamic programming and greedy algorithms have problems such as reference interdependence, storage conflicts and uneven load in parallel computing environments, making it difficult to effectively decompose task parallel computing, and lack of a global perspective, resulting in Resource allocation efficiency is inefficient.
Using a parallel adaptive auction algorithm, combined with the DQN model in deep reinforcement learning, the best α value is selected for each task through the trained DQN model, and parallel calculation is performed using heterogeneous multithreading to realize resource allocation Independent bidding and efficient allocation.
It significantly improves computing efficiency and resource allocation efficiency, overcomes the limitations of traditional algorithms in parallel processing, realizes decentralized and efficient resource allocation, and improves system performance and energy efficiency.
Smart Images

Figure CN118981372A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of computer resource scheduling, and in particular relates to a non-uniform memory access resource allocation method based on a parallel adaptive auction algorithm. Background Art
[0002] The non-uniform memory access (NUMA) architecture is a computer memory hardware design architecture for multi-processor systems, in which processors and memory are organized into multiple nodes, each with its own memory and multiple processors. The NUMA architecture allows processors to directly access memory within the local node, thereby reducing memory access latency and improving system performance. In the field of computer underlying operating systems, the NUMA architecture is widely used in scenarios such as memory management and task scheduling. The knapsack problem is an important combinatorial optimization problem that is widely used in production scheduling, cloud computing, network communications, and asset management. In the field of computer underlying operating systems, different types of knapsack problems can be applied to memory management and task scheduling. In this case, task scheduling can be conceptualized as a dynamic and real-time online market in which various tasks (or "buyers") compete for limited resources (or "goods"), with the goal of optimizing and allocating limited resources based on requirements and priorities.
[0003] In order to better optimize the resource allocation problem in the NUMA architecture, researchers have proposed a variety of extended versions of the knapsack problem, including the multidimensional knapsack problem, the unbounded knapsack problem, the bounded knapsack problem, and the multiple knapsack problem. These extended versions can more accurately simulate complex scenarios in the real world, but in production scheduling and hardware resource allocation, the relative position constraints between items must be considered. This extended knapsack problem with position constraints increases the complexity of the problem. It is necessary not only to consider the value and weight of the items, but also to impose specific constraints on the position of the items in the solution. The non-uniform memory access (NUMA) architecture is an important application of the extended knapsack problem. It is a computer memory hardware design architecture for multi-processor systems. In the NUMA architecture, processors and memory are organized into multiple nodes, each node has its own memory and multiple processors (such as Figure 1 This design allows the processor to directly access the memory within its local node without crossing other nodes, which helps reduce memory access latency and improve system performance. Compared with traditional system architecture, NUMA architecture has more advantages in scalability and reducing memory latency.
[0004] In the NUMA architecture, optimizing memory access latency is the key to improving the overall performance of the system. Therefore, solving the knapsack problem with location constraints is crucial to the performance and energy efficiency of the system. However, this complex large-scale optimization problem requires allocating resources to limited nodes. Traditional optimization methods include dynamic programming algorithms and greedy algorithms. These methods have the following shortcomings in parallel computing environments:
[0005] Dynamic programming algorithms have the disadvantages of reference interdependence, storage conflicts, and uneven loads: since the current solution usually depends on the solution of the previously calculated subproblem, the interdependence between subproblems makes it difficult to divide the subproblems into subtasks for parallel computing; at the same time, dynamic programming algorithms usually need to frequently access and update storage structures (such as tables or arrays). In a parallel environment, multiple processors or threads accessing and modifying the same storage location at the same time may cause conflicts and inconsistencies; different stages of dynamic programming may require different amounts of calculations, which makes it difficult to evenly distribute the workload among processors.
[0006] Greedy algorithms have the disadvantages of local optimality, non-online sequential decision-making, and lack of global perspective: they are an algorithm that takes the current optimal solution in each step of selection, but because they make local optimal decisions at each step, these decisions usually need to rely on the current state; and when it comes to sequential decision-making processes, each decision is based on the previous decision, and this continuity makes parallelization complicated; at the same time, because they usually have no global perspective and only focus on the optimal solution of the current step, they may lead to inconsistent or inefficient decisions in a parallel environment. In addition, in complex and uncertain environments, it is crucial to make robust decisions for different states; to solve these two problems, researchers have proposed some methods, such as GPU-accelerated dynamic programming algorithms and distributed computing optimization schemes for sub-problem decomposition. However, these methods cannot completely decouple subtasks. Summary of the invention
[0007] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and provide a non-uniform memory access resource allocation method based on a parallel adaptive auction algorithm, combining economic theory with computer science to achieve decentralized and efficient resource allocation.
[0008] In order to achieve the above object, the present invention adopts the following technical solutions:
[0009] On the one hand, a non-uniform memory access resource allocation method based on a parallel adaptive auction algorithm is provided, comprising the following steps:
[0010] When the operating system is started, node information and task information in the non-uniform memory access architecture are obtained; the node information includes the number of nodes, the node start position and the node end position; the task information includes the number of tasks and the number of nodes occupied by the tasks;
[0011] The resource problem of non-uniform memory access architecture is regarded as a knapsack problem with location constraints, and a parallel adaptive auction algorithm is used to optimize and solve it to obtain the allocation result.
[0012] As a preferred technical solution, the resource allocation problem of the non-uniform memory access architecture is regarded as an extended knapsack problem with location constraints, which is expressed as:
[0013]
[0014] W i =(Ni) 0.5 , 0≤i <N,
[0015]
[0016] where(position i <position j +width j )and(position j <position i +width i ),
[0017] Among them, Score() is the objective function; W i is the weight of the i-th task; width i The amount of resources occupied by the i-th task; x i is the status of the i-th task, when x i =1 indicates that the i-th task is selected. i = 0 means that the i-th task is not selected; N is the number of tasks; C is the backpack capacity, that is, the maximum amount of resources that all selected tasks cannot exceed; position i is the location information of the i-th task, l i is the lower limit of the i-th task position, u i is the upper limit of the i-th task position; x j is the status of the jth task; position j is the location information of the jth task.
[0018] As a preferred technical solution, the parallel adaptive auction algorithm is used for optimization and solution, and the process is as follows:
[0019] Use the trained DQN model to select the best α value for each task that is suitable for the current environment and state; the best α value is used to adjust the bid price for each task;
[0020] Heterogeneous multithreading is used to parallelly calculate the bid price of the starting position of each task for bidding and store it in the task list; the bid price calculation formula is: bid i =W i *α i , among which, bid i is the bid price of the i-th task, α i is the optimal α value for the i-th task;
[0021] The tasks are assigned based on the task list to obtain the final starting positions and the best α value list of all tasks.
[0022] As a preferred technical solution, the DQN model includes a current Q network and a target Q network with the same architecture but different parameters, and an experience playback buffer;
[0023] The current Q network is used to predict the Q value of each possible bidding action of each task under the current task state, and select the bidding action with the highest Q value as the optimal bidding action under the current task state;
[0024] The target Q network is used to calculate the target Q value in the next task state, and the parameters of the target Q network are periodically copied from the current Q network;
[0025] The experience replay buffer is used to store the experience of the DQN model training process and update the parameters of the current Q network.
[0026] As a preferred technical solution, the trained DQN model is used to select the best α value for each task that is suitable for the current environment and state, specifically:
[0027] Randomly extract a small batch of experience data from the experience replay buffer; each experience data contains the task status, bidding action, reward of the task at the current time step, as well as the task status and bidding action of the next time step;
[0028] The target Q network in the trained DQN model is used to calculate the target Q value of the task, and the bidding action with the highest target Q value of the task is selected as the optimal α value in the current task state; the calculation formula of the target Q value is:
[0029] Q(state t ,action t )=reward t +γmax(action t+1 )*Q(state t+1,action t+1 θ - ),
[0030] Among them, Q(state t ,action t ) is the target Q value of the task at the current time step t, state t is the task status at the current time step t, action t is the bidding action of the task at the current time step t, reward t is the reward of the task at the current time step t; γ is the discount factor, which is used to balance the proportion of the reward of the task at the current time step and the reward of the next time step; Q(state t+1 ,action t+1 θ - ) is the task at the next time step t+1, parameter θ - The target Q value under max(action t+1 ) is the maximum target Q value predicted by the target Q network in the next task state; θ - is the parameter of the target Q network at the next time step t+1; state t+1 is the task status at the next time step t+1; action t+1 is the bidding action of the task at the next time step t+1.
[0031] As a preferred technical solution, the DQN model uses the mean square error function as the loss function, which is expressed as:
[0032]
[0033] δ i =Q(state t ,action t )-Q(state t ,action t ; θ),
[0034] Among them, δ i is the TD error of the i-th experience, which represents the difference between the Q value predicted by the current Q network and the target Q value of the target Q network; θ is the parameter of the current Q network; Num ER is the amount of experience in the experience replay buffer; Q(state t ,action t ) is the target Q value output by the target Q network, Q(state t ,action t ; θ) is the predicted Q value output by the current Q network under parameter θ; state tis the task status at the current time step t, action t is the bidding action of the task at the current time step t.
[0035] As a preferred technical solution, the use of heterogeneous multithreading to parallelly calculate the bid price of the starting position of each task for bidding is specifically as follows:
[0036] Initialize a task list tasks, a thread pool thread_pool and a shared result structure shared_results; the task list tasks contains all subtasks that need to be executed, that is, the bidding tasks after each task calculates the bid price of the starting position; the thread pool thread_pool is used to allocate threads to execute subtasks; the shared result structure shared_results is used to store the execution results of each subtask;
[0037] Allocate a heterogeneous thread async_thread from the thread pool to each subtask and execute the subtask, and store the execution result of the subtask in shared_results until all threads are executed;
[0038] Process the execution results of all subtasks in shared_results to generate the final result final_result.
[0039] As a preferred technical solution, the task allocation based on the task list is specifically as follows:
[0040] When the task list is not empty, take out the task pattern with the highest bid price and the task starting position pos from the task list;
[0041] Determine whether the position range from the task start position pos to the task end position pos+grid[pattern][0] is covered, where grid[pattern][0] represents the amount of resources required by the task pattern;
[0042] If not covered, update the task’s final starting position final_positions[pattern] and coverage status covered;
[0043] If it has been covered, update the task list, remove the tasks that conflict with the current position, and re-stack the task list;
[0044] Repeat the above steps until all tasks are assigned, and return the final starting positions of all tasks final_positions.
[0045] On the other hand, a non-uniform memory access resource allocation system based on a parallel adaptive auction algorithm is provided, including an information acquisition module and a solution allocation module;
[0046] The information acquisition module is used to acquire node information and task information in the non-uniform memory access architecture when the operating system is started; the node information includes the number of nodes, the node start position and the node end position; the task information includes the number of tasks and the number of nodes occupied by the tasks;
[0047] The allocation solution module is used to regard the resource problem of the non-uniform memory access architecture as a knapsack problem with position constraints, and adopt a parallel adaptive auction algorithm to optimize and solve it to obtain an allocation result.
[0048] Another aspect of the present invention provides a computer-readable storage medium storing a program, which, when executed by a processor, implements the above-mentioned non-uniform memory access resource allocation method based on a parallel adaptive auction algorithm.
[0049] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0050] 1. Heterogeneous multi-threaded parallel accelerated computing:
[0051] When dealing with the extended knapsack problem with position constraints, traditional methods have difficulty effectively decomposing the main problem into independent parallel computing subtasks, and the efficiency is low. However, the present invention uses the idea of the auction mechanism to make the bidding strategies of each task independent of each other, and can decompose the main problem into independent parallel computing subtasks, significantly improving the computing efficiency.
[0052] 2. Efficient parallel adaptive auction algorithm:
[0053] Traditional algorithms have limitations in parallel processing and resource allocation, especially when they need to process large amounts of data and complex constraints. This invention, for the first time, breaks through the difficulty of parallelization of traditional algorithms and, based on the auction mechanism in economics, achieves more effective and efficient allocation of memory resources. This approach, which combines economic theory and computer science, provides a new perspective for parallel processing.
[0054] 3. Adaptive price bidding strategy based on DQN model:
[0055] The traditional auction mechanism is based on a static optimal solution strategy, and lacks research on dynamic problems; and the traditional DQN model has limited stability when implementing the dynamic optimization bidding strategy of tasks in the auction mechanism. The present invention combines the DQN model with the advertiser strategy solution under the auction mechanism, optimizes the stability of the traditional DQN model, and realizes autonomous decision-making of tasks based on local information, thereby achieving decentralized and efficient resource allocation. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0057] Figure 1 A schematic diagram of a non-uniform memory access (NUMA) architecture in the prior art.
[0058] Figure 2 Flow chart of a non-uniform memory access resource allocation method based on a parallel adaptive auction algorithm in an embodiment.
[0059] Figure 3 It is a framework diagram of the parallel adaptive auction algorithm in an embodiment of the present invention.
[0060] Figure 4 Schematic diagram of parallel and concurrent computing in an embodiment of the present invention.
[0061] Figure 5 This is a comparison chart of the average computing time of the parallel adaptive auction algorithm in the embodiment of the present invention and the existing algorithm.
[0062] Figure 6 This is a comparison diagram of the acceleration between the parallel adaptive auction algorithm in the embodiment of the present invention and the existing algorithm. DETAILED DESCRIPTION
[0063] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.
[0064] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.
[0065] Some technical terms in this application are explained below. Other terms not explained here should be interpreted according to conventional technical terms in the art unless otherwise specified.
[0066] Non-Uniform Memory Access Architecture (NUMA): It can be abstracted as a knapsack problem with location constraints. It is a special scenario of the knapsack problem and is difficult to solve by other algorithms. Non-Uniform Memory Access Architecture (NUMA) is a computer memory design for multi-processor systems in which each processor or core has its own local memory. In the NUMA architecture, a processor accesses its own local memory faster than accessing the memory of other processors. This design is intended to improve the performance and scalability of multi-processor systems.
[0067] Auction algorithm / mechanism: It is a very important mechanism design in game theory; it has won the Nobel Prize in Economics many times; it is a method of mechanism design in itself, but this application borrows and optimizes it as a method of resource allocation rather than as a mechanism design.
[0068] Parallel auction mechanism: Based on the two characteristics of parallelism and self-adaptation, the auction mechanism is optimized, which is an algorithm improvement that adapts to the knapsack problem based on position constraints. Traditional dynamic programming and greedy algorithms are usually difficult to parallelize, mainly because of their algorithm structure and processing methods. However, the present invention no longer relies on traditional dynamic programming and other methods, but for the first time performs algorithm parallelization processing under the auction mechanism, which significantly improves the computing efficiency.
[0069] Parallel computing: A computing architecture in which multiple processors perform computing tasks simultaneously to increase computing speed and efficiency. This approach is often used to tackle large-scale, complex computing problems such as scientific simulations, big data analysis, and complex graphics processing. Data parallelism: In data parallelism, data sets are split into smaller parts and then processed in parallel; each processor performs the same operation on a different subset of the data. In parallel computing, a large problem is often broken down into many smaller parts that can be independently computed on different processors at the same time.
[0070] DQN (Deep Q-Network) model: It combines deep learning and reinforcement learning methods, and is mainly used to solve decision-making problems with high-dimensional input space. It is an important breakthrough in the field of reinforcement learning. The DQN model uses a deep neural network to approximate the Q-value function, which enables the algorithm to process high-dimensional input space and learn from complex environments.
[0071] Experience replay: DQN breaks the correlation between data by storing the experience of the task (state, action, reward, etc.) in a memory bank and randomly extracting samples from this bank during training, which helps improve learning stability and efficiency. Fixed Q target: To further improve the stability of learning, the DQN model uses two networks: the current Q network is used for current value estimation, and the target Q network is used for target value estimation; the parameters of the target Q network are regularly updated from the current Q network, which helps reduce the correlation between the target and estimated values.
[0072] In the NUMA architecture, optimizing memory access latency is the key to improving the overall performance of the system. Therefore, the solution to the knapsack problem with location constraints plays a vital role in the performance and energy efficiency of the system. The goal of this application is to allocate appropriate node resources to process different applications, thereby improving system performance. Based on this, this application proposes a resource allocation method that can maximize system performance while taking into account location constraints (i.e., reducing memory latency and ensuring that the CPU and memory performance within the node can support the execution of tasks), ensuring that processes with different performance requirements can be allocated to appropriate nodes, thereby optimizing system performance. Figure 2 As shown, the non-uniform memory access resource allocation method based on the parallel adaptive auction algorithm in this embodiment includes the following steps:
[0073] When the operating system starts, node information and task information in the non-uniform memory access architecture are obtained; wherein the node information includes the number of nodes, the node start position and the node end position; the task information includes the number of tasks and the number of nodes occupied by the tasks (i.e., the amount of resources occupied by the tasks);
[0074] The node allocation problem of non-uniform memory access architecture is regarded as a knapsack problem with location constraints, and a parallel adaptive auction algorithm is used to solve it to obtain the allocation result.
[0075] Furthermore, compared with uniform memory access (UMA), non-uniform memory access (NUMA) architecture has lower memory access latency because the processor accesses local memory nodes faster than remote nodes, thereby improving system performance. In order to make full use of this advantage, the present invention adopts an extended knapsack problem with location constraints. When allocating processes on nodes with different CPU performance and local memory capacity, the characteristics of local memory are considered to ensure that processes with different performance requirements can be allocated to appropriate nodes, thereby optimizing system performance. Therefore, based on the local memory advantages of the above-mentioned hardware architecture and the different performance requirements of processes, the goal of the present invention is to maximize the utilization of system resources in the NUMA architecture while satisfying location constraints. Therefore, the node allocation problem of the NUMA architecture is regarded as an extended knapsack problem with location constraints, which is expressed as:
[0076]
[0077] W i =(Ni) 0.5 , 0≤i <N,
[0078]
[0079] where(position i <position j +width j )and(position j <position i +width i ),
[0080] Among them, Score() is the objective function, which needs to maximize the score or score of the task. A high score means more efficient resource allocation; W i is the weight of the i-th task; width i is the amount of resources occupied by the ith task (in the context of the knapsack problem, width usually refers to the resources required for the task, such as time, space, bandwidth, etc., so in the constraint condition, the combined width of all selected tasks cannot exceed the knapsack capacity C); x i is the status of the i-th task, when x i =1 indicates that the i-th task is selected. i = 0 means that the i-th task is not selected; N is the number of tasks; C is the backpack capacity, that is, the maximum amount of resources that all selected tasks cannot exceed; position i is the location information of the i-th task, l i is the lower limit of the i-th task position, u i is the upper limit of the i-th task position; x j is the status of the jth task; position j is the location information of the jth task.
[0081] like Figure 3 As shown, in order to avoid the task with a larger width in the preceding process from being blocked due to the inability to obtain continuous nodes, the present invention assigns priorities in sequence, W i represents this weight; different processes have different weights to compete for nodes; of course, the memory and CPU performance required by processes of different tasks are also different, and the starting node is used for location constraints; the nodes required by these processes are also different, Figure 3 , which is represented by continuous blocks, thus reducing the memory latency between consecutive nodes. Optimizing memory access latency in NUMA architecture is critical to maximizing overall system performance. For example, Task 1 ( Figure 3Agent1) requires two nodes to complete. In the NUMA architecture, only nodes with starting positions 0 and 4 can be matched. Once the match is successful, the node will be occupied and cannot complete other tasks. The utilization of system resources in the NUMA architecture is measured by the total score Score(), which may seem boring, but it is a basic process.
[0082] Furthermore, for the constructed extended knapsack problem with location constraints, the present application uses the parallel adaptive auction (PAA-EKP) algorithm for optimization and solution. The core idea of the algorithm is to regard task allocation as a market competition process, and allocate each resource (i.e., a node in the NUMA architecture) to the bidder with the highest bid (i.e., a task in the NUMA architecture). This method realizes independent competition between tasks and promotes parallel processing; and bidders evaluate resources according to their own needs and submit bid prices calculated in parallel using local information; then each node simultaneously allocates resources to the highest bidder, thereby achieving high parallelism. The steps are:
[0083] Use the trained DQN model to select the best α value for each task that is suitable for the current environment and state, and use it to bid for the final starting position of the task; the best α value is used to adjust the bid price of each task in order to obtain a more advantageous position in resource allocation;
[0084] Heterogeneous multithreading is used to parallelly calculate the bid price of the starting position of each task for bidding and store it in the task list; the bid price calculation formula is: bid i =W i *α i , among which, bid i is the bid price of the i-th task, α i is the optimal α value for the i-th task;
[0085] The tasks are assigned based on the task list to obtain the final starting positions and the best α value list of all tasks.
[0086] Furthermore, the Deep Q-Network (DQN) is superior to heuristic and dynamic programming methods in complex environments and reduces computational complexity. By using neural networks for continuous state space approximation, the DQN model effectively handles discrete and continuous state spaces, making it more scalable in large-scale problems. In order to improve stability and reduce the spatiotemporal correlation between data, the DQN model of this application includes a current Q network and a target Q network with the same architecture but different parameters, as well as an experience replay buffer; wherein the current Q network is used to predict the Q value of each possible bidding action for each task in the current task state, and selects the bidding action with the highest Q value as the optimal bidding action in the current task state, responsible for selecting the optimal bidding action and calculating the loss of each time step; and the target Q network is used to calculate the target Q value in the next task state, which helps to improve the stability of learning; at the same time, the experience replay buffer is used to store the experience of the DQN model training process and update the parameters of the Q network.
[0087] Furthermore, the trained DQN model is used to select the best α value for each task that is suitable for the current environment and state, specifically:
[0088] The experience replay buffer stores the experience of a task during the training process in a data structure called the experience replay buffer. First, a small batch (mini-batch) of experience data is randomly extracted from the experience replay buffer. Each piece of experience data contains the task status, bidding action, reward of the task in the current time step, as well as the task status, bidding action and other information in the next time step. This method can break the time correlation between data, thereby improving the stability and efficiency of learning.
[0089] The action value function output by the target Q network, i.e., the Q value, is calculated for each task; this value represents the expected utility or return that the task can obtain by taking a specific bidding action in a given state; during the auction process, the task can use the target Q network in the trained DQN model to calculate the target Q value of the task, and then select the bidding action with the highest target Q value of the task as the best α value under the current task state, and find the optimal bidding strategy in the auction, thereby improving the resource allocation and task allocation results; the calculation formula of the target Q value is:
[0090] Q(state t ,action t )=reward t +γmax(action t+1 )*Q(state t+1 ,action t+1 θ - ),
[0091] Among them, Q(statet ,action t ) is the target Q value of the task at the current time step t, state t is the task status at the current time step t, action t is the bidding action of the task at the current time step t, reward t is the reward of the task at the current time step t; γ is the discount factor, which is used to balance the proportion of the reward of the task at the current time step and the reward of the next time step; Q(state t+1 ,action t+1 θ - ) is the task at the next time step t+1, parameter θ - The target Q value under max(action t+1 ) is the maximum target Q value predicted by the target Q network in the next task state; θ - is the parameter of the target Q network at the next time step t+1; state t+1 is the task status at the next time step t+1; action t+1 is the bidding action of the task at the next time step t+1.
[0092] During the training process, the parameters of the current Q network are continuously updated, and the parameters of the target Q network are periodically copied from the current Q network. In this embodiment, by using the mean square error (MSE) function as the loss function, the loss between the predicted Q value and the target Q value can be minimized, which is expressed as:
[0093]
[0094] δ i =Q(state t ,action t )-Q(state t ,action t ; θ),
[0095] Among them, δ i is the TD (Temporal Difference Error) error of the i-th experience, which represents the difference between the Q value predicted by the current Q network and the target Q value of the target Q network; θ is the parameter of the current Q network; Num ER is the amount of experience in the experience replay buffer; Q(state t ,action t ) is the target Q value output by the target Q network, Q(state t ,action t ; θ) is the predicted Q value output by the current Q network under parameter θ; statet is the task status at the current time step t, action t is the bidding action of the task at the current time step t.
[0096] In the traditional Q-Learning algorithm, the Q value update process involves the Q values of the current and next states, which may cause oscillations. The DQN model of this application decouples the Q value calculation from the network parameter update process by implementing the target Q network, thereby improving the stability of learning.
[0097] Furthermore, the present application also uses heterogeneous multithreading to parallelize the bid price of the starting position of each task for bidding, which cannot be done by concurrent computing; Figure 4 As shown in the figure, in concurrent computing, multiple tasks are performed alternately in the same time period, but in fact, only one task is executed at a time, and the other tasks are waiting. This method simulates multi-tasking on a single-core processor, but it does not significantly improve computing performance. Parallel computing greatly improves computing speed and performance by utilizing available resources. Heterogeneous multithreading can overcome the limitations of parallel computing and improve the performance of optimization algorithms, which is crucial for solving large-scale complex problems where time and resources are critical. The steps for heterogeneous multithreading parallel acceleration computing are as follows:
[0098] Initialize a task list tasks, a thread pool thread_pool and a shared result structure shared_results; the task list tasks contains all subtasks that need to be executed, that is, the bidding tasks after each task calculates the bid price of the starting position; the thread pool thread_pool is used to allocate threads to execute subtasks; the shared result structure shared_results is used to store the execution results of each subtask;
[0099] Allocate a heterogeneous thread async_thread from the thread pool to each subtask and execute the subtask, and store the execution result of the subtask in shared_results until all threads are executed;
[0100] Process the execution results of all subtasks in shared_results to generate the final result final_result.
[0101] Furthermore, after obtaining the task list, the tasks are assigned, specifically:
[0102] First, determine whether the task list is empty. If the task list is not empty, take out the task pattern with the highest bid price and the task starting position pos from the task list;
[0103] Determine whether the position range from the task start position pos to the task end position pos+grid[pattern][0] is covered, where grid[pattern][0] represents the amount of resources required by the task pattern;
[0104] If not covered, update the task’s final starting position final_positions[pattern] and coverage status covered;
[0105] If it has been covered, update the task list, remove the tasks that conflict with the current position, and re-stack the task list;
[0106] Repeat the above steps until all tasks are assigned, and return the final starting positions of all tasks final_positions.
[0107] In summary, the present invention is able to solve the extended knapsack problem with location constraints, which is essential for various applications, including operating systems, production scheduling, and especially online markets, where efficient resource allocation is crucial for economic optimization. The present invention proposes an innovative and efficient parallel adaptive auction (PAA-EKP) algorithm to solve this problem, combining the concept of non-uniform memory access (NUMA) architecture and task scheduling for analysis and experiments. In view of the problem that traditional optimization methods involve sequential dependencies, which makes it difficult to decompose the main problem into independent parallel computing subtasks, the present invention strategically optimizes the bid price through a robust DQN model; and the bid prices can be calculated and submitted in parallel, which speeds up the auction allocation process. The results show that in large-scale tasks involving a large number of tasks, the present invention improves computing performance by 20 times over dynamic programming and improves resource allocation by 19%. As Figure 5 , Figure 6As shown, this embodiment uses the PAA-EKP algorithm proposed in this application to conduct comparative tests with existing algorithms, and uses average computing time and acceleration to verify the performance of each method. It can be seen that the PAA-EKP algorithm proposed in the present invention is significantly better than other algorithms. Among them, the multi-threaded and single-threaded situations of the PAA-EKP algorithm are experimentally compared. In addition, it is compared with the commonly used greedy algorithm (Greedy), first adaptation algorithm (First Fit) and dynamic programming algorithm (Dynamic Programming) in terms of average computing time (Average Computetional Time) and acceleration performance (Speedup Factor), reflecting the performance improvement of different algorithms under the same conditions, and verifying the significant performance advantages of the PAA-EKP algorithm proposed in the present invention. It can be seen that the multi-threaded PAA-EKP algorithm has obvious computing efficiency and performance improvement when processing large-scale tasks.
[0108] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.
[0109] Based on the same idea as the non-uniform memory access resource allocation method based on a parallel adaptive auction algorithm in the above-mentioned embodiment, the present invention also provides a non-uniform memory access resource allocation system based on a parallel adaptive auction algorithm, which can be used to execute the above-mentioned non-uniform memory access resource allocation method based on a parallel adaptive auction algorithm. For ease of explanation, the structural diagram of the embodiment of the non-uniform memory access resource allocation system based on a parallel adaptive auction algorithm only shows the parts related to the embodiment of the present invention. Those skilled in the art can understand that the illustrated structure does not constitute a limitation on the device, and may include more or fewer components than shown in the diagram, or combine certain components, or arrange the components differently.
[0110] Another embodiment of the present invention provides a non-uniform memory access resource allocation system based on a parallel adaptive auction algorithm, including an information acquisition module and a solution allocation module;
[0111] The information acquisition module is used to acquire node information and task information in the non-uniform memory access architecture when the operating system is started; wherein the node information includes the number of nodes, the node start position and the node end position; the task information includes the number of tasks and the number of nodes occupied by the tasks;
[0112] The allocation solution module is used to regard the resource problem of the non-uniform memory access architecture as a knapsack problem with location constraints, and uses a parallel adaptive auction algorithm to optimize and solve it to obtain the allocation result.
[0113] It should be noted that the technical features and beneficial effects described in the above-mentioned embodiment of the non-uniform memory access resource allocation method based on the parallel adaptive auction algorithm are applicable to the embodiment of the non-uniform memory access resource allocation system based on the parallel adaptive auction algorithm. For specific contents, please refer to the description in the embodiment of the method of the present invention, which will not be repeated here. It is hereby declared. In addition, in the implementation of the non-uniform memory access resource allocation system based on the parallel adaptive auction algorithm in the above-mentioned embodiment, the logical division of each program module is only an example. In actual application, according to needs, for example, for the configuration requirements of the corresponding hardware or the convenience of software implementation, the above-mentioned function allocation can be completed by different program modules, that is, the internal structure of the non-uniform memory access resource allocation system based on the parallel adaptive auction algorithm is divided into different program modules to complete all or part of the functions described above.
[0114] In one embodiment, a computer-readable storage medium is provided, which stores a program in a memory. When the program is executed by a processor, a non-uniform memory access resource allocation method based on a parallel adaptive auction algorithm is implemented, specifically:
[0115] When the operating system starts, node information and task information in the non-uniform memory access architecture are obtained; wherein the node information includes the number of nodes, the starting position of the nodes, and the ending position of the nodes; the task information includes the number of tasks and the number of nodes occupied by the tasks;
[0116] The resource problem of non-uniform memory access architecture is regarded as a knapsack problem with location constraints, and a parallel adaptive auction algorithm is used to optimize and solve it to obtain the allocation result.
[0117] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0118] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0119] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.
Claims
1. A non-uniform memory access resource allocation method based on a parallel adaptive auction algorithm, characterized in that: The steps include: When the operating system is started, node information and task information in the non-uniform memory access architecture are obtained; the node information includes the number of nodes, the node start position and the node end position; the task information includes the number of tasks and the number of nodes occupied by the tasks; The resource problem of non-uniform memory access architecture is regarded as a knapsack problem with location constraints, and a parallel adaptive auction algorithm is used to optimize and solve it to obtain the allocation result.
2. The non-uniform memory access resource allocation method based on the parallel adaptive auction algorithm according to claim 1, characterized in that: The resource allocation problem of the non-uniform memory access architecture is regarded as an extended knapsack problem with location constraints, which can be expressed as: W i =(N-i) 0.5 ,0≤i<N, where(position i <position j +width j )and(position j <position i +width i ), Among them, Score() is the objective function; W i is the weight of the i-th task; width i The amount of resources occupied by the i-th task; x i is the status of the i-th task, when x i =1 indicates that the i-th task is selected. i = 0 means that the i-th task is not selected; N is the number of tasks; C is the backpack capacity, that is, the maximum amount of resources that all selected tasks cannot exceed; position i is the location information of the i-th task, l i is the lower limit of the i-th task position, u i is the upper limit of the i-th task position; x j is the status of the jth task; position j is the location information of the jth task.
3. The non-uniform memory access resource allocation method based on parallel adaptive auction algorithm according to claim 2, characterized in that: The parallel adaptive auction algorithm is used to optimize and solve the problem, and the process is as follows: Use the trained DQN model to select the best α value for each task that is suitable for the current environment and state; the best α value is used to adjust the bid price for each task; Heterogeneous multithreading is used to parallely calculate the bid price of the starting position of each task for bidding and store it in the task list; The bid price calculation formula is: i =W i *α i , among which, bid i is the bid price of the i-th task, α i is the optimal α value for the i-th task; The tasks are assigned based on the task list to obtain the final starting positions and the best α value list of all tasks.
4. The non-uniform memory access resource allocation method based on parallel adaptive auction algorithm according to claim 3, characterized in that: The DQN model includes a current Q network and a target Q network with the same architecture but different parameters, and an experience replay buffer; The current Q network is used to predict the Q value of each possible bidding action of each task under the current task state, and select the bidding action with the highest Q value as the optimal bidding action under the current task state; The target Q network is used to calculate the target Q value in the next task state, and the parameters of the target Q network are periodically copied from the current Q network; The experience replay buffer is used to store the experience of the DQN model training process and update the parameters of the current Q network.
5. The non-uniform memory access resource allocation method based on parallel adaptive auction algorithm according to claim 4, characterized in that: The trained DQN model is used to select the best α value for each task that is suitable for the current environment and state, specifically: Randomly extract a small batch of experience data from the experience replay buffer; each experience data contains the task status, bidding action, reward of the task at the current time step, as well as the task status and bidding action of the next time step; The target Q network in the trained DQN model is used to calculate the target Q value of the task, and the bidding action with the highest target Q value of the task is selected as the optimal α value in the current task state; the calculation formula of the target Q value is: Q(state t ,action t )=reward t +γmax(action t+1 )*Q(state t+1 ,action t+1 ;θ - ), Among them, Q(state t ,action t ) is the target Q value of the task at the current time step t, state t is the task status at the current time step t, action t is the bidding action of the task at the current time step t, reward t is the reward of the task at the current time step t; γ is the discount factor, which is used to balance the proportion of the reward of the task at the current time step and the reward of the next time step; Q(state t+1 ,action t+1 θ - ) is the task at the next time step t+1, parameter θ - The target Q value under max(action t+1 ) is the maximum target Q value predicted by the target Q network in the next task state; θ - is the parameter of the target Q network at the next time step t+1; state t+1 is the task status at the next time step t+1; action t+1 is the bidding action of the task at the next time step t+1.
6. The non-uniform memory access resource allocation method based on parallel adaptive auction algorithm according to claim 4, characterized in that: The DQN model uses the mean square error function as the loss function, which is expressed as: δ i =Q(state t ,action t )-Q(state t ,action t ;θ), Among them, δ i is the TD error of the i-th experience, which represents the difference between the Q value predicted by the current Q network and the target Q value of the target Q network; θ is the parameter of the current Q network; Num ER is the amount of experience in the experience replay buffer; Q(state t ,action t ) is the target Q value output by the target Q network, Q(state t ,action t ; θ) is the predicted Q value output by the current Q network under parameter θ; state t is the task status at the current time step t, action t is the bidding action of the task at the current time step t.
7. The non-uniform memory access resource allocation method based on parallel adaptive auction algorithm according to claim 3, characterized in that: The method of using heterogeneous multithreading to parallelly calculate the bid price of the starting position of each task for bidding is as follows: Initialize a task list tasks, a thread pool thread_pool and a shared result structure shared_results; the task list tasks contains all subtasks that need to be executed, that is, the bidding tasks after each task calculates the bid price of the starting position; the thread pool thread_pool is used to allocate threads to execute subtasks; the shared result structure shared_results is used to store the execution results of each subtask; Allocate a heterogeneous thread async_thread from the thread pool to each subtask and execute the subtask, and store the execution result of the subtask in shared_results until all threads are executed; Process the execution results of all subtasks in shared_results to generate the final result final_result.
8. The non-uniform memory access resource allocation method based on parallel adaptive auction algorithm according to claim 3, characterized in that: The task allocation based on the task list is specifically as follows: When the task list is not empty, take out the task pattern with the highest bid price and the task starting position pos from the task list; Determine whether the position range from the task start position pos to the task end position pos+grid[pattern][0] is covered, where grid[pattern][0] represents the amount of resources required by the task pattern; If not covered, update the task’s final starting position final_positions[pattern] and coverage status covered; If it has been covered, update the task list, remove the tasks that conflict with the current position, and re-stack the task list; Repeat the above steps until all tasks are assigned, and return the final starting positions of all tasks final_positions.
9. A non-uniform memory access resource allocation system based on a parallel adaptive auction algorithm, characterized in that: The non-uniform memory access resource allocation method based on the parallel adaptive auction algorithm as applied to any one of claims 1 to 8, wherein the system comprises an information acquisition module and a solution allocation module; The information acquisition module is used to acquire node information and task information in the non-uniform memory access architecture when the operating system is started; the node information includes the number of nodes, the node start position and the node end position; the task information includes the number of tasks and the number of nodes occupied by the tasks; The allocation solution module is used to regard the resource problem of the non-uniform memory access architecture as a knapsack problem with position constraints, and adopt a parallel adaptive auction algorithm to optimize and solve it to obtain an allocation result.
10. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, the non-uniform memory access resource allocation method based on a parallel adaptive auction algorithm as described in any one of claims 1 to 8 is implemented.
Citation Information
Patent Citations
Sending routing data based on times that servers joined a cluster
CN101300796A
Mobile edge network resource allocation method based on online combined auction
CN110996396A
Internet of vehicles edge computing task unloading method based on hierarchical reinforcement learning
CN113568675A
User association and power distribution method and system in heterogeneous network
CN116528248A
Method, device and system for solving combinatorial optimization problem
CN117828966A
Cited By
Heterogeneous resource scheduling cloud optimization method and system based on space-time task intensity evaluation
CN120104281A
