A FPGA task scheduling optimization method and system using DQN

Abstract

The present invention discloses an FPGA task scheduling optimization method and system using DQN, defines a non-uniformly distributed FPGA on-chip reconfigurable resource model, models CLB, BRAM and DSP resources, uses the modeling results as the placement chip carrier of the online task to characterize the online task, uses the optional modeling results of the online task as the target chip, uses the fixed-length data defined by the online task and the target chip as the input data of the DQN network, uses the reward function of DQN as the feedback of the neural network, and updates it with the DQN update value function; uses DQN and the optimal longest fit algorithm to jointly train the DQN network; finds a suitable target chip and the optimal placement result in each scheduling event, inputs the online task and the target chip into the trained DQN network, selects the target chip with the highest reward value as the running chip of the corresponding task, and realizes task scheduling optimization. The present invention improves the scalability of the model by fixed-length input.

Description

Technical Field

[0001] This invention belongs to the field of electronic technology and neural network inference computing technology, specifically relating to an FPGA task scheduling optimization method and system using DQN. Background Technology

[0002] Compared to Application Specific Integrated Circuits (ASICs), which have fixed functions and lack the ability to dynamically reconfigure on-chip features, Field Programmable Gate Arrays (FPGAs) achieve real-time dynamic reconfiguration of chip functions by dynamically reprogramming on-chip resources at runtime. This allows for uninterrupted online circuit updates and offers high flexibility. Therefore, FPGAs are widely used in data center acceleration, software-defined networking, large-scale network switching, autonomous driving in automobiles, and IoT computing.

[0003] The resource types and locations on an FPGA chip are fixed. Each module (task) of the designed circuit is placed in a different chip location according to the placement and routing algorithm, occupying the resources of that location to achieve its respective function. Based on the determinacy of the number and scale of tasks, tasks running on an FPGA chip can be divided into two types:

[0004] 1) Offline tasks: The number and scale of these tasks are determined before running. For this type of task, the placement and routing algorithm should prioritize resource consumption and place the tasks as compactly as possible. Since the number of tasks is determined before running, in addition to heuristic algorithms, more complex and time-consuming optimization placement and routing algorithms can be selected to obtain better placement results without worrying about task placement and routing failures due to excessive algorithm time consumption.

[0005] 2) Online tasks: The arrival time of these tasks is unknown. Therefore, unlike offline tasks, these tasks should use simple and fast placement and routing algorithms to avoid the online tasks missing the latest execution time, resulting in task placement and routing failure and affecting the function of the entire circuit.

[0006] Meanwhile, with the rapid development of application fields, a single FPGA can no longer meet the circuit scale requirements, and multi-FPGA collaborative work has become the mainstream. How to select FPGA chips, set reasonable task scheduling strategies, place online tasks on appropriate FPGA chips, and make reasonable layouts after selecting target chips is an NP-hard problem.

[0007] Therefore, a reasonable multi-FPGA online task scheduling strategy and on-chip placement and routing algorithm are key to reducing on-chip resource waste and reducing the total task time. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide an FPGA task scheduling optimization method and system using DQN to address the shortcomings of the prior art. This method determines a suitable scheduling order and FPGA chip select strategy for each FPGA online task, thereby effectively improving FPGA resource utilization and reducing the total task time.

[0009] The present invention adopts the following technical solution:

[0010] An FPGA task scheduling optimization method using DQN is characterized by defining a non-uniformly distributed on-chip reconfigurable resource model for FPGA, modeling CLB, BRAM and DSP resources, and using the modeling results as the chip carrier for placing online tasks.

[0011] The online task is characterized and represented. The optional modeling results of the online task are used as the target chip. The fixed-length data defined by the online task and the target chip are used as the input data of the DQN network. The reward function of DQN is used as the feedback of the neural network. The reward function is updated by updating the value function of DQN as the feedback of each step. The DQN network is jointly trained using DQN and the optimal longest fit algorithm.

[0012] In each scheduling event, a suitable target chip and the optimal placement result are found. The online task and the target chip are input into the trained DQN network, and the target chip with the highest reward value is selected as the running chip for the corresponding task, thereby achieving task scheduling optimization.

[0013] Specifically, the modeling of CLB, BRAM, and DSP resources involves the following steps:

[0014] The on-chip resources of the FPGA are abstracted into a two-dimensional matrix, with the vertical axis in the clock domain and the horizontal axis in the minimum available resource. According to the different resource types represented, the values ​​in the two-dimensional matrix are defined as 1, 2, and 3, respectively, representing CLB, BRAM, and DSP resources.

[0015] Specifically, the online task set is as follows:

[0016] T = {id,clb,bram,dsp,w,h,t} a ,t s ,t d}

[0017] Where id is the task type marker, clb, bram, and dsp represent the number of CLB, BRAM, and DSP resources required for the task, respectively, w and h are the width and height of the rectangle corresponding to the task, and t a t is the time when the task arrives. st represents the latest execution time of the task. d This refers to the runtime.

[0018] Specifically, the target chip is F = {F1, F2, ..., F...} n}; For any FPGA F i F, i∈{1,2,...,n} i ={Cell[w][h],T i,1 ,T i,2 ,...,T i,m The two-dimensional array Cell takes values ​​1, 2, and 3, corresponding to the model of on-chip resources on the FPGA; T i,k ,k∈{1,2,...,m} indicates that there are m tasks placed on FPGA i; for any one of these tasks, T i,k ∈T.

[0019] Specifically, the fixed-length data is the state space of DQN fixed-length multi-batch input data based on FPGA-online task pairs.

[0020] Specifically, the DQN reward function is:

[0021]

[0022] Where α, β, and γ are constants; covered res (t) and needed res (t) represents the amount of resources occupied and required by all tasks deployed on the target chip at time t, respectively; the adhesion(t) function represents the size of the internal fragmentation of the target chip's resources at time t; res(t) is the amount of resources on the FPGA chip; and r(t) is the benefit of the current decision.

[0023] Specifically, in order to maximize cumulative rewards, the state-action pair (s) is... t The value function of (a) is defined as follows: in state s t The expected value of the reward function after performing action a:

[0024] Q π (s t ,a)=E[R(t)|s t ,a]=E[r(t)+Q π (s t+1 ,a′)|s t ,a]

[0025] To maximize the cumulative reward function, we need to find the optimal value function.

[0026]

[0027] in, Let E be the maximum payoff that can be obtained at time t+1 after taking action a, and let r(t) be the payoff of the current decision. t Let 'a' be the state at time t, and 'a' be the action at time t.

[0028] Furthermore, we introduce the definition of cumulative reward R(t), which represents the sum of rewards from the current time t to the future, as follows:

[0029]

[0030] Where T is the time taken to schedule all online tasks, t is the scheduling time of each task, and r(t) is the benefit of the current decision.

[0031] Specifically, DQN updates the value function Q(s) t a) is:

[0032]

[0033] Where r(t) is the payoff of the current decision, and γ is the discount factor. For the benefits of memory.

[0034] Another technical solution of the present invention is an FPGA task scheduling optimization system employing DQN, comprising:

[0035] Select modules, define a non-uniformly distributed on-chip reconfigurable resource model for FPGA, model CLB, BRAM and DSP resources, and use the modeling results as the chip carrier for placing online tasks.

[0036] The training module performs feature representation on the online task, uses the optional modeling results of the online task as the target chip, and uses the fixed-length data defined by the online task and the target chip as the input data of the DQN network. The reward function of DQN is used as the feedback of the neural network, and the reward function is updated by using the updated value function of DQN as the feedback at each step. The DQN network is jointly trained using DQN and the optimal longest fit algorithm.

[0037] The optimization module finds the appropriate target chip and the optimal placement result in each scheduling event. It inputs the online task and the target chip into the trained DQN network and selects the target chip with the highest reward value as the running chip for the corresponding task, thereby optimizing task scheduling.

[0038] Compared with the prior art, the present invention has at least the following beneficial effects:

[0039] This invention discloses an FPGA task scheduling optimization method using DQN, which can place each FPGA task on the most suitable target chip for execution, resulting in the shortest running time and the least resource consumption. The placement strategy uses a DQN deep Q-value neural network to score (reward value) all candidate FPGA chips and selects the chip with the highest score to run an online task on a certain FPGA. This solves the problem of low efficiency in optimizing online tasks with multiple FPGAs, which leads to waste of on-chip resources and affects the success rate of task execution.

[0040] Furthermore, the advantages of modeling CLB, BRAM, and DSP resources are: 1. Current placement methods for FPGA online tasks only consider the single CLB resource. This invention considers the three resources of CLB, BRAM, and DSP on-chip in current advanced mainstream FPGAs, thus having a wider range of application scenarios and value. 2. By using mathematical modeling, the complexity of chip modeling is reduced, the efficiency of placement algorithms is improved, and chip selection and placement time is reduced.

[0041] Furthermore, as input to DQN, the online task set is defined mathematically for more accurate and efficient processing.

[0042] Furthermore, an optional FPGA can be used as a runnable chip for a certain online task. The purpose of this setting is to compare the reward function values ​​of different chips, and finally, the placement algorithm selects the FPGA chip with the highest reward value from the optional FPGAs for placement.

[0043] Furthermore, fixed-length input solves the problem of the explosion of neural network input dimensions caused by the excessive number of FPGAs; it also solves the problem of increased training difficulty caused by a large amount of useless information input into the neural network; and the problem of poor scalability of scheduling strategies caused by the fixed length of neural network input vectors.

[0044] Furthermore, the reward function determines the placement of the chip for each task; the larger the reward function, the higher the score. The chip with the highest score among the candidate FPGAs is selected for placement.

[0045] Furthermore, the placement of each task does not only consider the chip select result in the current state, but also, based on past experience, DQN calculates the total reward value of several subsequent states through expectation, and selects the chip select result with the largest total reward.

[0046] Furthermore, DQN calculates the total reward value for several subsequent states based on expectations, and selects the slice selection result with the largest total reward.

[0047] Furthermore, the purpose of updating the value function is to make DQN more intelligent, and the continuous learning process of the neural network can make predictions more accurate.

[0048] In summary, this invention addresses the problem of wasted on-chip resources due to low optimization efficiency of multi-FPGA placement online tasks, which in turn affects the success rate of task execution. It not only considers the optimal placement scheduling in the current state, but also the possible maximum benefits in several future states, and improves the scalability of the model by using a fixed-length input method.

[0049] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of the on-chip resources of the FPGA;

[0051] Figure 2 A schematic diagram illustrating the numerical definition of on-chip resources in an FPGA;

[0052] Figure 3 This is a diagram illustrating the input data.

[0053] Figure 4 This is a flowchart of the scheduling strategy. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] In the description of this invention, it should be understood that the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0056] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0057] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0058] The accompanying drawings illustrate various structural schematic diagrams according to embodiments disclosed in this invention. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0059] This invention discloses an FPGA task scheduling optimization method using DQN, comprising the following steps:

[0060] S1. Define a non-uniformly distributed on-chip reconfigurable resource model for FPGAs;

[0061] FPGA chips contain various resources, including CLBs, BRAMs, and DSPs. Currently, scheduling and placement strategies for online tasks only consider CLBs. While BRAMs and DSPs are fewer in number compared to the abundant CLBs, and the number of tasks requiring these specialized resources is also relatively small, if a task with such special requirements arises and the scheduling and placement strategy does not consider allocating these special resources, additional CLB resources will be needed to complete the logic configuration. This prevents the full utilization of the chip's hardware resources, leading to a decrease in FPGA acceleration performance.

[0062] Therefore, in order to solve the problem of ineffective utilization of special on-chip resources, this invention defines an FPGA on-chip reconfigurable resource model. The FPGA on-chip reconfigurable resource model can model CLB, BRAM and DSP resources to meet the different needs of online tasks for the three types of resources, thereby improving resource utilization.

[0063] The specific modeling method is as follows:

[0064] S101. Abstract the FPGA on-chip resources into a two-dimensional matrix, with the vertical axis in clock region units and the horizontal axis in a minimum available resource unit. For example... Figure 1 The FPGA chip shown can be abstracted into a 4*21 two-dimensional matrix.

[0065] S102. Based on the different resource types represented, the values ​​in the two-dimensional matrix are defined as 1, 2, and 3, respectively, representing CLB, BRAM, and DSP resources. Figure 1 The FPGA on-chip resources shown are available Figure 2 express.

[0066] The modeling method of this invention represents the three types of FPGA on-chip resources in a two-dimensional matrix. This method enables precise management of all resources and facilitates the scheduling and placement of subsequent tasks.

[0067] S2. FPGA Task Scheduling Method Based on Deep Q Network (DQN) Deep Reinforcement Learning

[0068] When multiple FPGAs work collaboratively, different tasks can run on different FPGA devices, providing higher task processing capabilities but also making task scheduling more complex. Deep Reinforcement Learning (DRL) adopts a data-driven approach, making it suitable for complex system control. Among many DRL methods, value-based DRL methods, although having a slower convergence speed, are more likely to converge to the global optimum. Furthermore, the training data in this invention is trained offline, so the convergence speed does not affect the actual scheduling algorithm performance. Therefore, this invention uses value-based DRL: Deep Q Network to implement an online task scheduling strategy for FPGAs, such as... Figure 4 As shown.

[0069] S201. Characterize the online task;

[0070] This invention defines: an online task set can be T = {id,clb,bram,dsp,w,h,t} a ,t s ,t d} represents the task type identifier, where id is the task type marker, clb, bram, and dsp represent the number of CLB, BRAM, and DSP resources required for the task, respectively, w and h are the width and height of the rectangle corresponding to the task, and t is the length of the rectangle. a t is the time when the task arrives. s t represents the latest execution time of the task. d This refers to the runtime.

[0071] S202, Use the task definition in step S201 to select the FPGA;

[0072] This invention defines: the number of FPGAs in the system is n, and all selectable FPGAs are represented as: F = {F1, F2, ..., F...} n}, for any one of the FPGAs F i F, i∈{1,2,...,n} i ={Cell[w][h],T i,1 ,T i,2 ,...,T i,m}, where the two-dimensional array Cell takes values ​​1, 2, and 3, corresponding to the model of FPGA on-chip resources in this invention; T i,k ,k∈{1,2,...,m} indicates that there are m tasks placed on FPGA i; for any one of these tasks, T i,k ∈T.

[0073] S203, Define fixed-length input data;

[0074] To address the weakness in scheduling strategy scalability caused by the fixed length of DQN input data, this invention defines a DQN fixed-length multi-batch input data state space based on "FPGA-online task" pairs.

[0075] By using this method, the paired grouping of tasks and optional FPGAs in steps S201 and S202 is input into the DQN, solving the following problems:

[0076] 1) The problem of neural network input dimension explosion caused by an excessive number of FPGAs;

[0077] 2) The problem of increased training difficulty caused by a large amount of useless information being input into the neural network;

[0078] 3) The scheduling strategy has poor scalability due to the fixed length of the neural network input vector.

[0079] Specific input data is as follows: Figure 3 As shown, it consists of a task to be run and an optional FPGA that meets the various resources required for that task.

[0080] S204. Design the DQN reward function;

[0081]

[0082] Where α, β, and γ are constants; covered res (t) and needed res The functions `adhesion(t)` and `res(t)` represent the total resources used and required by all tasks deployed on the FPGA chip at time `t`, respectively. `adhesion(t)` represents the internal fragmentation size of the FPGA chip's resources at time `t`. `res(t)` represents the total number of resources on the FPGA chip. The three parts of the reward function are defined such that tasks using fewer resources, tasks being more closely spaced, and tasks having a shorter overall execution time result in a higher reward value.

[0083] Joint training of S205, DQN and the optimal longest fit algorithm.

[0084] The FPGA chip select scheduling strategy (action a) is based on DQN, while the specific chip placement algorithm adopts the optimal longest fit algorithm. The reward for step S204 is calculated based on the placement result. Therefore, a joint training method is used to find the appropriate target FPGA chip and the optimal placement result in each scheduling event.

[0085] Each time DQN makes a scheduling decision, the placement algorithm provides a reward r(t), the formula for which r(t) is calculated as shown in step S204. The goal of DQN is to maximize the sum of rewards. Assuming the total time taken to complete all tasks is T, the objective of DQN is expressed in the following form:

[0086]

[0087] To maximize the sum of rewards, it is clear that we need to maximize the future rewards at each time step. Therefore, we introduce the definition of cumulative reward R(t), which represents the sum of rewards from the current time t to the future, as follows:

[0088]

[0089] Furthermore, to maximize cumulative rewards, the state-action pair (s) is... t The value function of (a) is defined as follows: in state s t Expected value of the reward function after performing action a

[0090] Q π (s t ,a)=E[R(t)|s t ,a]=E[r(t)+Q π (s t+1 ,a′)|s t ,a]

[0091] To maximize the cumulative reward function, we need to find the optimal value function.

[0092]

[0093] In deep reinforcement learning, the fitting of the most valuable function is done using a neural network approach; DQN updates the value function Q(s). t a)

[0094]

[0095] In this way, when performing FPGA chip selection, the DQN update of the value function consists of two parts: the payoff r(t) of the current decision and the payoff in memory. Based on past experience, the payoff at time t+1 is recorded and influences the action 'a' at time t. Here, γ represents the discount factor; as γ increases, the value function places greater emphasis on past experience, while the payoff proportion of the current state decreases.

[0096] In another embodiment of the present invention, an FPGA task scheduling optimization system using DQN is provided. This system can be used to implement the above-mentioned FPGA task scheduling optimization method using DQN. Specifically, the FPGA task scheduling optimization system using DQN includes a selection module, a training module, and an optimization module.

[0097] Among them, the module is selected, the on-chip reconfigurable resource model of non-uniformly distributed FPGA is defined, the CLB, BRAM and DSP resources are modeled, and the modeling results are used as the chip carrier for placing online tasks.

[0098] The training module performs feature representation on the online task, uses the optional modeling results of the online task as the target chip, and uses the fixed-length data defined by the online task and the target chip as the input data of the DQN network. The reward function of DQN is used as the feedback of the neural network, and the reward function is updated by using the updated value function of DQN as the feedback at each step. The DQN network is jointly trained using DQN and the optimal longest fit algorithm.

[0099] The optimization module finds the appropriate target chip and the optimal placement result in each scheduling event. It inputs the online task and the target chip into the trained DQN network and selects the target chip with the highest reward value as the running chip for the corresponding task, thereby optimizing task scheduling.

[0100] In another embodiment of the present invention, a terminal device is provided, comprising a processor and a memory. The memory stores a computer program, the computer program including program instructions, and the processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to implement a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used for operation of an FPGA task scheduling optimization method employing DQN, including:

[0101] A non-uniformly distributed on-chip reconfigurable resource model for FPGAs is defined, and CLB, BRAM, and DSP resources are modeled. The modeling results are used as the placement chip carriers for online tasks. The online tasks are characterized and represented. The modeling results of the online tasks are used as target chips. Fixed-length data defined by the online tasks and target chips are used as input data for the DQN network. The DQN reward function is used as the feedback of the neural network, and the DQN update value function is used as the feedback to update the reward function at each step. The DQN network is jointly trained using DQN and the optimal longest fit algorithm. In each scheduling event, a suitable target chip and the optimal placement result are found. The online task and target chip are input into the trained DQN network, and the target chip with the highest reward value is selected as the running chip for the corresponding task, thereby achieving task scheduling optimization.

[0102] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory). This computer-readable storage medium is a memory device in a terminal device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and extended storage media supported by the terminal device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device.

[0103] One or more instructions stored in a computer-readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the FPGA task scheduling optimization method using DQN in the above embodiments; one or more instructions in the computer-readable storage medium are loaded and executed by the processor in the following steps:

[0104] A non-uniformly distributed on-chip reconfigurable resource model for FPGAs is defined, and CLB, BRAM, and DSP resources are modeled. The modeling results are used as the placement chip carriers for online tasks. The online tasks are characterized and represented. The modeling results of the online tasks are used as target chips. Fixed-length data defined by the online tasks and target chips are used as input data for the DQN network. The DQN reward function is used as the feedback of the neural network, and the DQN update value function is used as the feedback to update the reward function at each step. The DQN network is jointly trained using DQN and the optimal longest fit algorithm. In each scheduling event, a suitable target chip and the optimal placement result are found. The online task and target chip are input into the trained DQN network, and the target chip with the highest reward value is selected as the running chip for the corresponding task, thereby achieving task scheduling optimization.

[0105] Experimental results

[0106] Using the method proposed in this invention, the experimental task scale includes 360 tasks, with randomized task size, execution time, and arrival time. The target FPGA quantity is 3, and the same placement strategy is employed. Compared with a sequential chip select scheduling strategy, this method improves the task success rate by 23.34%.

[0107] In summary, this invention presents an FPGA task scheduling optimization method and system using DQN. Addressing the problem of low on-chip resource waste caused by low optimization efficiency when multiple FPGAs are deployed in online tasks, thus affecting task success rates, this invention proposes a scheduling strategy using deep reinforcement learning across multiple FPGAs. This effectively improves the success rate of online task execution and significantly reduces the total task execution time, thereby saving hardware overhead and improving chip utilization.

[0108] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0109] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0110] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0111] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0112] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A FPGA task scheduling optimization method using DQN, characterized in that: Define a non-uniformly distributed FPGA on-chip reconfigurable resource model, model CLB, BRAM and DSP resources, and use the modeling results as the chip carrier for placing online tasks; characterize online tasks, use the optional modeling results of online tasks as the target chip, use the fixed-length data defined by the online tasks and the target chip as the input data of the DQN network, use the DQN reward function as the feedback of the neural network, and use the DQN update value function as the feedback of each step to update the reward function; use DQN and the optimal longest fit algorithm to jointly train the DQN network; find the appropriate target chip and the optimal placement result in each scheduling event, input the online task and the target chip into the trained DQN network, and select the target chip with the highest reward value as the running chip for the corresponding task to achieve task scheduling optimization; Modeling of CLB, BRAM and DSP resources is as follows: The FPGA on-chip resources are abstracted into a two-dimensional matrix, with the vertical axis in clock domains and the horizontal axis in a minimum available resource. According to the different resource types represented, the values ​​in the two-dimensional matrix are defined as 1, 2, and 3, representing CLB, BRAM, and DSP resources, respectively. The online task set is: T={id,clb,bram,dsp,w,h,t a ,t s ,t d } Among them, id is the task type tag placed, clb, bram and dsp represent the number of CLB, BRAM and DSP resources required by the task respectively, w and h are the width and height of the rectangular shape corresponding to the task, t a is the task arrival time, t s is the latest execution time of the task, t d is the running time; The target chip is F = {F1, F2, ..., F n }; For any FPGAF i ,i∈{1,2,...,n} has F i ={Cell[w][h],T i,1 ,T i,2 ,...,T i,m }, the two-dimensional array Cell takes values ​​1, 2, 3, corresponding to the model of FPGA on-chip resources; T i,k , k∈{1,2,...,m} means that there are m tasks placed on FPGA i; for any of the tasks, there is T i,k ∈T; The fixed-length data is the fixed-length multi-batch input data state space of DQN based on FPGA-online task pairs; The DQN reward function is: Among them, α, β and γ are constants; covered res (t) and needed res (t) respectively represent the number of resources occupied and the number of resources required by all tasks deployed on the target chip at time t; the adhesion(t) function represents the size of the internal resource fragmentation of the target chip at time t, res(t) is the number of resources on the FPGA chip, and r(t) is the benefit of the current decision.

2. The method according to claim 1, characterized in that In order to maximize the cumulative reward, the state-action pair (s t ,a) is defined as the value function in state s t The expectation of the reward function after executing action a is: Q π (s t ,a)=E[R(t)|s t ,a]=E[r(t)+Q π (s t+1 ,a′)|s t ,a] Solve the optimal value function based on maximizing the cumulative reward function: in, is the maximum benefit value that can be obtained at time t+1 after taking action a, E is the expectation, r(t) is the benefit of the current decision, s t is the state at the current time t, and a is the action at the current time t.

3. The method according to claim 2, characterized in that The definition of cumulative reward R(t) is introduced to represent the sum of rewards from the current time t to the future as follows: Among them, T is the time taken to complete the scheduling of all online tasks, t is the scheduling time of each task, and r(t) is the benefit of the current decision.

4. The method according to claim 1, characterized in that DQN updates the value function Q(s t ,a) is: Among them, r(t) is the benefit of the current decision, γ is the discount factor, For the benefit of memory.

5. An FPGA task scheduling optimization system using DQN, characterized in that: include: Select modules, define the non-uniformly distributed FPGA on-chip reconfigurable resource model, model CLB, BRAM and DSP resources, and use the modeling results as the chip carrier for placing online tasks; Modeling of CLB, BRAM and DSP resources is as follows: The FPGA on-chip resources are abstracted into a two-dimensional matrix, with the vertical axis in clock domains and the horizontal axis in a minimum available resource. According to the different resource types represented, the values ​​in the two-dimensional matrix are defined as 1, 2, and 3, representing CLB, BRAM, and DSP resources, respectively. The training module characterizes the online tasks, uses the optional modeling results of the online tasks as the target chip, uses the fixed-length data defined by the online tasks and the target chip as the input data of the DQN network, uses the DQN reward function as the feedback of the neural network, and updates the reward function with the DQN updated value function as the feedback of each step; and uses DQN and the optimal longest fit algorithm to jointly train the DQN network; The online task set is: T={id,clb,bram,dsp,w,h,t a ,t s ,t d } Among them, id is the task type tag placed, clb, bram and dsp represent the number of CLB, BRAM and DSP resources required by the task respectively, w and h are the width and height of the rectangular shape corresponding to the task, t a is the task arrival time, t s is the latest execution time of the task, t d is the running time; The target chip is F = {F1, F2, ..., F n }; For any FPGAF i ,i∈{1,2,...,n} has F i ={Cell[w][h],T i,1 ,T i,2 ,...,T i,m }, the two-dimensional array Cell takes values ​​1, 2, 3, corresponding to the model of FPGA on-chip resources; T i,k , k∈{1,2,...,m} means that there are m tasks placed on FPGA i; for any of the tasks, there is T i,k ∈T; The fixed-length data is the fixed-length multi-batch input data state space of DQN based on FPGA-online task pairs; The DQN reward function is: Among them, α, β and γ are constants; covered res (t) and needed res (t) respectively represent the number of resources occupied and the number of resources required by all tasks deployed on the target chip at time t; the adhesion(t) function represents the size of the internal fragmentation of the target chip resources at time t, res(t) is the number of resources on the FPGA chip, and r(t) is the benefit of the current decision; The optimization module finds the appropriate target chip and the optimal placement result in each scheduling event, inputs the online task and the target chip into the trained DQN network, and selects the target chip with the highest reward value as the running chip for the corresponding task to achieve task scheduling optimization.

Images