Batch account checking task scheduling method and device

Abstract

The invention provides a batch reconciliation task scheduling method and device, and the method comprises the steps: obtaining a batch reconciliation task abstracted as a directed acyclic graph, the batch reconciliation task comprises a plurality of subtasks, and the directed acyclic graph comprises a parent-child dependency relationship between the subtasks; calling a decision model constructed based on a Markov decision process, and generating a scheduling decision of each subtask; and scheduling the subtasks to a local scheduling server or a batch server for execution according to the scheduling decision. According to the scheme, the dependency relationship between the tasks is explored through the decision model constructed based on the Markov decision process, so that a more appropriate scheduling decision is made for batch account checking tasks, and the task processing efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method and device for scheduling batch reconciliation tasks. Background Art

[0002] As the scale of financial business continues to expand, a large number of back-end IT systems have been established to support business operations. Some of these IT systems need to perform batch reconciliation of transactions occurring within the system after the end of daily operations in order to generate reports that meet regulatory requirements.

[0003] At present, IT systems use serial execution to process batch reconciliation tasks. However, there are complex dependencies between batch reconciliation tasks. Under the aforementioned existing processing method, the current task can only be executed after all upstream tasks have been completed, and the task processing efficiency is low. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a method and device for scheduling batch reconciliation tasks to solve the problem of low task processing efficiency in serially executing batch reconciliation tasks.

[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0006] A first aspect of an embodiment of the present invention discloses a method for scheduling batch reconciliation tasks, the method comprising:

[0007] Acquire a batch reconciliation task abstracted as a directed acyclic graph, wherein the batch reconciliation task includes multiple subtasks, and the directed acyclic graph includes parent-child dependency relationships between the subtasks;

[0008] Calling a decision model built based on a Markov decision process to generate a scheduling decision for each of the subtasks;

[0009] According to the scheduling decision, the subtask is scheduled to a local scheduling server or a batch server for execution.

[0010] Preferably, the process of scheduling the subtask to the local scheduling server for execution includes:

[0011] Scheduling the input data of the subtask to be executed currently to the local scheduling server;

[0012] The subtask to be executed currently is added to the calculation queue of the local scheduling server to wait for execution.

[0013] Preferably, the process of dispatching the subtasks to a batch server for execution includes:

[0014] Identifying a first set of parent nodes that are scheduled to be executed in a batch server, wherein the first set of parent nodes includes the subtasks of type parent nodes that are related to the subtask to be executed currently in the directed acyclic graph;

[0015] Collecting input data of the subtask to be executed currently and collecting output data of each subtask in the first parent node set through the local scheduling server;

[0016] Using the local scheduling server, adding the input data of the subtask to be executed currently and the output data of each subtask in the first parent node set to the transmission queue of the uplink subchannel of the batch server to upload to the batch server;

[0017] The subtask to be executed currently is added to the computing queue of the batch server to wait for execution.

[0018] Preferably, calling a decision model constructed based on a Markov decision process to generate a scheduling decision for each of the subtasks includes:

[0019] The reward discount factor, experience pool size, learning rate, parameter update times and the number of subtasks are input into a decision model constructed based on a Markov decision process, and the scheduling decision for each subtask is generated by combining the task characteristics of the subtasks and the available resources of the batch server.

[0020] Preferably, each of the batch servers at least comprises a plurality of uplink sub-channels for data transmission and a plurality of processors for providing computing services.

[0021] A second aspect of an embodiment of the present invention discloses a batch reconciliation task scheduling device, the device comprising:

[0022] An acquisition module, used to acquire a batch reconciliation task abstracted as a directed acyclic graph, wherein the batch reconciliation task includes multiple subtasks, and the directed acyclic graph includes parent-child dependency relationships between the subtasks;

[0023] A decision module, used to call a decision model built based on a Markov decision process to generate a scheduling decision for each of the subtasks;

[0024] The scheduling module is used to schedule the subtask to a local scheduling server or a batch server for execution according to the scheduling decision.

[0025] Preferably, the scheduling module for scheduling the subtask to the local scheduling server for execution includes:

[0026] The scheduling submodule is used to schedule the input data of the subtask to be executed to the local scheduling server;

[0027] The first adding submodule is used to add the subtask to be executed currently to the calculation queue of the local scheduling server to wait for execution.

[0028] Preferably, the scheduling module for scheduling the subtasks to the batch server for execution includes:

[0029] An identification submodule, used for identifying a first set of parent nodes that are scheduled to be executed in a batch server, wherein the first set of parent nodes includes the subtasks of a parent node type that are related to the subtask to be currently executed in the directed acyclic graph;

[0030] A collecting submodule, used for collecting input data of the subtask to be executed currently through a local scheduling server, and collecting output data of each of the subtasks in the first parent node set;

[0031] A second adding module is used to use the local scheduling server to add the input data of the subtask to be executed and the output data of each subtask in the first parent node set to the transmission queue of the uplink subchannel of the batch server to upload to the batch server;

[0032] The third adding module is used to add the subtask to be executed currently to the calculation queue of the batch server to wait for execution.

[0033] Preferably, the decision module is specifically used to: input the reward discount factor, experience pool size, learning rate, number of parameter updates and the number of subtasks into a decision model constructed based on the Markov decision process, and combine the task characteristics of the subtasks and the available resources of the batch server to generate a scheduling decision for each of the subtasks.

[0034] Preferably, each of the batch servers at least comprises a plurality of uplink sub-channels for data transmission and a plurality of processors for providing computing services.

[0035] Based on the above-mentioned embodiment of the present invention, a method and device for scheduling batch reconciliation tasks are provided. The method is as follows: obtaining a batch reconciliation task abstracted as a directed acyclic graph, the batch reconciliation task includes multiple subtasks, and the directed acyclic graph includes parent-child dependencies between subtasks; calling a decision model constructed based on the Markov decision process to generate a scheduling decision for each subtask; and scheduling the subtasks to a local scheduling server or a batch server for execution according to the scheduling decision. In this scheme, the batch reconciliation task is abstracted as a directed acyclic graph, and a decision model constructed based on the Markov decision process is used to generate scheduling decisions for each subtask in the batch reconciliation task. Then, according to the scheduling decision, the subtasks are scheduled to a local scheduling server or a batch server for execution. The decision model constructed based on the Markov decision process is used to explore the dependencies between tasks, thereby making more appropriate scheduling decisions for the batch reconciliation task and improving the efficiency of task processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0037] Figure 1 An example diagram of the architecture of a batch reconciliation task scheduling system provided by an embodiment of the present invention;

[0038] Figure 2 A flowchart of a batch reconciliation task scheduling method provided by an embodiment of the present invention;

[0039] Figure 3 An example diagram of the network structure of the Seq2Seq model provided in an embodiment of the present invention;

[0040] Figure 4 A structural block diagram of a batch reconciliation task scheduling device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0043] As the scale of financial business continues to expand, a large number of back-end IT systems have been established to support business operations. Some of these IT systems need to perform batch reconciliation of transactions occurring within the system after the end of daily operations in order to generate reports that meet regulatory requirements.

[0044] At present, IT systems use serial execution to process batch reconciliation tasks. However, there are complex dependencies between batch reconciliation tasks. Under the above existing processing methods, the current task can only be executed after all upstream tasks are completed, which leads to low task processing efficiency and long processing time. In addition, there is heterogeneity in computing resources between batch servers and within batch servers, so it is necessary to design corresponding task scheduling algorithms for specific scenarios.

[0045] To this end, this scheme proposes a batch reconciliation task scheduling method and device, which abstracts the batch reconciliation task into a directed acyclic graph, and uses a decision model built based on the Markov decision process to generate scheduling decisions for each subtask in the batch reconciliation task, and then schedules the subtask to a local scheduling server or a batch server for execution according to the scheduling decision. The decision model built based on the Markov decision process is used to explore the dependencies between tasks, so as to make more appropriate scheduling decisions for batch reconciliation tasks and improve task processing efficiency.

[0046] It should be noted that batch tasks refer to the following: most of the banking IT systems need to summarize, analyze and migrate business data after daily operations, and generate relevant reports that meet regulatory requirements. This processing operation is called daily batch job;

[0047] Reconciliation means that at the end of each day, the bank will obtain the day's reconciliation records from all parties involved in the transaction process and reconcile them on a transaction basis. If inconsistent data is found among the parties involved at the end of the day, the inconsistent reconciliation transaction records will be recorded.

[0048] It can be understood that, in practical applications, this solution implements the scheduling of batch reconciliation tasks based on a batch reconciliation task scheduling system (which may also be referred to as a batch reconciliation task scheduling system based on subtask dependencies).

[0049] like Figure 1 As shown in the example diagram of the architecture of the batch reconciliation task scheduling system, the batch reconciliation task scheduling system includes a batch server layer, a scheduling layer and a batch reconciliation task layer;

[0050] Among them, multiple batch reconciliation tasks (such as Figure 1 In the batch reconciliation task 1 to batch reconciliation task 3), each batch reconciliation task performs processing 1 to processing 3;

[0051] In the scheduling layer, the scheduling server makes scheduling decisions for the subtasks in each pending batch reconciliation task and also provides computing services for the subtasks;

[0052] In the batch server layer, multiple batch servers adjacent to each other are deployed. Each batch server has multiple wireless sub-channels (such as uplink sub-channels) and processors, which can provide communication and computing servers for subtasks. The batch servers can also be connected by wire, and the data transmission time between the batch servers can be ignored.

[0053] See also Figure 2 , shows a flow chart of a batch reconciliation task scheduling method provided by an embodiment of the present invention, the batch reconciliation task scheduling method comprising:

[0054] Step S201: Obtain a batch reconciliation task abstracted as a directed acyclic graph.

[0055] In the specific implementation of step S201, a batch reconciliation task abstracted as a directed acyclic graph (DAG) is obtained from the batch reconciliation task layer. The batch reconciliation task includes multiple subtasks, and the directed acyclic graph includes parent-child dependency relationships between the subtasks.

[0056] Specifically, the batch reconciliation task is abstracted into a directed acyclic graph, in which the vertices represent subtasks, and the directed edges represent the data dependency relationship between two subtasks (i.e., parent-child dependency relationship), where the successor subtask and the predecessor subtask are called parent nodes and child nodes, respectively; a subtask of type child node can only start running after receiving the calculation results of all its parent nodes.

[0057] It should be noted that the task characteristics of a batch reconciliation task are characterized by the topological structure of the DAG, the required computing resources, and the size of the dependencies between subtasks. The task characteristics vary with different batch reconciliation tasks. The subtasks in a batch reconciliation task are the basic scheduling units, and a subtask can be scheduled to a processor on a batch server.

[0058] Step S202: calling a decision model constructed based on the Markov decision process to generate a scheduling decision for each subtask.

[0059] In the process of implementing step S202, in the scheduling layer, the decision model built based on the Markov decision process is called by the scheduling server to generate a corresponding scheduling decision for each subtask. The scheduling decision is used to indicate which processor of which batch server the subtask is to be processed on or on the local scheduling server.

[0060] Specifically, the scheduling server receives relevant information of the subtasks of the batch reconciliation task. Then, at the beginning of scheduling, the scheduling server sorts all the subtasks to be processed according to the designed scheduling algorithm, and makes scheduling decisions for each subtask in sequence. The scheduling decision determines which processor of the local scheduling server or batch server the subtask is processed on, and then the final processing results are aggregated to the scheduling server.

[0061] In some embodiments, each batch server includes at least a plurality of uplink sub-channels for data transmission and a plurality of processors for providing computing services.

[0062] In some embodiments, the specific method of generating a scheduling decision for each subtask is: inputting the reward discount factor, experience pool size, learning rate, number of parameter updates and the number of subtasks into a decision model constructed based on the Markov decision process, combining the task characteristics of the subtask and the available resources of the batch server to generate a scheduling decision for each subtask.

[0063] Step S203: according to the scheduling decision, the subtask is scheduled to the local scheduling server or batch server for execution.

[0064] In the specific implementation of step S203, for each subtask, according to the scheduling decision of the subtask, the subtask is scheduled to a local scheduling server or a batch server for execution.

[0065] In some embodiments, the specific method of scheduling a subtask to a local scheduling server for execution is: scheduling the input data of the current subtask to be executed to the local scheduling server; adding the current subtask to be executed to the computing queue of the local scheduling server to wait for execution. This process does not require the transmission of the data of the subtask itself, nor does it require the transmission of the output data of the parent node in the second parent node set scheduled to the local scheduling server. The second parent node is included in the directed acyclic graph and is a subtask of type a parent node related to the current subtask to be executed.

[0066] In some other embodiments, the specific method of scheduling the subtask to the batch server for execution is: identifying a first parent node set to be scheduled to be executed in the batch server, the first parent node set including subtasks of type parent nodes related to the subtask to be currently executed in a directed acyclic graph; collecting input data of the subtask to be currently executed through a local scheduling server, and collecting output data of each subtask in the first parent node set;

[0067] Using the local scheduling server, the input data of the subtask to be executed currently and the output data of each subtask in the first parent node set are added to the transmission queue of the uplink subchannel of the batch server to be uploaded to the batch server, that is, the input data and the output data are transmitted to the batch server;

[0068] The subtask to be executed is added to the computation queue of the batch server to wait for execution.

[0069] To better explain the details of scheduling subtasks to the local scheduling server or batch server for execution, the following is a detailed description:

[0070] V represents a set of batch reconciliation tasks. Each batch reconciliation task v (v∈V) can be abstractly modeled as a directed acyclic graph (DAG). The batch reconciliation task v consists of a set of subtasks and a set of directed edges. The directed edges represent the parent-child dependency relationship (also called data dependency relationship) between two subtasks.

[0071] The pth subtask on the batch reconciliation task v is represented by "t v,p , v∈V, p∈[1,P]”, P is the total number of subtasks of v; a triple To describe t v,p ,in, is the data of the subtask itself (that is, the program code and related parameters), c v,p is the computing resources required by the subtask, is the output data of the subtask; in addition, the subtask t v,p The input data is To indicate that the input data includes the data of the subtask itself and the output data of its parent node, It can be expressed by formula (1).

[0072]

[0073] In formula (1), P(t v,p ) is t v,p The set of parent nodes of , and it is assumed that the subtasks share the output data of the same parent node.

[0074] The above is a partial description of the subtasks. From another perspective, the batch server is characterized by using wireless bandwidth and computing power. The batch server has R uplink subchannels, where the wireless bandwidth of r subchannels is denoted as “ω r , r∈[1, R]”, to simplify the problem, assume that all subtasks share the downlink channel of the batch server, and the bandwidth of the downlink channel is denoted as ω d ; The batch server has M processors, and the computing power of the mth processor is recorded as "f m , m∈[1,M]”.

[0075] Subtask v,p The scheduling decision is composed of a set of binary integer variables and To indicate that, and Respectively indicate whether to select the rth subchannel and the mth processor for task scheduling, and when or When , the subtask is processed by the local scheduling server.

[0076] This scheme adopts the first-come, first-served (FIFS) queuing model to simulate the computing process of the processor and the transmission process of the sub-channel. v,p It is the basic scheduling unit, so only one subchannel and one processor can be selected to provide scheduling services for it. This process can be expressed by the constraints in formula (2).

[0077]

[0078] The task scheduling model provided by this solution consists of a task transmission model and a task calculation model. The task transmission model and the task calculation model are described below:

[0079] Subtask v,p The input data of needs to be scheduled to the computing server, such as the local scheduling server or batch server; for the convenience of explanation, the subtask t v,p The parent node set P(t v,p ) is divided into two categories, one of which is the first parent node set P scheduled to the batch server. M (t v,p ), and the other type is the second parent node set P scheduled to the local scheduling server L (t v,p ); let TE(t v,p ) indicates that subtask t is fully received v,p At the time of input data, TE(t v,p ) for analysis and explanation.

[0080] 1. When subtask t v,p (The subtask currently to be executed) is scheduled to the local scheduling server (i.e. ), then there is no need to transmit the data of the subtask itself and there is no need to transmit the second parent node set P L (t v,p ) in the parent node, so only the subtask "t v,k ,t v,k ∈P M (t v,p )” needs to be transmitted back to the local scheduling server through the downlink channel, and once the subtask t v,k The calculation is completed, then the subtask t v,k The output data will be immediately placed in the downstream queue and transmitted in FIFS mode.

[0081] Use CE(t v,k ) and WT d (t v,k ) to represent the subtask t v,k The calculation completion time and the waiting time in the downstream queue are used to calculate the subtask t v,k Output data The transmission start time

[0082]

[0083] According to Shannon's formula, output data Data transfer time It can be calculated by formula (4).

[0084]

[0085] In formula (4), ω d is the downlink bandwidth, q m is the transmission power, g v is the channel gain between the local scheduling server and the batch server, is the noise power.

[0086] Based on formula (4), subtask t v,p The time TE (t v,p ) is the first parent node set P M (t v,p ) is the maximum value of the sum of the transmission start time and transmission time of the output data of all subtasks in the task. Specifically, TE(t v,p ).

[0087]

[0088] 2. When subtask t v,p (the subtask currently to be executed) is scheduled to the batch server (i.e. ), then the rth uplink subchannel is used for data transmission, and the data to be transmitted is: the subtask itself t v,p The data and the second parent node set P L (t v,p ) Output data of all subtasks in the local scheduling server. After the data to be transmitted mentioned above are gathered together by the local scheduling server, the data to be transmitted are placed in the transmission queue of the uplink subchannel to wait for data upload (uploaded to the batch server).

[0089] In this case, we only need to focus on the second parent node set P L (t v,p ) with the maximum computation completion time Should The specific content is shown in formula (6).

[0090]

[0091] According to formula (7), we can calculate The transmission start time of the output data for: The calculation end time and the waiting time in the uplink subchannel sum.

[0092]

[0093] Since subtask t v,p The input data of the subtasks are uploaded together, so Transmission time For: subtask t v,p The quotient of the total amount of input data and the transmission rate of the rth subchannel; specifically calculated by formula (8)

[0094]

[0095] In formula (8), q v The transmission power of the local scheduling server.

[0096] By using formula (9), the subtask t is calculated v,p The transmission end time TE(t v,p )for: and sum.

[0097]

[0098] After the data transmission is completed, according to the scheduling decision of the subtask, the subtask t v,p are placed in the computation queue of the local scheduling server or batch server to wait for computation; l (t v,p ) and WT m (t v,p ) to represent the subtask t v,p Waiting time in the computation queues of the local scheduling server and batch server;

[0099] Therefore, the subtask t is calculated by formula (10): v,p The calculation start time CS(t v,p ); Calculate the subtask t by formula (11) v,p The calculation time C(t v,p ), the calculation time C(t v,p ) is the required computing resources C v,p Divide by the computing power of the processor (computing server) f v or m ; Calculate the subtask t by formula (12) v,p The calculation completion time CE(t v,p ).

[0100]

[0101] CE(t v,p )=CS(t v,p )+C(t v,p )(12)

[0102] Based on the above formulas (10) to (12), the batch reconciliation task t v The running time ET(t v ) is defined as: the maximum value of the subtask calculation completion time and the task start scheduling time ST(t v ); Specifically, the batch reconciliation task t is calculated by formula (13) v The running time ET(t v ).

[0103]

[0104] In order to evaluate the performance of the batch reconciliation task scheduling system, this scheme defines the average time AET of the task as the objective function. AET is the "sum of the running time of all batch reconciliation tasks" divided by the "total number of batch reconciliation tasks". The average time AET of the task is calculated by formula (14).

[0105]

[0106] All the above contents are detailed instructions on scheduling subtasks to the local scheduling server or batch server for execution.

[0107] In an embodiment of the present invention, a batch reconciliation task is abstracted into a directed acyclic graph, and a decision model constructed based on a Markov decision process is used to generate scheduling decisions for each subtask in the batch reconciliation task. The subtasks are then scheduled to a local scheduling server or a batch server for execution according to the scheduling decisions. The dependency relationship between tasks is explored by using a decision model constructed based on a Markov decision process, thereby making more appropriate scheduling decisions for the batch reconciliation task and improving task processing efficiency.

[0108] For the above-mentioned embodiments of the present invention Figure 1 As mentioned in step S202, the scheduling decision of the subtask is generated by the decision model constructed based on the Markov decision process. This solution proposes a scheduling method based on deep reinforcement learning, which models the problem of the formula as a Markov decision process (MDP) to obtain a decision model that can generate scheduling decisions for subtasks.

[0109] Specifically, the basic elements of the Markov decision process (MDP) include system state, action space, and reward function, etc. These basic elements are described in detail below.

[0110] System state: As shown in formula (14), the running time of a subtask depends on the task characteristics (such as the required computing resources and data dependencies), the available resources of the batch server (such as communication and computing power), and the scheduling decisions of previous subtasks. Therefore, the state S of the system at time step t can be expressed as follows: Formula (15).

[0111] S={s t |s t =(e,a 1:t-1 )},t∈[1,N](15)

[0112] In formula (15), e is the task embedding, a 1:t-1 is the scheduling decision for the first t-1 subtasks, and N is the number of subtasks.

[0113] In order to transform a directed acyclic graph (DAG) into a sequence, this scheme schedules and sorts subtasks according to their levels in the DAG. The sorting rules are defined as follows: subtasks with lower levels have higher scheduling priorities and smaller scheduling indexes; subtasks with the same level have their scheduling indexes randomly assigned; subtasks with smaller scheduling indexes can be scheduled and processed preferentially in the upload queue and the computing queue.

[0114] The task embedding e can be expressed as e = [e0, e1, …, e j ,…,e N-1 ] indicates that e j To schedule subtask t with index j v,p The embedding e(t v,p ), as shown in formula (16), the embedding e(t v,p ) includes the subtask scheduling index, its own data size, the amount of computing resources required by itself, the subchannel bandwidth and the processor processing rate, etc.

[0115]

[0116] In formula (16), I v,p Schedule indexes for subtasks, is the data size of the subtask itself, c v,p The computing resources required by the subtask, f1 to f M is the computing power of the processor (that is, the processing rate), ω1 to ω R is the subchannel bandwidth.

[0117] It can be understood that the structure of DAG is dynamic and the length of the task embedding e changes with the number of tasks. In order to learn the implicit relationship between different subtasks and their scheduling strategies, this solution adopts a Seq2Seq model to characterize task embeddings e of different lengths. The relevant content of the Seq2Seq model will be explained in subsequent embodiments after explaining the system state, action space and reward function.

[0118] Action space: This scheme uses one-hot encoding to represent the scheduling decision at time step t, which can be represented by a binary vector of "R×M+1" dimensions, where "a tk ,k=(r-1)×M+m” indicates whether the rth subchannel and the mth processor are selected to provide scheduling services, where a t0 =1 means the subtask is scheduled to the local scheduling server.

[0119] From formula (17), we can see that the dimension of the action space is 2 M×R+1 , and it grows exponentially with the increase of the number of sub-channels and the number of processors. Therefore, traditional reinforcement learning algorithms (such as Q-learning) cannot solve the problem studied in this scheme.

[0120] a t =[a t0 ,a t1 ,…,a tk ,…](17)

[0121] Reward function: It is set to the previous subtask tv′,p′ The calculation completion time minus the current subtask t v,p The calculation completion time is as shown in formula (18).

[0122]

[0123] In formula (18), R(t v,p ) is the reward, CE(t v,p ) is the current subtask t v,p The calculation completion time, CE(t v′,p′ ) is subtask t v′,p′ The calculation completion time, I v,p is the current subtask t v,p The scheduling index (t v,p In t v′,p′ Afterwards, I v′,p′ t v′,p′ The scheduling index of .

[0124] It can be seen from the reward function that the higher the reward, the smaller the increase in time consumed to complete the subtask, which helps to consume the objective function defined in formula (14).

[0125] Regarding the Seq2Seq model mentioned above, the Seq2Seq model is an effective paradigm. The Seq2Seq model can cyclically encode the input information sequence into a vector and decode the vector to output the result.

[0126] This scheme incorporates an attention mechanism into the Seq2Seq model, which allows the decoder to show different preferences for different parts of the input sequence to reduce information loss.

[0127] like Figure 3 As shown in the network structure example diagram of the Seq2Seq model, the Seq2Seq model consists of two parts: an encoder and a decoder. The encoder and the decoder are described in detail below.

[0128] Encoder: The encoder is a bidirectional gated recurrent unit (BiGRU). The forward and reverse functions of BiGRU are defined as ( Figure 3 In to )and ( Figure 3 In to ).

[0129] BiGRU has significant advantages in exploring task topology. As can be seen in formula (19), the subtasks are encoded in ascending order and in a cyclic manner according to the scheduling index and the output vector is generated.

[0130]

[0131] Decoder: The decoder is an attention GRU, the function of GRU is passed through h dec To indicate that the subtask with scheduling index j is generated by the decoder, the output of this subtask is denoted by d j , as can be seen from formula (20), d j By the previous scheduling action a j-1 , context vector c j And the previous output d j-1 Joint decision.

[0132] d j =h dec (d j-1 ,c j ,a j-1 )(20)

[0133] It should be noted that c j is the context vector based on the attention mechanism, which is used to predict the action of the subtask with scheduling index j, c j The calculation method of is as shown in formula (21).

[0134]

[0135] In formula (21), is the encoder output vector, a jk To measure For output d j The weight of contribution degree, a jk It can accurately evaluate the intrinsic relationship between different subtasks; in this scheme, the attention mechanism can be implemented based on a trainable forward neural network.

[0136] It should be noted that the basic idea of ​​the proximal policy optimization algorithm (PPO) of the policy gradient is to convert the on-policy training process into off-policy through importance sampling, thereby speeding up the training process. There are two different strategies in PPO: sampling strategy π θ′ and target strategy π θ .

[0137] Before each sampling, π θ′ are initialized to π θ , then PPO is based on the sampling strategy π θ′Generate experience trajectory data, which is used to update the target policy π θ ; In order to make the training more stable, PPO adds two items, PPO-clip and KL divergence, to the objective function as shown in formula (22).

[0138]

[0139] In formula (22), J P (θ) is the target value, E is the expectation, T is the number of samplings, θ is the target strategy parameter, θ′ is the sampling strategy parameter, ε is the clipping coefficient, is the clipping function, is the advantage function value at time step t, Pr t The target strategy π θ and sampling strategy π θ′ is the probability ratio at time step t, KL(θ,θ′) is the KL divergence, and μ is the weight of KL divergence KL(θ,θ′) in the objective function.

[0140] It should be noted that KL divergence is used to evaluate the sampling strategy π θ′ and target strategy π θ The difference between them can be added into the objective function to avoid the gap between them being too large, thereby reducing the number of samplings; the clipping function Limit Pr t The value of is in the interval [1-ε,1+ε], which avoids excessive parameter updates and ensures that π θ′ and π θ The difference between them will not be too big.

[0141] Pr t The target strategy π θ and sampling strategy π θ′ The probability ratio at time step t can be calculated by formula (23): t ; is the advantage function value at time step t, which can be calculated by formula (24) This scheme adopts General Advantage Estimator (GAE) as the advantage function.

[0142]

[0143] In formula (24), γ is the reward discount factor, λ is the bias and variance used to balance the advantage function value, V(s t ) is the state value function (indicates the transition from s t The expected return of the starting experience trajectory), the loss of the state value function is defined as formula (25).

[0144]

[0145] In formula (25), J V (θ) is the loss of the state value function and r is the reward.

[0146] Based on the above formulas for the PPO algorithm, the objective function and update formula of the PPO algorithm are respectively as formula (26) and formula (27).

[0147] J(θ)=J P (θ)-ηJ V (θ)(26)

[0148]

[0149] In formula (26) and formula (27), η is the state value loss function coefficient, α is the learning rate, J(θ) is the objective function of the PPO algorithm, are the updated policy parameters.

[0150] In summary, the final algorithm steps for implementing batch reconciliation task scheduling are the following processes A1-A8:

[0151] A1. Input: reward discount factor γ, experience pool size M, learning rate α, parameter update time m, number of subtasks n (or number of sampling tasks);

[0152] A2. Output: Scheduling decisions for each subtask in the batch reconciliation task;

[0153] A3、for i=1,2,…,n do;

[0154] A4. Initialize sampling strategy network parameters θ i ′ and target policy network parameters θ i , initialize the experience pool is empty;

[0155] A5. According to the sampling strategy Sample trajectory dataset (τ1, τ2, ...) and store it in middle;

[0156] A6. Calculate the target value J(θ i );

[0157] A7. Use formula To update the parameters of the target strategy m times;

[0158] A8. End.

[0159] It can be seen from the contents of the above embodiments that this solution uses a bidirectional recurrent neural network to iteratively encode sequence inputs to explore the data dependencies between batch reconciliation tasks, and uses an attention mechanism to weight different input parts, thereby making more appropriate scheduling decisions for batch reconciliation tasks. It does not require prior knowledge from experts, ensuring the stability and speed of the algorithm.

[0160] Corresponding to the batch reconciliation task scheduling method provided in the above embodiment of the present invention, see Figure 4 , the embodiment of the present invention also provides a structural block diagram of a batch reconciliation task scheduling device, the batch reconciliation task scheduling device includes: an acquisition module 100, a decision module 200 and a scheduling module 300;

[0161] The acquisition module 100 is used to acquire a batch reconciliation task abstracted as a directed acyclic graph, where the batch reconciliation task includes multiple subtasks, and the directed acyclic graph includes parent-child dependency relationships between the subtasks.

[0162] The decision module 200 is used to call a decision model constructed based on the Markov decision process to generate a scheduling decision for each subtask.

[0163] In the specific implementation, the decision module 200 is specifically used to: input the reward discount factor, experience pool size, learning rate, number of parameter updates and the number of subtasks into the decision model constructed based on the Markov decision process, and combine the task characteristics of the subtask and the available resources of the batch server to generate a scheduling decision for each subtask.

[0164] The scheduling module 300 is used to schedule the subtasks to the local scheduling server or batch server for execution according to the scheduling decision.

[0165] In some embodiments, each batch server includes at least a plurality of uplink sub-channels for data transmission and a plurality of processors for providing computing services.

[0166] In an embodiment of the present invention, a batch reconciliation task is abstracted into a directed acyclic graph, and a decision model constructed based on a Markov decision process is used to generate scheduling decisions for each subtask in the batch reconciliation task. The subtasks are then scheduled to a local scheduling server or a batch server for execution according to the scheduling decisions. The dependency relationship between tasks is explored by using a decision model constructed based on a Markov decision process, thereby making more appropriate scheduling decisions for the batch reconciliation task and improving task processing efficiency.

[0167] Preferably, combined Figure 4 As shown in the figure, the scheduling module 300 for scheduling the subtask to the local scheduling server for execution includes a scheduling submodule and a first adding submodule. The execution principle of each submodule is as follows:

[0168] The scheduling submodule is used to schedule the input data of the subtask to be executed to the local scheduling server.

[0169] The first adding submodule is used to add the subtask to be executed currently to the calculation queue of the local scheduling server to wait for execution.

[0170] Preferably, combined Figure 4 As shown in the content, the scheduling module 300 for scheduling subtasks to batch servers for execution includes an identification submodule, a collection submodule, a second adding submodule and a third adding submodule. The execution principle of each submodule is as follows:

[0171] The identification submodule is used to identify a first parent node set that is scheduled to be executed in the batch server, where the first parent node set includes subtasks of parent node type related to the subtask to be currently executed in the directed acyclic graph.

[0172] The collecting submodule is used to collect the input data of the subtask to be executed currently through the local scheduling server, and collect the output data of each subtask in the first parent node set.

[0173] The second adding module is used to use the local scheduling server to add the input data of the subtask to be executed and the output data of each subtask in the first parent node set to the transmission queue of the uplink subchannel of the batch server for uploading to the batch server.

[0174] The third adding module is used to add the subtask to be executed currently to the calculation queue of the batch server to wait for execution.

[0175] In summary, the embodiments of the present invention provide a method and device for scheduling batch reconciliation tasks, which abstract the batch reconciliation tasks into a directed acyclic graph, use a decision model constructed based on the Markov decision process to generate scheduling decisions for each subtask in the batch reconciliation task, and then schedule the subtasks to a local scheduling server or a batch server for execution according to the scheduling decisions. The decision model constructed based on the Markov decision process is used to explore the dependencies between tasks, thereby making more appropriate scheduling decisions for batch reconciliation tasks and improving task processing efficiency.

[0176] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0177] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0178] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A batch reconciliation task scheduling method, characterized in that: The method comprises: Acquire a batch reconciliation task abstracted as a directed acyclic graph, wherein the batch reconciliation task includes multiple subtasks, and the directed acyclic graph includes parent-child dependency relationships between the subtasks; Calling a decision model built based on a Markov decision process to generate a scheduling decision for each of the subtasks; According to the scheduling decision, the subtask is scheduled to a local scheduling server or a batch server for execution.

2. The method according to claim 1, characterized in that: The process of scheduling the subtask to the local scheduling server for execution includes: Scheduling the input data of the subtask to be executed currently to the local scheduling server; The subtask to be executed currently is added to the calculation queue of the local scheduling server to wait for execution.

3. The method according to claim 1, characterized in that The process of dispatching the subtask to a batch server for execution includes: Identifying a first set of parent nodes that are scheduled to be executed in a batch server, wherein the first set of parent nodes includes the subtasks of type parent nodes that are related to the subtask to be executed currently in the directed acyclic graph; Collecting input data of the subtask to be executed currently and collecting output data of each subtask in the first parent node set through the local scheduling server; Using the local scheduling server, adding the input data of the subtask to be executed currently and the output data of each subtask in the first parent node set to the transmission queue of the uplink subchannel of the batch server to upload to the batch server; The subtask to be executed currently is added to the computing queue of the batch server to wait for execution.

4. The method according to any one of claims 1 to 3, characterized in that: The decision model constructed based on the Markov decision process is called to generate a scheduling decision for each of the subtasks, including: The reward discount factor, experience pool size, learning rate, parameter update times and the number of subtasks are input into a decision model constructed based on a Markov decision process, and the scheduling decision for each subtask is generated by combining the task characteristics of the subtasks and the available resources of the batch server.

5. The method according to any one of claims 1 to 3, characterized in that: Each of the batch servers at least includes a plurality of uplink sub-channels for data transmission and a plurality of processors for providing computing services.

6. A batch reconciliation task scheduling device, characterized in that: The device comprises: An acquisition module, used to acquire a batch reconciliation task abstracted as a directed acyclic graph, wherein the batch reconciliation task includes multiple subtasks, and the directed acyclic graph includes parent-child dependency relationships between the subtasks; A decision module, used to call a decision model built based on a Markov decision process to generate a scheduling decision for each of the subtasks; The scheduling module is used to schedule the subtask to a local scheduling server or a batch server for execution according to the scheduling decision.

7. The device according to claim 6, characterized in that The scheduling module for scheduling the subtask to the local scheduling server for execution includes: The scheduling submodule is used to schedule the input data of the subtask to be executed to the local scheduling server; The first adding submodule is used to add the subtask to be executed currently to the calculation queue of the local scheduling server to wait for execution.

8. The device according to claim 6, characterized in that The scheduling module for scheduling the subtasks to the batch server for execution includes: An identification submodule, used for identifying a first set of parent nodes that are scheduled to be executed in a batch server, wherein the first set of parent nodes includes the subtasks of a parent node type that are related to the subtask to be currently executed in the directed acyclic graph; A collecting submodule, used for collecting input data of the subtask to be executed currently through a local scheduling server, and collecting output data of each of the subtasks in the first parent node set; A second adding module is used to use the local scheduling server to add the input data of the subtask to be executed and the output data of each subtask in the first parent node set to the transmission queue of the uplink subchannel of the batch server to upload to the batch server; The third adding module is used to add the subtask to be executed currently to the calculation queue of the batch server to wait for execution.

9. The device according to any one of claims 6 to 8, characterized in that: The decision module is specifically used to: input the reward discount factor, experience pool size, learning rate, number of parameter updates and the number of subtasks into a decision model constructed based on the Markov decision process, and combine the task characteristics of the subtasks and the available resources of the batch server to generate a scheduling decision for each subtask.

10. The device according to any one of claims 6 to 8, characterized in that: Each of the batch servers at least includes a plurality of uplink sub-channels for data transmission and a plurality of processors for providing computing services.

Images