Blockchain task allocation method and device based on DQN, equipment and medium

Abstract

The application discloses a blockchain task allocation method and device based on DQN, equipment and medium, relates to the field of task allocation, comprising: determining the state vector of the DQN model, selecting the target action according to the state vector, and determining the predicted execution time and the actual execution time of the target action; the predicted execution time and the actual execution time are used as the input of the preset time length reward formula to obtain the time length reward; a plurality of experiences are randomly extracted, the target Q value of the DQN model is determined based on the plurality of experiences, and the DQN model is updated through the observation Q value and the target Q value of the DQN model; multiple rounds of updating are performed until the DQN model meets the preset requirements to obtain the target DQN model; the target blockchain task allocation model is constructed through the target DQN model, and after receiving the blockchain task allocation request, the target blockchain task allocation model is used for intelligent task allocation. Therefore, the task allocation efficiency can be improved, the dynamic environment change can be responded to, and the manual intervention can be reduced.

Description

Technical Field

[0001] This invention relates to the field of task allocation, and in particular to a blockchain task allocation method, apparatus, device, and medium based on DQN. Background Technology

[0002] Blockchain services are a service model based on blockchain technology that provides various functions through a blockchain network, such as smart contract execution, data storage, and security verification. These services can include decentralized application development, digital asset management, supply chain tracking, and identity verification. Blockchain services are typically accessed by developers, businesses, and individuals through open APIs or dedicated platforms, and possess characteristics such as distributed nature, immutability, and high security to meet diverse business needs and application scenarios.

[0003] With the development of cloud computing, migrating blockchain services to the cloud can significantly improve their flexibility and scalability while reducing deployment and management costs. Due to the highly volatile availability of resources in cloud environments, application runtime depends on the current workload, necessitating dynamic task allocation. In this context, performance prediction techniques can be used to estimate the runtime of each task and dynamically allocate tasks to available resources based on the current environmental load. This dynamic task allocation method can be achieved by continuously collecting resource consumption data (such as CPU and memory usage) and using analytical methods for prediction. Combining blockchain with cloud services not only enables efficient task allocation and resource utilization but also provides a more secure and reliable infrastructure, promoting the widespread application and development of blockchain technology.

[0004] However, existing dynamic task allocation schemes have many problems, such as uncertainty in task execution time, high rental prices, and lack of transaction security. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a blockchain task allocation method, apparatus, device, and medium based on DQN, which can improve task allocation efficiency, cope with dynamic environmental changes, and reduce manual intervention. The specific solution is as follows:

[0006] Firstly, this application discloses a blockchain task allocation method based on DQN, including:

[0007] The state vector of the DQN model is determined based on the preset state space. A target action is selected from the preset action space according to the state vector, and the predicted execution time and actual execution time of the target action are determined.

[0008] The predicted execution time and the actual execution time are used as inputs to a preset duration reward formula to obtain a duration reward. The state vector, the target action, and the duration reward are added as current experience to a preset experience replay buffer.

[0009] Several experiences are randomly extracted from the preset experience replay buffer, and the target Q value of the DQN model is determined based on the several experiences. The DQN model is then updated using the observed Q value of the DQN model and the target Q value.

[0010] The process jumps to the step of determining the state vector of the DQN model based on the preset state space, selecting a target action from the preset action space according to the state vector, and determining the predicted execution time and actual execution time of the target action, so as to update the DQN model in the next round until the DQN model meets the preset requirements to obtain the target DQN model.

[0011] A target blockchain task allocation model is constructed using the target DQN model, and upon receiving a blockchain task allocation request, the target blockchain task allocation model is used to intelligently allocate blockchain tasks.

[0012] Optionally, the step of determining the state vector of the DQN model based on a preset state space, and selecting the target action from a preset action space according to the state vector, includes:

[0013] Obtain the current task parameters and hardware parameters from the preset state space, and determine the state vector of the DQN model based on the task parameters and hardware parameters;

[0014] Based on the state vector, an action is randomly selected from the action space with a preset probability to obtain the target action.

[0015] Optionally, determining the predicted execution time and actual execution time of the target action includes:

[0016] Determine the predicted execution time corresponding to the target action, select a target node based on the target action, execute the target action through the target node, and determine the actual execution time of the target action.

[0017] Optionally, the preset duration reward formula is as follows:

[0018] ;

[0019] Among them, RD i，j，m As a time-based reward, TK max For the maximum reward, et i，j，m To predict execution time, at i，j，mThe actual execution time is represented by , i is the label of the client that initiated the request, j is the request label, and m is the DQN model label.

[0020] Optionally, the step of randomly extracting several experiences from the preset experience replay buffer, determining the target Q value of the DQN model based on the several experiences, and updating the DQN model using the observed Q value of the DQN model and the target Q value includes:

[0021] Several experiences are randomly extracted from the preset experience playback buffer, and these experiences are used as inputs to the Bellman equation to calculate the target Q value corresponding to the experiences.

[0022] The Q value corresponding to the target action is used as the observed Q value of the DQN model;

[0023] The DQN model is updated using the observed Q-value and the target Q-value.

[0024] Optionally, updating the DQN model using the observed Q-value and the target Q-value includes:

[0025] The Q-value error of the DQN model is determined by the observed Q-value and the target Q-value, and the current loss function of the DQN model is determined based on the Q-value error.

[0026] The model parameters of the DQN model are updated using the gradient descent algorithm based on the negative gradient direction of the current loss function, so that the DQN model can be updated in the next round based on the updated model parameters.

[0027] Optionally, the DQN-based blockchain task allocation method further includes:

[0028] The model error of the target DQN model is verified based on the Q-value error. If the model error is less than a preset error threshold, the target blockchain task allocation model is constructed using the target DQN model.

[0029] Secondly, this application discloses a blockchain task allocation device based on DQN, comprising:

[0030] The parameter selection module is used to determine the state vector of the DQN model based on the preset state space, select the target action from the preset action space according to the state vector, and determine the predicted execution time and actual execution time of the target action.

[0031] The experience addition module is used to take the predicted execution time and the actual execution time as inputs to a preset duration reward formula to obtain a duration reward, and add the state vector, the target action and the duration reward as the current experience to a preset experience replay buffer.

[0032] The model update module is used to randomly extract a number of experiences from the preset experience replay buffer, determine the target Q value of the DQN model based on the number of experiences, and update the DQN model through the observed Q value of the DQN model and the target Q value.

[0033] The model training module is used to jump to the steps of determining the state vector of the DQN model based on the preset state space, selecting a target action from the preset action space according to the state vector, and determining the predicted execution time and actual execution time of the target action, so as to update the DQN model in the next round until the DQN model reaches the preset requirements to obtain the target DQN model.

[0034] The task allocation module is used to construct a target blockchain task allocation model through the target DQN model, and to intelligently allocate blockchain tasks through the target blockchain task allocation model after receiving a blockchain task allocation request.

[0035] Thirdly, this application discloses an electronic device, including:

[0036] Memory, used to store computer programs;

[0037] A processor for executing the computer program to implement the DQN-based blockchain task allocation method as described above.

[0038] Fourthly, this application discloses a computer-readable storage medium for storing a computer program, which, when executed by a processor, implements the aforementioned DQN-based blockchain task allocation method.

[0039] Therefore, the proposed method can determine the state vector of the DQN model based on a pre-constructed state space. Then, based on the state vector, a target action is selected from a pre-constructed action space, and the predicted and actual execution times of the target action are determined. A duration reward is then obtained based on the predicted and actual execution times. Several experiences are randomly extracted from a pre-constructed experience replay buffer to determine the target Q-value of the DQN model. The DQN model is updated using the observed Q-value and the target Q-value. Finally, a target blockchain task allocation model is constructed using the trained target DQN model. Upon receiving a blockchain task allocation request, the target blockchain task allocation model intelligently allocates blockchain tasks. This approach improves task allocation efficiency, addresses dynamic environmental changes, enhances task prediction accuracy, reduces human intervention, and promotes the development of blockchain technology. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0041] Figure 1 This application discloses a flowchart of a blockchain task allocation method based on DQN.

[0042] Figure 2 This application discloses a specific flowchart of a blockchain task allocation method based on DQN.

[0043] Figure 3 This is a sequence diagram of a blockchain task allocation method based on DQN disclosed in this application;

[0044] Figure 4 This is a schematic diagram of a blockchain task allocation device based on DQN disclosed in this application;

[0045] Figure 5 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] Existing methods for dynamically allocating tasks rely on continuously collecting resource consumption data (such as CPU and memory usage) and using analytical methods for prediction. Combining blockchain with cloud services can not only achieve efficient task allocation and resource utilization but also provide a more secure and reliable infrastructure, promoting the widespread application and development of blockchain technology. However, existing dynamic task allocation schemes suffer from several problems, such as the uncertainty of task execution time, high rental prices, and lack of guaranteed transaction security.

[0048] To overcome the aforementioned technical problems, this application discloses a blockchain task allocation method, apparatus, device, and medium based on DQN, which can improve task allocation efficiency, cope with dynamic environmental changes, and reduce human intervention.

[0049] See Figure 1 As shown in the figure, this invention discloses a blockchain task allocation method based on DQN, including:

[0050] Step S11: Determine the state vector of the DQN model based on the preset state space, select the target action from the preset action space according to the state vector, and determine the predicted execution time and actual execution time of the target action.

[0051] In this embodiment, the current task parameters and hardware parameters need to be obtained from the preset state space. Then, the state vector of the DQN model is determined based on the obtained task parameters and hardware parameters. It should be noted that in the DQN framework, the current state of each agent can be represented as a snapshot at a certain moment. The state consists of multiple parts. The first part is the task parameters, such as task type, task input parameters, and task submission time. The second part reflects the static performance of the current node, including information such as CPU (Central Processing Unit) speed, number of cores, maximum memory, hard disk type, and available network bandwidth. The third part reflects the dynamic resource data of the node, including CPU time, memory size, network I / O, disk I / O, and number of processes. This dynamic resource data can be obtained in real time through the Docker interface, thereby providing the model with key information so that the intelligent agent can make effective decisions and allocate tasks. The preset state space includes the above three parts. The action space refers to the set of all possible actions that the agent can perform.

[0052] Therefore, the state vector of the DQN (Deep Q-network) model can be determined by obtaining the current task parameters, node CPU, memory, and other metrics from the preset state space. Then, based on the current state and an ε-greedy policy, a random action is selected with a certain probability ε, and the action with the largest Q value is selected with a probability of 1-ε. The selected action with the largest Q value is then taken as the target action. Finally, the predicted execution time required for each node to execute the target action is determined based on the current hardware information.

[0053] Then, based on the action, a node needs to be selected, and the node with the lowest predicted execution time needs to be selected to execute the task. The actual execution time is then calculated to obtain the actual execution time.

[0054] Step S12: Use the predicted execution time and the actual execution time as inputs to the preset duration reward formula to obtain the duration reward, and add the state vector, the target action, and the duration reward as the current experience to the preset experience replay buffer.

[0055] In this embodiment, the predicted execution time and the actual execution time can be used as inputs to the preset duration reward formula to obtain the duration reward. The inputs to the preset duration reward formula are as follows:

[0056] ;

[0057] Among them, RD i，j，m As a time-based reward, TK max For the maximum reward, et i，j，m To predict execution time, at i，j，m The actual execution time is represented by , i is the label of the client that initiated the request, j is the request label, and m is the DQN model label.

[0058] To enable experience reuse and enhance the stability and efficiency of the learning algorithm, an experience replay buffer can be set up to store and utilize past experiences. These experiences can be randomly sampled and reused during training. The current state (state vector), the defined target action, and the obtained temporal reward can be added to the pre-defined experience replay buffer as current experience. In this way, the experience in the replay buffer can be used to update the model, making the DQN model more accurate.

[0059] Step S13: Randomly extract several experiences from the preset experience replay buffer, determine the target Q value of the DQN model based on the several experiences, and update the DQN model through the observed Q value of the DQN model and the target Q value.

[0060] In this embodiment, several experiences need to be randomly extracted from a preset experience replay buffer to determine the Q-value of the DQN model based on the extracted experiences, and then the DQN model is updated based on the obtained Q-value. The specific process is as follows: Several experiences are randomly extracted from the preset experience replay buffer, and the experiences are used as inputs to the Bellman equation to calculate the target Q-value corresponding to the experiences; the Q-value corresponding to the target action is used as the observed Q-value of the DQN model; the DQN model is updated based on the observed Q-value and the target Q-value.

[0061] In other words, generally, the larger the number of retrieved experiences, the more accurate the gradient estimation of the Q-network will be, but the computational cost will also increase accordingly. Therefore, we can set up a method to randomly sample 50 experiences from the experience replay buffer to update the Q-network. Then, the selected 50 experiences are used as input to the Bellman equation to calculate the target Q-value. It should be noted that by calculating the target Q-value (Quality Value), the DQN model can understand the long-term benefit of taking a certain action in the current state, thereby guiding the learning process and updating its Q-value function.

[0062] Further explanation is needed. In the DQN model, the Q-value represents the expected value of taking an action in a specific state. Therefore, a Q-value function is needed to evaluate the time required to execute a task under the resource state of each node. This application uses a DNN (Deep Neural Network) to fit the Q-value function. The DNN has three layers: the first is the input layer with 21 neurons. In this layer, some elements of the state are combined to reduce dimensionality, and OneHotEncoding is used to represent some categorical variables, such as task parameters. The middle layer contains 70 neurons, which is the average between the input and output dimensions. The output layer represents the discretized action space and contains 120 neurons, where 1 second is the minimum expected running time, and 120 seconds is the last accepted value before a timeout error occurs.

[0063] In this embodiment, the DQN model needs to be updated once using the observed Q-value and the target Q-value. The specific process is as follows: the Q-value error of the DQN model is determined using the observed Q-value and the target Q-value, and the current loss function of the DQN model is determined based on the Q-value error; the model parameters of the DQN model are updated based on the negative gradient direction of the current loss function using the gradient descent algorithm, so as to perform the next round of updates to the DQN model based on the updated model parameters.

[0064] That is, it is necessary to calculate the loss function based on the Q-value error between the observed Q-value and the calculated target Q-value of the DQN model, and then use the gradient descent algorithm to update the parameters of the neural network in the DQN model along the negative gradient direction of the loss function in order to reduce the value of the loss function.

[0065] Step S14: Jump to the step of determining the state vector of the DQN model based on the preset state space, selecting the target action from the preset action space according to the state vector, and determining the predicted execution time and actual execution time of the target action, so as to perform the next round of updates to the DQN model until the DQN model meets the preset requirements to obtain the target DQN model.

[0066] In this embodiment, the model needs to be updated multiple times, continuously iterating the Q-value error to gradually converge the parameters of the neural network in the DQN model to their optimal values. This minimizes the error between the current Q-value and the target Q-value, allowing the DQN model to learn a more accurate action-value function, thus better guiding its decision-making in the environment and improving its behavioral effectiveness and performance. As training progresses, the value of ε can be gradually reduced, making the model more inclined to make decisions based on existing experience. This helps accelerate the convergence of the training process, and since the DQN model utilizes more of the learned knowledge, it no longer needs to spend excessive time randomly exploring unknown actions.

[0067] Step S15: Construct a target blockchain task allocation model using the target DQN model, and intelligently allocate blockchain tasks using the target blockchain task allocation model after receiving a blockchain task allocation request.

[0068] In this embodiment, a target blockchain task allocation model needs to be constructed using the target DQN model. It should be noted that the performance of the blockchain task allocation model needs to be verified during its construction. Specifically, the model error of the target DQN model needs to be verified based on the Q-value error. If the obtained model error is less than a preset error threshold, then the target blockchain task allocation model is constructed using the target DQN model. Each time the blockchain request client initiates a request, it passes the request parameters and the resource parameters of each node into the target blockchain task allocation model. The target blockchain task allocation model intelligently selects a node to run the request task based on the input. After the task is completed, the client receives the task return.

[0069] Therefore, this method can determine the state vector of the DQN model based on a pre-constructed state space. Then, based on the state vector, a target action is selected from a pre-constructed action space, and the predicted and actual execution times of the target action are determined. A duration reward is then obtained based on the predicted and actual execution times. Several experiences are randomly extracted from a pre-constructed experience replay buffer to determine the target Q-value of the DQN model. The DQN model is updated using the observed and target Q-values. Finally, a target blockchain task allocation model is constructed using the trained target DQN model. Upon receiving a blockchain task allocation request, the target blockchain task allocation model is used for intelligent allocation of blockchain tasks. This approach improves task allocation efficiency, adapts to dynamic environmental changes, enhances task prediction accuracy, reduces human intervention, and promotes the development of blockchain technology. Furthermore, by combining deep learning and reinforcement learning techniques, this method can intelligently optimize resource utilization, adapt to constantly changing environments, and improve the accuracy and automation of task allocation.

[0070] See Figure 2 As shown in the figure, this invention discloses a blockchain task allocation method based on DQN, including:

[0071] Step S21: Obtain the current task parameters and hardware parameters from the preset state space, and determine the state vector of the DQN model based on the task parameters and hardware parameters.

[0072] like Figure 3 As shown, the model needs to be set up based on a pre-defined state space, action space, and reward function. First, the current task parameters and hardware parameters need to be obtained from the pre-defined state space. It is assumed that each agent can run a fixed number of concurrent tasks as a separate container. Therefore, the execution time of a task may be affected by three combined parameters: first, the type of task submitted, the task's input parameters, and the task submission time; second, the total resources available to each agent, including CPU speed, number of cores, maximum memory, hard disk type, and available network bandwidth; and third, the resource consumption of the agent in the first few tasks, such as CPU time, memory size, network I / O, disk I / O, and the number of processes. In summary, the model's state space S can be represented as S = {<task type, task input parameters, task submission time>, <CPU speed, number of cores, maximum memory, hard disk type, available network bandwidth>, <CPU time, memory size, network I / O, disk I / O, number of processes>}. Therefore, the current task parameters and hardware parameters can be selected from the state space. Then, the state vector can be initialized based on the selected task parameters and hardware parameters.

[0073] Step S22: Based on the state vector, randomly select an action from the action space with a preset probability to obtain the target action.

[0074] In this embodiment, the action space refers to the set of all possible actions that the model can execute in DQN. The model predicts the expected runtime (in seconds) required for an upcoming request task based on the current state. To simplify the training process, the execution time can be divided into discrete sets with a step size of 1 second, and a predefined upper limit of 120 seconds can be set. This means that the system's processing time for each instance cannot exceed 120 seconds; exceeding this will trigger a global timeout. This setting allows the model to learn more easily and make reasonable decisions while ensuring the stability and performance of the system. Therefore, based on the current state, an ε-greedy policy can be used to select the action with the largest Q-value as the target action with a probability of 1-ε.

[0075] In reinforcement learning, exploration means that the agent randomly explores the entire action space to improve its understanding of each action, thereby gaining long-term benefits. Typically, this value starts at 1, meaning that initially the agent knows nothing about the actions and must therefore explore all available actions for each state. Subsequently, it slowly decays in future cycles until it approaches 0 at the end of training, meaning it has fully utilized its knowledge. For the evaluation performed, in this embodiment, it can start at ε=1 and implement a linear decay strategy until ε reaches a minimum of 0.001. This makes the final DQN model more accurate, resulting in better performance of the target blockchain task allocation model built based on the DQN model.

[0076] Step S23: Determine the predicted execution time corresponding to the target action, and select a target node based on the target action to execute the target action through the target node, thereby determining the actual execution time of the target action.

[0077] Step S24: Use the predicted execution time and the actual execution time as inputs to the preset duration reward formula to obtain the duration reward, and add the state vector, the target action, and the duration reward as the current experience to the preset experience replay buffer.

[0078] Step S25: Randomly extract several experiences from the preset experience replay buffer, determine the target Q value of the DQN model based on the several experiences, and update the DQN model through the observed Q value of the DQN model and the target Q value.

[0079] Step S26: Jump to the step of obtaining the current task parameters and hardware parameters from the preset state space, and determining the state vector of the DQN model based on the task parameters and hardware parameters, so as to perform the next round of updates on the DQN model until the DQN model reaches the preset requirements to obtain the target DQN model.

[0080] Step S27: Construct a target blockchain task allocation model using the target DQN model, and intelligently allocate blockchain tasks using the target blockchain task allocation model after receiving a blockchain task allocation request.

[0081] Therefore, in this embodiment, the current task parameters and hardware parameters can be obtained from a preset state space. Based on these parameters, the state vector of the DQN model is determined. Then, based on the state vector, an action is randomly selected from the action space with a preset probability to obtain the target action. Since the model requires multiple rounds of training, a linear decay strategy can be implemented to gradually decrease the probability ε, thereby making the final DQN model more accurate and resulting in better performance of the target blockchain task allocation model built based on the DQN model.

[0082] See Figure 4 As shown, this embodiment of the invention discloses a blockchain task allocation device based on DQN, comprising:

[0083] The parameter selection module 11 is used to determine the state vector of the DQN model based on the preset state space, select the target action from the preset action space according to the state vector, and determine the predicted execution time and actual execution time of the target action.

[0084] The experience addition module 12 is used to take the predicted execution time and the actual execution time as inputs to the preset duration reward formula to obtain the duration reward, and add the state vector, the target action and the duration reward as the current experience to the preset experience playback buffer.

[0085] The model update module 13 is used to randomly extract a number of experiences from the preset experience replay buffer, determine the target Q value of the DQN model based on the number of experiences, and update the DQN model through the observed Q value of the DQN model and the target Q value.

[0086] The model training module 14 is used to jump to the steps of determining the state vector of the DQN model based on the preset state space, selecting a target action from the preset action space according to the state vector, and determining the predicted execution time and actual execution time of the target action, so as to update the DQN model in the next round until the DQN model reaches the preset requirements to obtain the target DQN model.

[0087] The task allocation module 15 is used to construct a target blockchain task allocation model through the target DQN model, and to intelligently allocate blockchain tasks through the target blockchain task allocation model after receiving a blockchain task allocation request.

[0088] Therefore, the proposed method can determine the state vector of the DQN model based on a pre-constructed state space. Then, based on the state vector, a target action is selected from a pre-constructed action space, and the predicted and actual execution times of the target action are determined. A duration reward is then obtained based on the predicted and actual execution times. Several experiences are randomly extracted from a pre-constructed experience replay buffer to determine the target Q-value of the DQN model. The DQN model is updated using the observed Q-value and the target Q-value. Finally, a target blockchain task allocation model is constructed using the trained target DQN model. Upon receiving a blockchain task allocation request, the target blockchain task allocation model intelligently allocates blockchain tasks. This approach improves task allocation efficiency, addresses dynamic environmental changes, enhances task prediction accuracy, reduces human intervention, and promotes the development of blockchain technology.

[0089] In some embodiments, the parameter selection module 11 may specifically include:

[0090] Parameter determination unit. Used to obtain current task parameters and hardware parameters from a preset state space, and to determine the state vector of the DQN model based on the task parameters and the hardware parameters;

[0091] An action selection unit is used to randomly select an action from the action space based on the state vector with a preset probability to obtain a target action.

[0092] In some embodiments, the parameter selection module 11 may specifically include:

[0093] A time determination unit is used to determine the predicted execution time corresponding to the target action, select a target node based on the target action, execute the target action through the target node, and determine the actual execution time of the target action.

[0094] In some embodiments, the model update module 13 may specifically include:

[0095] The first Q-value determination unit is used to randomly extract several experiences from the preset experience playback buffer and use the several experiences as input to the Bellman equation to calculate the target Q value corresponding to the several experiences through the Bellman equation.

[0096] The second Q-value determination unit is used to take the Q-value corresponding to the target action as the observed Q-value of the DQN model;

[0097] The model update submodule is used to update the DQN model using the observed Q-values ​​and the target Q-values.

[0098] In some embodiments, the model update submodule may specifically include:

[0099] The loss function determination unit is used to determine the Q-value error of the DQN model through the observed Q-value and the target Q-value, and to determine the current loss function of the DQN model based on the Q-value error;

[0100] The model update unit is used to update the model parameters of the DQN model based on the negative gradient direction of the current loss function using the gradient descent algorithm, so as to perform the next round of updates on the DQN model based on the obtained updated model parameters.

[0101] In some embodiments, the DQN-based blockchain task allocation device may further include:

[0102] The model building unit is used to verify the model error of the target DQN model based on the Q-value error. If the model error is less than a preset error threshold, the target blockchain task allocation model is built through the target DQN model.

[0103] Furthermore, embodiments of this application also disclose an electronic device, Figure 5 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0104] Figure 5 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the DQN-based blockchain task allocation method disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0105] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0106] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored thereon can include an operating system 221, computer programs 222, etc., and the storage method can be temporary storage or permanent storage.

[0107] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the DQN-based blockchain task allocation method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0108] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned DQN-based blockchain task allocation method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0109] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0110] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0111] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0112] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0113] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A blockchain task allocation method based on DQN, characterized in that, include: The state vector of the DQN model is determined based on the preset state space. A target action is selected from the preset action space according to the state vector, and the predicted execution time and actual execution time of the target action are determined. The predicted execution time and the actual execution time are used as inputs to a preset duration reward formula to obtain a duration reward. The state vector, the target action, and the duration reward are added as current experience to a preset experience replay buffer. Several experiences are randomly extracted from the preset experience replay buffer, and the target Q value of the DQN model is determined based on the several experiences. The DQN model is then updated using the observed Q value of the DQN model and the target Q value. The process jumps to the step of determining the state vector of the DQN model based on the preset state space, selecting a target action from the preset action space according to the state vector, and determining the predicted execution time and actual execution time of the target action, so as to update the DQN model in the next round until the DQN model meets the preset requirements to obtain the target DQN model. A target blockchain task allocation model is constructed using the target DQN model, and upon receiving a blockchain task allocation request, the target blockchain task allocation model is used to intelligently allocate blockchain tasks. The step of randomly extracting several experiences from the preset experience replay buffer, determining the target Q value of the DQN model based on the several experiences, and updating the DQN model using the observed Q value of the DQN model and the target Q value includes: Several experiences are randomly extracted from the preset experience playback buffer, and these experiences are used as inputs to the Bellman equation to calculate the target Q value corresponding to the experiences. The Q value corresponding to the target action is used as the observed Q value of the DQN model; The DQN model is updated using the observed Q-value and the target Q-value; The step of updating the DQN model using the observed Q-value and the target Q-value includes: The Q-value error of the DQN model is determined by the observed Q-value and the target Q-value, and the current loss function of the DQN model is determined based on the Q-value error. The model parameters of the DQN model are updated using the gradient descent algorithm based on the negative gradient direction of the current loss function, so that the DQN model can be updated in the next round based on the updated model parameters. The DQN-based blockchain task allocation method further includes: The model error of the target DQN model is verified based on the Q-value error. If the model error is less than a preset error threshold, the target blockchain task allocation model is constructed using the target DQN model.

2. The blockchain task allocation method based on DQN according to claim 1, characterized in that, The step of determining the state vector of the DQN model based on a preset state space, and selecting a target action from a preset action space according to the state vector, includes: Obtain the current task parameters and hardware parameters from the preset state space, and determine the state vector of the DQN model based on the task parameters and hardware parameters; Based on the state vector, an action is randomly selected from the action space with a preset probability to obtain the target action.

3. The blockchain task allocation method based on DQN according to claim 1, characterized in that, Determining the predicted execution time and actual execution time of the target action includes: Determine the predicted execution time corresponding to the target action, select a target node based on the target action, execute the target action through the target node, and determine the actual execution time of the target action.

4. The blockchain task allocation method based on DQN according to claim 1, characterized in that, The preset duration reward formula is as follows: ； Among them, RD i，j，m As a time-based reward, TK max For the maximum reward, et i，j，m To predict execution time, at i，j，m The actual execution time is represented by , i is the label of the client that initiated the request, j is the request label, and m is the DQN model label.

5. A blockchain task allocation device based on DQN, characterized in that, include: The parameter selection module is used to determine the state vector of the DQN model based on the preset state space, select the target action from the preset action space according to the state vector, and determine the predicted execution time and actual execution time of the target action. The experience addition module is used to take the predicted execution time and the actual execution time as inputs to a preset duration reward formula to obtain a duration reward, and add the state vector, the target action and the duration reward as the current experience to a preset experience replay buffer. The model update module is used to randomly extract a number of experiences from the preset experience replay buffer, determine the target Q value of the DQN model based on the number of experiences, and update the DQN model through the observed Q value of the DQN model and the target Q value. The model training module is used to jump to the steps of determining the state vector of the DQN model based on the preset state space, selecting a target action from the preset action space according to the state vector, and determining the predicted execution time and actual execution time of the target action, so as to update the DQN model in the next round until the DQN model reaches the preset requirements to obtain the target DQN model. The task allocation module is used to construct a target blockchain task allocation model through the target DQN model, and to intelligently allocate blockchain tasks through the target blockchain task allocation model after receiving a blockchain task allocation request. The model update module includes: The first Q-value determination unit is used to randomly extract several experiences from the preset experience playback buffer and use the several experiences as input to the Bellman equation to calculate the target Q value corresponding to the several experiences through the Bellman equation. The second Q-value determination unit is used to take the Q-value corresponding to the target action as the observed Q-value of the DQN model; The model update submodule is used to update the DQN model using the observed Q-value and the target Q-value; The model update submodule includes: The loss function determination unit is used to determine the Q-value error of the DQN model through the observed Q-value and the target Q-value, and to determine the current loss function of the DQN model based on the Q-value error; The model update unit is used to update the model parameters of the DQN model based on the negative gradient direction of the current loss function using the gradient descent algorithm, so as to perform the next round of updates on the DQN model based on the obtained updated model parameters; The DQN-based blockchain task allocation device further includes: The model building unit is used to verify the model error of the target DQN model based on the Q-value error. If the model error is less than a preset error threshold, the target blockchain task allocation model is built through the target DQN model.

6. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the DQN-based blockchain task allocation method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, Used to store a computer program, which, when executed by a processor, implements the DQN-based blockchain task allocation method as described in any one of claims 1 to 4.

Images