Deep reinforcement learning-based task time optimization method in edge computing network

Abstract

The present application relates to the technical field of edge computing, and, in particular, to a deep reinforcement learning-based task time optimization method in an edge computing network. The task time optimization method comprises: receiving a data acquisition task; according to the acquisition task, controlling first edge computing nodes to acquire surrounding data information, and generating a task; acquiring current operation state information of a first edge computing network; according to the operation state information, splitting the task into a first number of first tasks, and splitting the first tasks into a second number of sub-tasks having constraint relationships; according to the operation state information, on the basis of deep reinforcement learning, executing a corresponding scheduling decision on the sub-tasks; and according to the scheduling decision, issuing the sub-tasks to each of the corresponding edge computing nodes. The task execution time is optimized by means of splitting the tasks. The sub-tasks are issued to the corresponding edge computing nodes according to the scheduling decision so as to execute the plurality of sub-tasks in parallel, thereby improving task execution efficiency and increasing the utilization rate of the nodes in the edge computing network.

Description

Task time optimization method in edge computing network based on deep reinforcement learning Technical Field

[0001] The present application relates to the field of edge computing technology, and in particular to a method for optimizing task time in an edge computing network based on deep reinforcement learning. Background Art

[0002] With the rapid development of the Internet of Things (IoT), the development of edge computing (EC) technology has effectively alleviated the problems of cloud computing in terms of network resource utilization efficiency and poor user experience quality. However, edge computing is facing significant challenges in optimizing user demand response.

[0003] To improve response speed to user requests, related technologies schedule tasks submitted by individual users on a single edge service network. However, this results in low node utilization within the edge computing network. Furthermore, the computing power and response speed of edge computing nodes within a single edge computing network are limited, making it difficult to complete tasks in a timely manner. This results in long task execution times and low efficiency. When the amount of data in a task is too large, this can also cause latency within the edge computing network, resulting in a poor user experience.

[0004] Summary of the Invention

[0005] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0006] This application aims to solve at least one of the technical problems existing in the prior art. To this end, the embodiment of the present application provides a task time optimization method in an edge computing network based on deep reinforcement learning, which is conducive to solving the problem of long task execution time and low efficiency. By splitting the tasks, the task execution time is optimized; according to the scheduling decision, subtasks are sent to the corresponding edge computing nodes to execute multiple subtasks in parallel, thereby improving the execution efficiency of the tasks and improving the utilization rate of the nodes in the edge computing network.

[0007] In a first aspect, an embodiment of the present application provides a method for optimizing task time in an edge computing network based on deep reinforcement learning, comprising: receiving a data collection task;

[0008] Controlling the first edge computing node to collect surrounding data information according to the collection task and generate a task;

[0009] Obtaining current operating status information of a first edge computing network formed by the first edge computing nodes;

[0010] Splitting the task into a first number of first tasks according to the running status information, and splitting the first task into a second number of subtasks having constraint relationships;

[0011] Execute a corresponding scheduling decision on the subtask based on the operating status information based on deep reinforcement learning;

[0012] The subtasks are distributed to the corresponding edge computing nodes according to the scheduling decision.

[0013] The technical solution of the above-mentioned first aspect of the present application has at least one of the following advantages or beneficial effects: splitting the task into a first number of first tasks according to the current operating status information of the first edge computing network, and splitting the first task into a second number of sub-tasks with constraint relationships, thereby realizing the splitting of tasks and optimizing the task execution time; dispatching sub-tasks to the corresponding edge computing nodes according to the scheduling decision to execute the sub-tasks, and executing multiple sub-tasks in parallel through the corresponding nodes, thereby improving the execution efficiency of the tasks and improving the utilization rate of the nodes in the edge computing network.

[0014] Furthermore, executing a corresponding scheduling decision on the subtask based on deep reinforcement learning according to the running status information includes:

[0015] When the operating status information indicates that the current load of the first edge computing network is insufficient to execute the subtask, the subtask is sent to each second edge computing node in the second edge computing network for execution;

[0016] When the operating status information indicates that the current load of the first edge computing network is sufficient to execute the subtask, the second task is executed.

[0017] Further, the performing of the second task includes:

[0018] Obtaining remaining idle nodes of the first edge computing network according to the operating status information;

[0019] When the number of the remaining idle nodes is greater than the number of nodes required to execute the subtask, the subtask is sent to the remaining idle nodes in the first edge computing network.

[0020] Furthermore, the method further comprises:

[0021] When the number of remaining idle nodes is less than the number of nodes required to execute the subtask, a third number of subtasks corresponding to the number of remaining idle nodes is sent to the corresponding first edge computing nodes in the first edge computing network; and a fourth number of subtasks that are not assigned edge computing nodes are sent to the corresponding second edge computing nodes in the second edge computing network.

[0022] Furthermore, the method further comprises:

[0023] Set the minimum split unit;

[0024] Determining the number of splits of the task according to the operating status information of the first edge computing network and the minimum split unit;

[0025] The task is split according to the split quantity of the task to obtain a first number of first tasks, and the first tasks are split into a second number of subtasks having constraint relationships.

[0026] Furthermore, after issuing the subtask to each corresponding edge computing node according to the scheduling decision, the method further includes:

[0027] Update the operating status information in the first edge computing network.

[0028] Further, updating the operating status information in the first edge computing network includes:

[0029] Obtaining a state space and an action space of the first edge computing network;

[0030] Inputting the state space and the action space into the trained DQN algorithm model to obtain a Q-value function of the cumulative reward value obtained by the first edge computing network after executing the action space in the state space;

[0031] The Q-value function is updated to update the operating status information in the first edge computing network.

[0032] Furthermore, the trained DQN algorithm model is obtained through the following training steps:

[0033] Obtaining an action state for training and a target Q-value function corresponding to the action state;

[0034] Training the DQN algorithm model using the action state as input of the DQN algorithm model and the target Q-value function as output of the DQN algorithm model to obtain a primary DQN algorithm model;

[0035] Input the updated action state into the primary DQN algorithm model to obtain the predicted Q-value function;

[0036] Obtaining a loss function using the target Q-value function and the predicted Q-value function;

[0037] The primary DQN algorithm model is iteratively updated using the loss function, and when the number of iterative updates reaches a preset number, a trained DQN algorithm model is obtained.

[0038] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present application. The purposes and other advantages of the present application can be achieved and obtained through the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] FIG1 is a flowchart of a method for optimizing task time in an edge computing network based on deep reinforcement learning according to an embodiment of the present application;

[0040] FIG2 is a flowchart of the steps of S500 in FIG1 ;

[0041] FIG3 is a flowchart of the steps of S530 in FIG2 ;

[0042] FIG4 is another step flow chart of S530 in FIG2 ;

[0043] FIG5 is a flowchart of another method for optimizing task time in an edge computing network based on deep reinforcement learning provided by an embodiment of the present application;

[0044] FIG6 is a flowchart of another method for optimizing task time in an edge computing network based on deep reinforcement learning provided by an embodiment of the present application;

[0045] FIG7 is a flowchart of the steps of S800 in FIG6 ;

[0046] Figure 8 is a flow chart of the training process of the DQN algorithm model. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementation methods. At the same time, the steps or actions in the method description can also be swapped or adjusted in order in a manner that is obvious to those skilled in the art. Therefore, the various orders in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a necessary order, unless otherwise specified that a certain order must be followed.

[0048] In the description of this application, "several" means one or more, "many" means more than two, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. The terms "first" and "second" are used solely to distinguish technical features and are not to be construed as indicating or implying relative importance, or as implicitly specifying the number or order of the technical features indicated.

[0049] The serial numbers assigned to components herein, such as "first," "second," etc., are used solely to distinguish the objects being described and do not convey any sequential or technical meaning. References to "connection" and "coupling" herein, unless otherwise specified, include both direct and indirect connections (couplings).

[0050] In related technologies, for time-sensitive applications, edge computing places user-submitted tasks within a single edge service network for scheduling in order to generate faster network service applications and improve response speed. However, this results in low node utilization within the edge computing network. Furthermore, the computing power of edge computing nodes within a single edge computing network is limited, and there is transmission delay between nodes when transmitting tasks. This results in slow response times and the inability to complete tasks in a timely manner, resulting in long task execution times and low efficiency. When the amount of data in a task is too large, this can also cause delays in the edge computing network, resulting in a poor user experience.

[0051] To this end, an embodiment of the present application provides a task time optimization method in an edge computing network based on deep reinforcement learning, which is conducive to solving the problem of long task execution time and low efficiency. By splitting the tasks, the task execution time is optimized, and the corresponding scheduling decisions are made for the task execution according to the running status information. According to the scheduling decisions, the subtasks are sent to the corresponding edge computing nodes to execute multiple subtasks in parallel, thereby improving the execution efficiency of the tasks and improving the utilization rate of the nodes in the edge computing network.

[0052] 1 , which is a flowchart of a method for optimizing task time in an edge computing network based on deep reinforcement learning according to an embodiment of the present application, including steps S100 to S600. Specifically,

[0053] S100: receiving data collection tasks;

[0054] S200: Control the first edge computing node to collect surrounding data information according to the collection task and generate a task;

[0055] S300: Obtain current operating status information of a first edge computing network composed of a first edge computing node;

[0056] S400: Splitting the task into a first number of first tasks according to the running status information, and splitting the first task into a second number of subtasks having constraint relationships;

[0057] S500: Execute corresponding scheduling decisions on subtasks based on the operating status information based on deep reinforcement learning;

[0058] S600: Send subtasks to corresponding edge computing nodes according to the scheduling decision.

[0059] According to the current operating status information of the first edge computing network, the task is split into a first number of first tasks, and the first task is split into a second number of subtasks with constraint relationships, thereby realizing the splitting of the task and optimizing the task execution time; according to the scheduling decision, the subtasks are sent to the corresponding edge computing nodes to execute the subtasks, and multiple subtasks are executed in parallel through the corresponding nodes, thereby improving the execution efficiency of the task and improving the utilization rate of the nodes in the edge computing network.

[0060] It should be noted that the data information in the embodiment of the present application includes image or video data collected by the first edge computing node through a camera, or monitoring video data resources obtained through a monitoring device. The present application does not limit the type of data information.

[0061] It should be noted that the operating status information in the embodiment of the present application includes one or more of the current load status of the edge computing network, the remaining idle nodes in the edge computing network, the type of nodes in the edge computing network, and the number of nodes.

[0062] It should be noted that in the embodiment of the present application, the first quantity is one or more of 5, 10, 15, and 20. The first quantity is determined based on the data information collected by the first edge computing node and the current operating status of the first edge computing network. The embodiment of the present application does not limit the size of the first quantity.

[0063] It should be noted that in the embodiment of the present application, the second number is one or more of 5, 10, 15, and 20. The second number is determined based on the size of the first number and the current operating status of the first edge computing network. The embodiment of the present application does not limit the size of the second number.

[0064] 2 , which is a flowchart of the steps of S500 in FIG1 , the corresponding scheduling decision for executing the subtask based on deep reinforcement learning according to the running state information includes steps S510 to S530. Specifically,

[0065] S510: Determine whether the current load of the first edge computing network indicated by the operating status information is sufficient to execute the subtask;

[0066] S520: When the operation status information indicates that the current load of the first edge computing network is insufficient to execute the subtask, the subtask is sent to each second edge computing node in the second edge computing network for execution;

[0067] S530: When the operating status information indicates that the current load of the first edge computing network is sufficient to execute the subtask, execute the second task.

[0068] By determining whether the current load situation in the first edge computing network is sufficient to execute the subtask, the execution node corresponding to the subtask is determined, thereby improving the response speed of the subtask and reducing the execution time of the subtask. Specifically, in some embodiments of the present application, when the current load situation of the first edge computing network is insufficient to execute the subtask, the subtask is sent to each second edge computing node in the second edge computing network for execution; when the current load situation of the first edge computing network is sufficient to execute the subtask, the second task is executed.

[0069] When the current load of the first edge computing network is insufficient to execute the subtask, the subtask is sent to each second edge computing node in the second edge computing network for execution. By sending the subtask to the corresponding computing nodes in the second edge computing network, the utilization rate of the nodes in the edge computing network is improved. By having the corresponding nodes execute multiple subtasks in parallel, the execution efficiency of the subtask is improved.

[0070] 3, FIG3 is a flowchart of the steps of S530 in FIG2, performing the second task includes steps S531 to S532, specifically,

[0071] S531: Obtain the remaining idle nodes of the first edge computing network according to the operation status information;

[0072] S532: When the number of remaining idle nodes is greater than the number of nodes required to execute the subtask, the subtask is sent to the remaining idle nodes in the first edge computing network.

[0073] The remaining idle nodes in the first edge computing network are obtained through operational status information, and the number of remaining idle nodes is determined to be greater than the number of nodes required to execute the subtask, thereby determining whether to execute the subtask. Only when the number of remaining idle nodes is determined to be greater than the number of nodes required to execute the subtask are the subtasks dispatched to the remaining idle nodes in the first edge computing network. This ensures that each node in the first edge computing network can execute the current subtask efficiently, effectively preventing stalls and improving the response efficiency of the current subtask.

[0074] 4, FIG4 is another step flow chart of S530 in FIG2, performing the second task further includes step S531 and step S533, specifically,

[0075] S531: Obtain the remaining idle nodes of the first edge computing network according to the operation status information;

[0076] S533: When the number of remaining idle nodes is less than the number of nodes required to execute the subtasks, send the third number of subtasks corresponding to the number of remaining idle nodes to the corresponding first edge computing nodes in the first edge computing network; and send the fourth number of subtasks that are not assigned edge computing nodes to the corresponding second edge computing nodes in the second edge computing network.

[0077] The remaining idle nodes of the first edge computing network are obtained through the operating status information, and it is determined whether the number of remaining idle nodes is greater than the number of nodes required to execute the subtask, thereby determining whether to execute the subtask. When it is determined that the number of remaining idle nodes is less than the number of nodes required to execute the subtask, a third number of subtasks corresponding to the number of remaining idle nodes are sent to the corresponding first edge computing nodes in the first edge computing network, ensuring that the load in the first edge computing network is sufficient to execute the current subtask and effectively preventing stalls; and a fourth number of subtasks that have not been assigned edge computing nodes are sent to the corresponding second edge computing nodes in the second edge computing network, thereby improving the utilization rate of nodes in the edge computing network. Multiple nodes execute subtasks in parallel, thereby improving the execution efficiency of subtasks.

[0078] It should be noted that in the embodiment of the present application, the third number is one or more of 5, 10, 15, and 20. The third number is determined based on the remaining idle nodes in the first edge computing network. The embodiment of the present application does not limit the size of the third number.

[0079] It should be noted that in the embodiment of the present application, the fourth number is one or more of 5, 10, 15, and 20. The fourth number is determined based on the third number and the number of subtasks. The embodiment of the present application does not limit the size of the fourth number.

[0080] 5 , which is a flowchart of another method for optimizing task time in an edge computing network based on deep reinforcement learning provided by an embodiment of the present application, including steps S700 to S720. Specifically,

[0081] S700: Set the minimum split unit;

[0082] S710: Determine the number of splits of the task according to the operating status information of the first edge computing network and the minimum split unit;

[0083] S720: Split the task according to the task splitting quantity to obtain a first number of first tasks, and split the first task into a second number of subtasks having constraint relationships.

[0084] By setting a minimum splitting unit and determining the number of task splits based on the minimum splitting unit and the operating status information of the first edge computing network, the task is split according to the splitting number so that the split subtasks adapt to the operating status of the first edge computing network, thereby optimizing the execution time of the task and improving the response efficiency of the subtasks.

[0085] In some embodiments of the present application, the task is split according to the number of split tasks to obtain a first number of first tasks, and the first tasks are split into a second number of subtasks with constraint relationships, including

[0086] Among them, m represents the mth task; n represents the nth subtask; The nth subtask of the mth task is executed on the jth node of the ith type, where I represents the node type in the edge computing network, J represents the node number, M represents the task number, and N represents the subtask number. By splitting the first task into a second number of constrained subtasks, the current operating status of the first edge computing network can be clearly and unambiguously obtained, thereby better allocating subtasks to corresponding nodes.

[0087] 6 , which is a flowchart of another method for optimizing task time in an edge computing network based on deep reinforcement learning according to an embodiment of the present application, including steps S300 to S800 . Specifically,

[0088] S300: Obtain current operating status information of a first edge computing network composed of a first edge computing node;

[0089] S400: Splitting the task into a first number of first tasks according to the running status information, and splitting the first task into a second number of subtasks having constraint relationships;

[0090] S500: Execute corresponding scheduling decisions on subtasks based on the operating status information based on deep reinforcement learning;

[0091] S600: Send subtasks to corresponding edge computing nodes according to the scheduling decision;

[0092] S800: Update the operating status information in the first edge computing network.

[0093] After executing corresponding scheduling decisions for subtasks based on the operating status information and dispatching the subtasks to the corresponding edge computing nodes based on the scheduling decisions, the method for optimizing task time in an edge computing network based on deep reinforcement learning also includes updating the operating status information in the first edge computing network. By updating the operating status information in the first edge computing network, subsequently collected tasks can be split based on the current operating status information in the first edge computing network and dispatched to the corresponding nodes based on the scheduling decisions, thereby improving the accuracy and reliability of task splitting and task scheduling.

[0094] 7, FIG7 is a flowchart of the steps of S800 in FIG6, updating the operating status information in the first edge computing network includes steps S810 to S830, specifically,

[0095] S810: Obtain a state space and an action space of a first edge computing network;

[0096] S820: Input the state space and action space into the trained DQN algorithm model to obtain a Q-value function of the cumulative reward value obtained by the first edge computing network after executing the action space in the state space;

[0097] S830: Update the Q-value function to update the operating status information in the first edge computing network.

[0098] Updating the operating status information in the first edge computing network includes obtaining a state space and an action space of the first edge computing network, inputting the state space and the action space into a trained DQN algorithm model, obtaining a Q-value function of a cumulative reward value, and updating the Q-value function to update the operating status information in the first edge computing network. The DQN algorithm model takes the state space and the action space as input, and then analyzes the algorithm model to obtain a Q-value function of the action space reward. Recording the Q-value function in a table is omitted, and the Q-value function is directly predicted by the DQN algorithm model, thereby improving the efficiency of updating the operating status information in the first edge computing network.

[0099] It should be noted that the state space is a triple parameter, including the remaining task amount, the number of tasks on the edge device, and the number of subtasks to be split. z To represent the size of the task, where z is the remaining size of the task after the last state; p is the number of subtasks split; v ij represents the number of tasks of the smallest unit u on the jth device of type i. Therefore, the state space can be expressed as: S = [z,v ij ,p].

[0100] It should be noted that the action space is a finite set of actions including action G. This means that the nth subtask of task m is assigned to the jth device of type i. Therefore, the action space can be expressed as:

[0101] 8 , which is a flow chart of the training process of the DQN algorithm model, including steps S910 to S950 , specifically,

[0102] S910: Obtaining an action state for training and a target Q-value function corresponding to the action state;

[0103] S920: Training the DQN algorithm model using the action state as input and the target Q-value function as output to obtain a primary DQN algorithm model.

[0104] S930: Input the updated action state into the primary DQN algorithm model to obtain the predicted Q-value function;

[0105] S940: Obtain a loss function using the target Q-value function and the predicted Q-value function;

[0106] S950: Iteratively update the primary DQN algorithm model using the loss function. When the number of iterative updates reaches a preset number, a trained DQN algorithm model is obtained.

[0107] The training process of the DQN algorithm model includes taking the action state used for training as the input of the DQN algorithm model and the corresponding target Q-value function as the output of the DQN algorithm model to train the DQN algorithm model to obtain a primary DQN algorithm model; inputting the updated action state into the primary DQN algorithm model to obtain a predicted Q-value function.

[0108] Specifically, Represents the output of the primary DQN algorithm model, which is used to evaluate the target Q-value function and action space of the current state space; This represents the predicted Q-value function obtained by inputting the updated action state into the primary DQN algorithm model. The loss function is derived by using the target Q-value function and the predicted Q-value function, and the primary DQN algorithm model is iteratively updated using the loss function. When the number of iterative updates reaches the preset number, the trained DQN algorithm model is obtained.

[0109] By using the loss function to update the primary DQN algorithm model for a preset number of iterations, a trained DQN algorithm model is obtained; when the updated state space is input into the trained DQN algorithm model, the predicted Q-value function can be quickly obtained, omitting the need to record the Q-value function in a table, and directly predicting the Q-value function through the DQN algorithm model, thereby improving the updating efficiency of the operating status information in the edge computing network and improving the accuracy of the DQN algorithm model prediction.

[0110] The embodiments of the present application are described in detail above in conjunction with the accompanying drawings, but the present application is not limited to the above embodiments. Various changes can be made within the scope of knowledge possessed by ordinary technicians in the technical field without departing from the purpose of the present application.

Claims

1. Task time optimization method in edge computing network based on deep reinforcement learning, It is characterized in that include: Receive data collection tasks; Controlling the first edge computing node to collect surrounding data information according to the collection task and generating a task; Obtain current operating status information of a first edge computing network formed by the first edge computing nodes; Splitting the task into a first number of first tasks according to the running status information, and splitting the first task into a second number of subtasks having constraint relationships; Execute a corresponding scheduling decision on the subtask based on the operating status information based on deep reinforcement learning; The subtasks are sent to the corresponding edge computing nodes according to the scheduling decision.

2. The method for optimizing task time in edge computing network based on deep reinforcement learning according to claim 1, It is characterized in that The executing corresponding scheduling decision of the subtask based on the deep reinforcement learning according to the running status information includes: When the running status information indicates that the current load of the first edge computing network is insufficient to execute the subtask, the subtask is sent to each second edge computing node in the second edge computing network for execution; When the operating status information indicates that the current load condition of the first edge computing network is sufficient to execute the subtask, the second task is executed.

3. The method for optimizing task time in edge computing network based on deep reinforcement learning according to claim 2, It is characterized in that The performing of the second task comprises: Obtaining remaining idle nodes of the first edge computing network according to the running status information; When the number of the remaining idle nodes is greater than the number of nodes required to execute the subtask, the subtask is sent to the remaining idle nodes in the first edge computing network.

4. The method for optimizing task time in edge computing network based on deep reinforcement learning according to claim 3, It is characterized in that The method further comprises: When the number of remaining idle nodes is less than the number of nodes required to execute the subtask, a third number of subtasks corresponding to the number of remaining idle nodes is sent to the corresponding first edge computing nodes in the first edge computing network; and a fourth number of subtasks that are not assigned edge computing nodes are sent to the corresponding second edge computing nodes in the second edge computing network.

5. The method for optimizing task time in edge computing network based on deep reinforcement learning according to claim 1, It is characterized in that The method further comprises: Set the minimum split unit; Determine the number of splits of the task according to the operating status information of the first edge computing network and the minimum split unit; The task is split according to the split quantity of the task to obtain a first quantity of first tasks, and the first tasks are split into a second quantity of subtasks having constraint relationships.

6. The method for optimizing task time in edge computing network based on deep reinforcement learning according to claim 1, It is characterized in that After sending the subtask to each corresponding edge computing node according to the scheduling decision, the method further includes: Update the operating status information in the first edge computing network.

7. The method for optimizing task time in edge computing network based on deep reinforcement learning according to claim 6, It is characterized in that The updating of the operating status information in the first edge computing network includes: Obtaining a state space and an action space of the first edge computing network; Inputting the state space and the action space into the trained DQN algorithm model to obtain a Q value function of the cumulative reward value obtained by the first edge computing network after executing the action space in the state space; The Q value function is updated to update the operating status information in the first edge computing network.

8. The method for optimizing task time in edge computing network based on deep reinforcement learning according to claim 7, It is characterized in that The trained DQN algorithm model is obtained through the following training steps: Acquire an action state for training and a target Q value function corresponding to the action state; The DQN algorithm model is trained by taking the action state as the input of the DQN algorithm model and the target Q value function as the output of the DQN algorithm model to obtain a primary DQN algorithm model; Inputting the updated action state into the primary DQN algorithm model to obtain a predicted Q-value function; Obtaining a loss function using the target Q value function and the predicted Q value function; The primary DQN algorithm model is iteratively updated using the loss function, and when the number of iterative updates reaches a preset number, a trained DQN algorithm model is obtained.

Images