Edge node scheduling system, edge node scheduling method and edge node

Abstract

The invention relates to the field of wireless communication, in particular to an edge node scheduling system, an edge node scheduling method and an edge node.The edge node scheduling system comprises a data request module, a management back-end module, a performance collector, a scheduler, a predictor and a controller, classifying and putting into a dynamic queue; the management back-end module calls a concurrent reasoning model to process data in batches according to resource and task requirements, and generates a network performance index; the performance collector continuously monitors and collects the indexes and the resource utilization rate; the scheduler formulates a node resource scheduling strategy accordingly; the predictor predicts resource interference and assists the scheduler to make a decision; and the controller switches the model according to the scheduling strategy, adjusts batch size and concurrency number, and realizes dynamic optimization of resources. Through the innovative edge node scheduling framework, localized efficient processing of the data is realized, the execution efficiency of the AI reasoning task is improved, and the resource utilization rate is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and in particular to an edge node scheduling system, an edge node scheduling method and an edge node. Background Art

[0002] As a distributed computing architecture, edge computing migrates applications from cloud center nodes to edge nodes for processing, which is closer to user terminals than traditional cloud computing models, thereby significantly improving data processing and transmission speeds and reducing latency. The edge computing model has shown significant advantages in real-time data processing, security, privacy protection, scalability, location awareness, and traffic optimization. However, with the rapid development of the industrial Internet and the continuous expansion of application scenarios, the complexity of edge applications is increasing, the number of edge nodes is growing rapidly, access devices are becoming more complex, and the amount of data generated is also expanding dramatically, which has brought unprecedented challenges to the management of edge devices and applications.

[0003] Specifically, the complex and ever-changing network topology and unstable network access environment, as well as the need for highly customized and fast iterative updates, have put forward higher requirements for edge computing. In addition, the heterogeneous hardware characteristics of edge gateways also bring additional difficulties to resource management. In this context, the existing edge computing technology has exposed obvious deficiencies in computing power coordination and intelligent scheduling. For example, edge nodes do not fully utilize computing resources, GPU utilization is low, and it is difficult to meet computing power requirements; traditional fixed-precision models and simple scheduling strategies cannot take into account the latency, accuracy and service level objectives (SLO) of requests; the lack of support for multiple application scenarios and adaptive learning capabilities limits the practical value and application prospects of the technology; at the same time, the existing concurrent model framework often ignores the problems of resource competition and interference model reasoning when dealing with model parallelism, and fails to fully utilize the advantages of parallel models. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides an edge node scheduling system, an edge node scheduling method and an edge node. The edge node scheduling system includes: A data request module is used to receive multiple request data from a terminal device, each request data includes a service delay target indicator value, and put the multiple request data into different dynamic task request queues according to their data types; A management backend module is used to take out request data from the dynamic task request queue, call the concurrent reasoning model to batch process the request data according to the computing resource situation of the current node and the task requirements, and obtain a network performance index value; wherein the network performance index value includes concurrent reasoning model parameters, queue backlog data, accuracy, throughput and service delay target index values; A performance collector is used to monitor in real time and regularly collect network performance indicator values ​​and computing resource utilization; A scheduler, configured to perform data processing on the network performance indicator value and the computing resource utilization rate to obtain a node resource scheduling strategy for allocating tasks in a dynamic computing resource environment; A predictor, configured to perform data processing on the network performance indicator value and the computing resource utilization rate, obtain an evaluation result of the current node resource scheduling strategy output by the scheduler in each iteration, and feed back the evaluation result to the scheduler to change the probability of the scheduler selecting the current node resource scheduling strategy; and a controller, for switching concurrent reasoning models to perform data processing and reallocate network computing resources according to the node resource scheduling strategy; The scheduler is used to process the network performance indicator value and the computing resource utilization rate to obtain a node resource scheduling strategy for allocating tasks in a dynamic computing resource environment, specifically including: Constructing a node resource scheduling model, inputting the network performance indicator value and the computing resource utilization rate as system operation status data into the node resource scheduling model, constructing a reward function based on throughput and service delay targets, and performing iterative training with the goal of maximizing the value of the reward function to obtain a trained node resource scheduling model; Based on the trained node resource scheduling model, the current system operation status data is input to obtain the action with the maximum reward value, that is, the node resource scheduling strategy.

[0005] In one embodiment of the present invention, obtaining a trained node resource scheduling model specifically includes: Constructing the state space , Action Space and strategy space ,in Indicates the current time interval The accuracy of the concurrent reasoning model, the number of concurrent reasoning model instances, the batch size of each concurrent reasoning model, the number of unprocessed requests in the task queue, the accuracy of the output of the concurrent reasoning model, the average throughput, the constraint target of the service delay, the CPU utilization, the memory utilization, the GPU utilization, and the GPU memory utilization; Indicates the current time interval The concurrent inference model accuracy, number of concurrent inference model instances, and batch size specified for model running are required; The strategy space Initialize the parameters of the current Q network and use the current Q network to calculate the current state The Q-values ​​of all possible actions , and select actions according to the ε-greedy strategy a , perform actions in the environment a , the reward r is calculated through the reward function and the next state is selected , according to the reward r and the next state , using the target Q network to calculate the selected action a The target Q value ; Use target Q value and the Q value predicted by the current Q network Calculate the loss value, and update the parameters of the current Q network by back-propagating the minimization of the loss value until the network converges or reaches the maximum iteration round, and obtain the trained node resource scheduling model; The current Q network is a neural network structure of multiple convolutional layers and multiple fully connected layers, and the structure of the target Q network is the same as that of the current Q network.

[0006] In one embodiment of the present invention, actions are selected according to the ε-greedy strategy a , specifically including: Set the initial value ε of the exploration probability, generate a random number n between 0 and 1, and determine the size of n and ε: like , randomly select an action from the action space A a ; like , select the predicted action with the largest Q value a .

[0007] In one embodiment of the present invention, the reward function The expression used to balance the system throughput and service latency targets is as follows: , in, , , All are hyperparameters; Indicates the accuracy of throughput; Indicates the accuracy of the service delay target; represents the throughput of the i-th period; represents the service delay target for the i-th period; Indicates the GPU utilization during the i-th period.

[0008] In one embodiment of the present invention, the expression of the target Q value y is as follows: , in, is the discount factor, Indicates the status of the update. According to the status , execute the action with the maximum Q value, represents the target Q network.

[0009] In one embodiment of the present invention, the predictor is used to perform data processing on the network performance indicator value and the computing resource utilization to obtain the evaluation result of the current node resource scheduling strategy output by the scheduler for each iteration calculation, specifically including: Get the scheduler's last state s and last strategy The action a selected below is the current node resource scheduling strategy. Based on the current node resource scheduling strategy, the predictor uses a pre-trained two-layer deep neural network to obtain the predicted values ​​of the system service delay target and throughput; Calculate the percentage difference between the predicted values ​​of the service delay target and throughput and the actual values ​​of the service delay target and throughput , according to the difference percentage The comparison result with the set threshold value is used to obtain the evaluation result of the current node resource scheduling strategy; at the same time, the difference percentage is used To update the parameters of the two-layer deep neural network.

[0010] Based on the same inventive concept, the present invention also provides an edge node scheduling method, which uses the above-mentioned edge node scheduling system to implement node resource allocation. The method includes the following steps: S1: receiving multiple request data from a terminal device, each request data including a service delay target indicator value, and placing the multiple request data into different dynamic task request queues according to their data types; S2: taking out request data from the dynamic task request queue, calling the concurrent reasoning model to batch process the request data according to the computing resource situation of the current node and the task requirements, and obtaining a network performance index value; wherein the network performance index value includes concurrent reasoning model parameters, queue backlog data, accuracy, throughput and service delay target index values; S3: real-time monitoring and regular collection of the network performance index value and computing resource utilization; S4: performing data processing on the network performance indicator value and the computing resource utilization rate to obtain a node resource scheduling strategy for allocating tasks in a dynamic computing resource environment; S5: switching the concurrent reasoning model to perform data processing and reallocating network computing resources according to the node resource scheduling strategy; S6: Perform data processing on the network performance indicator value and the computing resource utilization to obtain an evaluation result of the current node resource scheduling strategy output by each iterative calculation, and feed the evaluation result back to step S4 to change the probability of selecting the current node resource scheduling strategy.

[0011] In one embodiment of the present invention, in step S4, the method for obtaining a node resource scheduling strategy for allocating tasks in a dynamic computing resource environment includes: S41: construct a node resource scheduling model, input the network performance indicator value and the computing resource utilization rate as system operation status data into the node resource scheduling model, construct a reward function based on throughput and service delay targets, perform iterative training with the goal of maximizing the value of the reward function, and obtain a trained node resource scheduling model; S42: Based on the trained node resource scheduling model, the current system operation status data is input to obtain the action with the maximum reward value, that is, the node resource scheduling strategy.

[0012] The present invention also provides an edge node, which includes a processor, a memory and a bus system, wherein the processor and the memory are connected through the bus system, the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the above-mentioned edge node scheduling method.

[0013] The above technical solution of the present invention has the following advantages compared with the prior art: 1. Meet service level objectives: Ensure that each request can achieve the service level objective (SLO) while taking into account throughput and latency, thereby significantly improving system reliability.

[0014] 2. Fully improve resource utilization: By using different models to call various hardware resources of the GPU, it can respond quickly under real-time or near real-time conditions, effectively reducing the cost of upward transmission, thereby improving the overall user experience.

[0015] 3. Realize intelligent scheduling: The constructed node resource scheduling model uses each scheduling operation as experience for learning, so that it can intelligently select model accuracy, batch_size of each model, and number of concurrent instances in subsequent scheduling, further improving the accuracy of scheduling.

[0016] 4. Adapt to diverse scenarios: After each scenario change, the node resource scheduling model can learn autonomously and automatically select the appropriate model combination according to the current load pressure without manual reselection of the model, thereby reducing the workload of model selection and enhancing the system's adaptability and robustness in the face of unknown or changing environments.

[0017] 5. Reduce the interference caused by resource competition: In each scheduling process, the current action space will be predicted to effectively prevent the interference caused by resource competition. This measure can avoid the performance degradation caused by resource conflicts, thereby enhancing the stability and response speed of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein: Figure 1 It is a schematic diagram of the working principle of the edge node scheduling system provided in the first embodiment of the present invention; Figure 2 is a flow chart of the edge node scheduling method provided in the second embodiment of the present invention; Explanation of the reference numerals in the specification: 10. Data request module; 20. Management backend module; 30. Performance collector; 40. Scheduler; 50. Predictor; 60. Controller. DETAILED DESCRIPTION

[0019] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0020] Embodiment 1

[0021] See also Figure 1 As shown, the present invention provides an edge node scheduling system, the edge node scheduling system comprising: The data request module 10 is used to receive multiple request data from the end-side device (such as a smart phone, a sensor, etc.), each request data includes a service delay objective (SLO) indicator value, that is, the performance requirement for the reasoning (such as response time, accuracy, etc.), and put the multiple request data into different dynamic task request queues according to their data types; The management backend module 20 is used to take out the request data from the dynamic task request queue, call the concurrent reasoning model to batch process the request data according to the computing resource situation of the current node and the task requirements, and obtain the network performance index value; wherein the network performance index value includes concurrent reasoning model parameters, queue backlog data, accuracy, throughput and service delay target index value; A performance collector 30, used for real-time monitoring and regular collection of the network performance index values ​​and computing resource utilization; A scheduler 40, configured to perform data processing on the network performance indicator value and the computing resource utilization rate to obtain a node resource scheduling strategy for allocating tasks in a dynamic computing resource environment; The predictor 50 is used to perform data processing on the network performance indicator value and the computing resource utilization rate to obtain the evaluation result of the current node resource scheduling strategy output by the scheduler 40 in each iteration calculation, and to feed back the evaluation result to change the probability of the scheduler selecting the current node resource scheduling strategy; And a controller 60 is used to switch the concurrent reasoning model to perform data processing and reallocate network computing resources according to the node resource scheduling strategy.

[0022] It can be seen from the above technical solution that the present invention sends the request data of the terminal device to the edge node nearby through the data request module 10, avoiding the delay caused by the long-distance transmission of data to the cloud, and reducing network congestion and bandwidth occupancy; considering that different request data may need to be processed by different inference models, the data in the request is classified according to the model requirements through the data request module 10, and the classified data is stored in different dynamic queues. These queues are highly flexible, and their size and number can be dynamically adjusted according to the inference requirements and data volume in the actual application scenario, thereby ensuring the efficiency and flexibility of data processing.

[0023] Furthermore, the AI ​​reasoning task is executed on the management backend module 20 of the edge node. According to the real-time computing resource status (such as CPU, memory, GPU, etc.) of the edge node and the task requirements, the AI ​​reasoning task is intelligently scheduled to avoid waste and overload of computing resources, meet the delay and SLO accuracy of different requests, and improve throughput, thereby improving resource utilization.

[0024] The performance collector 30 can monitor and collect network performance index values ​​and computing resource utilization in real time and regularly, providing comprehensive performance data support for the system. These performance data not only help system administrators to understand the operating status of the system in a timely manner, but also provide accurate data input for the scheduler 40 so that it can formulate a more reasonable node resource scheduling strategy.

[0025] The predictor 50 can predict the resource interference that the scheduler 40 may receive during data processing, and send the prediction results to the scheduler 40 for decision-making. This function not only improves the anti-interference ability of the system, but also ensures that the system can respond quickly when facing resource shortages or emergencies, and assists the scheduler 40 in formulating the optimal scheduling strategy.

[0026] The controller 60 can flexibly switch the concurrent reasoning model for data processing and reallocate network computing resources according to the node resource scheduling strategy. This mechanism ensures that the system can quickly adjust the resource allocation strategy to meet various complex business needs when facing different tasks and performance requirements.

[0027] Furthermore, the management backend module 20 includes functions such as loading models, AI reasoning, and controlling models, with the aim of using deep learning frameworks such as TensorRT, libtorch, onnx, paddle, and tvm to call the underlying hardware for reasoning.

[0028] Furthermore, the scheduler 40 is used to perform data processing on the network performance indicator value and the computing resource utilization to obtain a node resource scheduling strategy for allocating tasks in a dynamic computing resource environment, specifically including: Constructing a node resource scheduling model based on reinforcement learning, inputting the network performance indicator value and the computing resource utilization rate into the node resource scheduling model as system operation status data, constructing a reward function based on throughput and service delay targets, and performing iterative training with the goal of maximizing the value of the reward function to obtain a trained node resource scheduling model; Based on the trained node resource scheduling model, the current system operation status data is input to obtain the action with the maximum reward value, that is, the node resource scheduling strategy; The current Q network is a neural network structure of multiple convolutional layers and multiple fully connected layers, and the structure of the target Q network is the same as the structure of the current Q network.

[0029] The trained node resource scheduling model is obtained, which specifically includes: Constructing the state space , Action Space and strategy space ,in represents the accuracy of the concurrent reasoning model in the current time interval i, the number of concurrent reasoning model instances, the batch size of each concurrent reasoning model, the number of unprocessed requests in the task queue, the accuracy of the concurrent reasoning model output end, the average throughput, the constraint target of the service delay, the CPU usage, the memory usage, the GPU usage, and the GPU memory usage; Indicates the concurrent reasoning model accuracy, number of concurrent reasoning model instances, and batch size specified for model running that need to be run in the current time interval i; The strategy space Initialize the parameters of the current Q network and use the current Q network to calculate the Q values ​​of all possible actions in the current state s , and select actions according to the ε-greedy strategy a , specifically including: Set the initial value ε of the exploration probability, generate a random number n between 0 and 1, and determine the size of n and ε: like , randomly select an action from the action space A a ; like , select the predicted action with the largest Q value a ,Right now ; Perform actions in the environment a , the reward r is calculated through the reward function and the next state is selected , according to the reward r and the next state , using the target Q network to calculate the selected action a The target Q value y is: , in, is the discount factor, Indicates the status of the update. According to the status , execute the action with the maximum Q value; represents the target Q network; Use the target Q value y and the Q value predicted by the current Q network The loss value is calculated, and the parameters of the current Q network are updated by back-propagation by minimizing the loss value until the network converges or reaches the maximum iteration round, thereby obtaining a trained node resource scheduling model.

[0030] Specifically, the reward function The expression used to balance the system throughput and service latency targets is as follows: , in, 、 、 All are hyperparameters; Indicates the accuracy of throughput; Indicates the accuracy of the service delay target; represents the throughput of the i-th period; represents the service delay target for the i-th period; Indicates the GPU utilization during the i-th period.

[0031] In this embodiment, the predictor 50 can accurately predict the balance between the system throughput and the service delay target. This prediction result can intuitively show whether the scheduler 40 has adopted the optimal strategy. In addition, by comparing the predicted value with the actual value, the predictor 50 can obtain in advance the potential interference that these changes may cause to the reasoning model when the accuracy of the concurrent reasoning model carried by the management backend module 20, the batch size (batch_size) of each reasoning model, and the number of concurrent changes.

[0032] Furthermore, the predictor 50 is used to perform data processing on the network performance indicator value and the computing resource utilization to obtain the evaluation result of the current node resource scheduling strategy output by the scheduler 40 at each iteration, specifically including: Get the scheduler 40 in the last state s and the last strategy The action a selected under the above is the current node resource scheduling strategy. Based on the current node resource scheduling strategy, the predictor 50 uses a pre-trained two-layer deep neural network (DNN) to calculate the predicted values ​​of the system service delay target and throughput, and calculates the difference percentage between the predicted values ​​of the service delay target and throughput and the actual values ​​of the service delay target and throughput. , according to the difference percentage The comparison result with the set threshold value is used to obtain the evaluation result of the node resource scheduling strategy of the scheduler 40; if If the difference is greater than the set threshold, it means that the current node resource scheduling strategy will cause resource interference, and the predictor 50 will notify the scheduler 40 to reduce the probability of selecting the node resource scheduling strategy; at the same time, the difference percentage This process ensures that the predictor 50 can continuously learn and adapt to the changing environment, thereby continuously enhancing the node resource scheduling capability of the scheduler 40.

[0033] In this embodiment, the controller 60 implements node resource allocation according to the node resource scheduling strategy calculated by the scheduler 40, that is: When faced with different business scenarios and performance requirements, you can flexibly choose a higher-precision model (such as the FP32 model) to ensure data accuracy, or choose a lower-precision but more efficient model (such as the INT8, FP16 model) to accelerate the data processing process; Dynamically adjust the model's operating parameters, increase or decrease the batch size (batch_size) and adjust the number of concurrent model instances according to the current system load and data processing requirements; After executing a model configuration switch, the controller is responsible for starting or stopping the model instance. This feature ensures that the system can quickly and accurately adjust resource allocation when switching model configurations to meet new business needs and performance requirements.

[0034] To illustrate with a specific example, assume that the system originally runs the Tensor RT fp16 model with 2 parallel instances and a batch size of 16, and the Tensor RT int8 model with 1 parallel instance and a batch size of 16. Based on the calculation results of the scheduler 40, the controller 60 may switch these configurations to the Tensor RT int8 model with the number of parallel instances increased to 2 and the batch size remaining unchanged at 16; at the same time, the Tensor RT int4 model is added as another option with the number of parallel instances being 1 but the batch size being adjusted to 4. Such switching not only optimizes the accuracy and efficiency of the model, but also further improves the overall performance of the system by adjusting the number of concurrent instances and the batch size.

[0035] Embodiment 2

[0036] Based on the same inventive concept as that of the first embodiment, the present invention also provides an edge node scheduling method, which uses the system described in the first embodiment to allocate node resources, see Figure 2 As shown, the method comprises the following steps: S1: receiving multiple request data from a terminal device, each request data including a service delay target indicator value, and placing the multiple request data into different dynamic task request queues according to their data types; S2: taking out request data from the dynamic task request queue, calling the concurrent reasoning model to batch process the request data, and obtaining a network performance index value; wherein the network performance index value includes concurrent reasoning model parameters, queue backlog data, accuracy, throughput, and service delay target index values; S3: real-time monitoring and regular collection of the network performance index value and computing resource utilization; S4: performing data processing on the network performance indicator value and the computing resource utilization rate to obtain a node resource scheduling strategy for allocating tasks in a dynamic computing resource environment; S5: switching the concurrent reasoning model to perform data processing and reallocating network computing resources according to the node resource scheduling strategy; S6: Perform data processing on the network performance indicator value and the computing resource utilization to obtain an evaluation result of the current node resource scheduling strategy output by each iterative calculation, and feed the evaluation result back to step S4 to change the probability of selecting the current node resource scheduling strategy.

[0037] Further, in step S4, the method for obtaining a node resource scheduling strategy for allocating tasks in a dynamic computing resource environment includes: S41: construct a node resource scheduling model based on reinforcement learning, input the network performance indicator value and the computing resource utilization rate as system operation status data into the node resource scheduling model, construct a reward function based on throughput and service delay targets, perform iterative training with the goal of maximizing the value of the reward function, and obtain a trained node resource scheduling model; S42: Based on the trained node resource scheduling model, the current system operation status data is input to obtain the action with the maximum reward value, that is, the node resource scheduling strategy.

[0038] In this embodiment, the method of performing data processing on the network performance indicator value and the computing resource utilization rate to obtain the evaluation result of the current node resource scheduling policy output by each iterative calculation is the same as the method of performing data processing on the network performance indicator value and the computing resource utilization rate by a predictor to obtain the evaluation result of the current node resource scheduling policy output by each iterative calculation of the scheduler in Embodiment 1, and will not be repeated here.

[0039] Embodiment 3 The present invention also provides an edge node, which includes a processor, a memory and a bus system, wherein the processor and the memory are connected through the bus system, the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the steps of the edge node scheduling method described in the above-mentioned embodiment 2.

[0040] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0041] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0042] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0043] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0044] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims

1. An edge node scheduling system, characterized in that: Includes the following modules: A data request module is used to receive multiple request data from a terminal device, each request data includes a service delay target indicator value, and put the multiple request data into different dynamic task request queues according to their data types; A management backend module is used to take out request data from the dynamic task request queue, call the concurrent reasoning model to batch process the request data according to the computing resource situation of the current node and the task requirements, and obtain a network performance index value; wherein the network performance index value includes concurrent reasoning model parameters, queue backlog data, accuracy, throughput and service delay target index values; A performance collector is used to monitor in real time and regularly collect network performance indicator values ​​and computing resource utilization; A scheduler, configured to perform data processing on the network performance indicator value and the computing resource utilization rate to obtain a node resource scheduling strategy for allocating tasks in a dynamic computing resource environment; A predictor, configured to perform data processing on the network performance indicator value and the computing resource utilization rate, obtain an evaluation result of the current node resource scheduling strategy output by the scheduler in each iteration, and feed back the evaluation result to the scheduler to change the probability of the scheduler selecting the current node resource scheduling strategy; and a controller, for switching concurrent reasoning models to perform data processing and reallocate network computing resources according to the node resource scheduling strategy; The scheduler is used to process the network performance indicator value and the computing resource utilization rate to obtain a node resource scheduling strategy for allocating tasks in a dynamic computing resource environment, specifically including: Constructing a node resource scheduling model, inputting the network performance indicator value and the computing resource utilization rate as system operation status data into the node resource scheduling model, constructing a reward function based on throughput and service delay targets, and performing iterative training with the goal of maximizing the value of the reward function to obtain a trained node resource scheduling model; Based on the trained node resource scheduling model, the current system operation status data is input to obtain the action with the maximum reward value, that is, the node resource scheduling strategy.

2. The edge node scheduling system according to claim 1, characterized in that: The trained node resource scheduling model is obtained, which specifically includes: Constructing the state space , Action Space and strategy space ,in Indicates the current time interval The accuracy of the concurrent reasoning model, the number of concurrent reasoning model instances, the batch size of each concurrent reasoning model, the number of unprocessed requests in the task queue, the accuracy of the output of the concurrent reasoning model, the average throughput, the constraint target of the service delay, the CPU utilization, the memory utilization, the GPU utilization, and the GPU memory utilization; Indicates the current time interval The concurrent inference model accuracy, number of concurrent inference model instances, and batch size specified for model running are required; The strategy space Initialize the parameters of the current Q network and use the current Q network to calculate the current state The Q-values ​​of all possible actions , and select actions according to the ε-greedy strategy a , perform actions in the environment a , the reward r is calculated through the reward function and the next state is selected , according to the reward r and the next state , using the target Q network to calculate the selected action a The target Q value ; Use target Q value and the Q value predicted by the current Q network Calculate the loss value, and update the parameters of the current Q network by back-propagating the minimization of the loss value until the network converges or reaches the maximum iteration round, and obtain the trained node resource scheduling model; The current Q network is a neural network structure of multiple convolutional layers and multiple fully connected layers, and the structure of the target Q network is the same as that of the current Q network.

3. The edge node scheduling system according to claim 2, characterized in that: Select actions according to the ε-greedy strategy a , including: Set the initial value ε of the exploration probability, generate a random number n between 0 and 1, and determine the size of n and ε: like , randomly select an action from the action space A a ; like , select the predicted action with the largest Q value a .

4. The edge node scheduling system according to claim 2, characterized in that: The reward function The expression used to balance the system throughput and service latency targets is as follows: ， in, , , All are hyperparameters; Indicates the accuracy of throughput; Indicates the accuracy of the service delay target; represents the throughput of the i-th period; represents the service delay target for the i-th period; Indicates the GPU utilization during the i-th period.

5. The edge node scheduling system according to claim 2, characterized in that: The expression of the target Q value y is as follows: ， in, is the discount factor, Indicates the status of the update. According to the status , execute the action with the maximum Q value, represents the target Q network.

6. The edge node scheduling system according to claim 2, characterized in that: The predictor is used to perform data processing on the network performance indicator value and the computing resource utilization rate to obtain an evaluation result of the current node resource scheduling strategy output by the scheduler in each iteration calculation, specifically including: Get the scheduler's last state s and last strategy The action a selected below is the current node resource scheduling strategy. Based on the current node resource scheduling strategy, the predictor uses a pre-trained two-layer deep neural network to obtain the predicted values ​​of the system service delay target and throughput; Calculate the percentage difference between the predicted values ​​of the service delay target and throughput and the actual values ​​of the service delay target and throughput , according to the difference percentage The comparison result with the set threshold value is used to obtain the evaluation result of the current node resource scheduling strategy; at the same time, the difference percentage is used To update the parameters of the two-layer deep neural network.

7. An edge node scheduling method, characterized in that: Node resource allocation is implemented using the edge node scheduling system as described in any one of claims 1 to 6, the method comprising the following steps: S1: receiving multiple request data from a terminal device, each request data including a service delay target indicator value, and placing the multiple request data into different dynamic task request queues according to their data types; S2: taking out request data from the dynamic task request queue, calling the concurrent reasoning model to batch process the request data according to the computing resource situation of the current node and the task requirements, and obtaining a network performance index value; wherein the network performance index value includes concurrent reasoning model parameters, queue backlog data, accuracy, throughput and service delay target index values; S3: real-time monitoring and regular collection of the network performance index value and computing resource utilization; S4: performing data processing on the network performance indicator value and the computing resource utilization rate to obtain a node resource scheduling strategy for allocating tasks in a dynamic computing resource environment; S5: switching the concurrent reasoning model to perform data processing and reallocating network computing resources according to the node resource scheduling strategy; S6: Perform data processing on the network performance indicator value and the computing resource utilization to obtain an evaluation result of the current node resource scheduling strategy output by each iterative calculation, and feed the evaluation result back to step S4 to change the probability of selecting the current node resource scheduling strategy.

8. The edge node scheduling method according to claim 7, characterized in that: In step S4, the method for obtaining a node resource scheduling strategy for allocating tasks in a dynamic computing resource environment includes: S41: construct a node resource scheduling model, input the network performance indicator value and the computing resource utilization rate as system operation status data into the node resource scheduling model, construct a reward function based on throughput and service delay targets, perform iterative training with the goal of maximizing the value of the reward function, and obtain a trained node resource scheduling model; S42: Based on the trained node resource scheduling model, the current system operation status data is input to obtain the action with the maximum reward value, that is, the node resource scheduling strategy.

9. An edge node, characterized in that: The edge node includes a processor, a memory and a bus system, the processor and the memory are connected through the bus system, the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the edge node scheduling method described in claim 7 or claim 8.

Images