A processing method of a time-sensitive task and a related device

Abstract

This application discloses a method and related equipment for processing latency-sensitive tasks, which can be applied to the field of network resource scheduling technology. When a latency-sensitive task's computing power request is received, this application uses a deep reinforcement learning model trained with non-uniform federated aggregation to process the computing power of computing nodes. Then, based on real-time status data and network link monitoring data, it calculates the path fit of candidate transmission paths. Based on the path fit and a fit threshold, iterative optimization of a preset deep reinforcement learning model is performed to update the first set of computing nodes or select a target transmission path from the candidate transmission paths to execute the latency-sensitive task. This allows for the dynamic adaptation of computing nodes and network paths during the execution of latency-sensitive tasks, effectively reducing cumulative latency and ensuring that the execution of latency-sensitive tasks meets the constraints of service level agreement (SLA) standards, thus reducing the SLA breach rate.

Description

Technical Field

[0001] This application relates to the field of network resource scheduling technology, and in particular to a method and related equipment for processing latency-sensitive tasks. Background Technology

[0002] In related technologies, latency-sensitive tasks refer to tasks with strict upper limits on response time, such as obstacle avoidance control in autonomous driving, real-time grasping in industrial robots, AI in online games, or high-frequency trading decision-making. In the field of latency-sensitive tasks, existing computing power scheduling solutions mainly focus on node load balancing. However, in actual deployment scenarios, link latency can fluctuate due to network congestion, topology changes, etc. When this fluctuation exceeds a preset threshold, it will cause a sharp increase in task transmission latency, leading to a significant increase in task timeout rate. This independent approach of computing power scheduling and link selection results in the superposition of computing power scheduling latency and link latency, causing task latency to exceed the Service Level Agreement (SLA) constraints in complex dynamic scenarios, thereby increasing the SLA default rate.

[0003] Furthermore, because the centralized scheduling framework concentrates the entire process of "data collection-model computation-decision distribution" on a single scheduling node, additional latency occurs during data transmission, further increasing the cumulative latency of latency-sensitive tasks. Moreover, in federated learning frameworks, differences in network round-trip latency between nodes in different regions lead to inconsistent node training progress, resulting in an imbalance in global model optimization. Ultimately, this causes a decrease in node decision accuracy, making it difficult to provide precise scheduling strategies for latency-sensitive tasks.

[0004] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0005] The main objective of this application is to propose a method and related equipment for processing latency-sensitive tasks, which can effectively reduce superimposed latency, ensure that the execution process of latency-sensitive tasks meets the constraints of the Service Level Agreement (SLA), and reduce the SLA breach rate.

[0006] To achieve the above objectives, one aspect of this application proposes a method for processing time-latency-sensitive tasks, the method comprising the following steps: When a cross-domain routing node receives a computing power request for a latency-sensitive task, it processes the computing power of each computing power node through a deep reinforcement learning model trained with non-uniform federated aggregation, thus obtaining the first set of computing power nodes. Obtain real-time status data of each computing node in the first computing node set and network link monitoring data in the first computing node set; Based on the real-time status data and the network link monitoring data, calculate the path adaptability of all candidate transmission paths formed by all computing nodes in the first computing node set. When the fitness of all the paths is less than the fitness threshold, a delay feedback term is calculated based on the path fitness and the network link monitoring data; the reward function of the preset deep reinforcement learning model is adjusted based on the delay feedback term; the decision weight of the preset deep reinforcement learning model is adjusted based on the adjusted reward function; the preset deep reinforcement learning model with adjusted decision weights is called to re-process cross-domain node computing power to update the first computing power node set. When there exists a path fit degree greater than or equal to the fit degree threshold, a target transmission path is selected from the candidate transmission paths whose path fit degree is greater than or equal to the fit degree threshold to execute the delay-sensitive task.

[0007] In some embodiments, the non-uniform federated aggregation training process includes the following steps: In the global controller, the aggregate weight is calculated based on the network adaptation coefficient and data quality coefficient of each computing power node to be trained; The global controller sends model parameters and node hierarchical identifiers to all the computing power nodes to be trained; the node hierarchical identifiers are used to indicate the latency status of the corresponding computing power node to be trained; the model parameters include the aggregate weights; Within each of the computing power nodes to be trained, the deep reinforcement learning model corresponding to the model parameters is trained locally based on the node hierarchical identifier and local data, and then the target gradient is sent to the global controller. Within the global controller, all the target gradients are aggregated and calculated based on the aggregated weights to update the model parameters.

[0008] In some embodiments, calculating the aggregate weight based on the network adaptation coefficient and data quality coefficient of each computing power node to be trained includes: The network adaptation coefficient is calculated based on the baseline network round-trip time and the actual network round-trip time for each computing power node to be trained; The data quality coefficient is calculated based on the local dataset size and the global dataset size of each computing power node to be trained; The aggregate weight of each computing power node to be trained is calculated based on the network adaptation coefficient and the data quality coefficient.

[0009] In some embodiments, the step of locally training the deep reinforcement learning model corresponding to the model parameters based on the node hierarchical identifier and local data includes: When the latency status of the computing power node to be trained corresponding to the node hierarchical identifier is low latency, the deep reinforcement learning model corresponding to the model parameters is fully trained based on local data. When the latency status of the computing power node to be trained corresponding to the node hierarchical identifier is medium latency or high latency, the deep reinforcement learning model corresponding to the model parameters is incrementally trained based on local data.

[0010] In some embodiments, the calculation formula for the time delay feedback term is as follows: T_d = Path fit × Delay reward coefficient - Packet loss rate × Packet loss penalty coefficient; In the formula, T_d represents the value of the time delay feedback term.

[0011] In some embodiments, the reward function is calculated using the following formula: R = {Latency Coordination Coefficient × [User-preset Maximum Tolerable Latency - (Node Processing Delay + Path Transmission Delay)] / User-preset Maximum Tolerable Latency} - Resource Consumption Penalty Coefficient × (CPU Utilization + Memory Utilization) + Success Rate Reward Coefficient × Task Request Success Rate - Cross-Domain Imbalance Penalty Coefficient × Node Accuracy Deviation; In the formula, R represents the value of the reward function; The delay coordination coefficient is dynamically adjusted through the delay feedback term.

[0012] In some embodiments, the method further includes the following steps: When the target transmission path is in an abnormal state, an alternative transmission path is selected from the remaining candidate transmission paths whose path adaptability is greater than or equal to the adaptability threshold to execute the latency-sensitive task.

[0013] To achieve the above objectives, another aspect of the embodiments of this application proposes a processing apparatus for latency-sensitive tasks, the apparatus comprising: The first module is used to process the computing power of each computing power node to be processed through a deep reinforcement learning model trained by non-uniform federated aggregation when a cross-domain routing node obtains a computing power request for a latency-sensitive task, so as to obtain the first set of computing power nodes. The first module is used to acquire real-time status data of each computing node in the first computing node set and network link monitoring data in the first computing node set. The second module is used to calculate the path adaptability of all candidate transmission paths formed by all computing nodes in the third computing node set based on the real-time status data and the network link monitoring data. The fourth module is used to calculate a latency feedback term based on the path fitness and the network link monitoring data when all the path fitness is less than the fitness threshold; adjust the reward function of the preset deep reinforcement learning model based on the latency feedback term; adjust the decision weight of the preset deep reinforcement learning model based on the adjusted reward function; and call the preset deep reinforcement learning model with adjusted decision weights to re-process cross-domain node computing power to update the first computing power node set. The fifth module is used to select a target transmission path from the candidate transmission paths whose path fitness is greater than or equal to the fitness threshold to perform the delay-sensitive task when there is a path fitness greater than or equal to the fitness threshold.

[0014] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0015] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described above.

[0016] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer program product, including a computer program that, when executed by a processor, implements the aforementioned method.

[0017] The embodiments of this application include at least the following beneficial effects: This application provides a method and related equipment for processing latency-sensitive tasks. This method, upon receiving a computing power request from a latency-sensitive task, processes each computing power node to be processed using a deep reinforcement learning model trained with non-uniform federated aggregation to obtain a first set of computing power nodes. Then, based on the real-time status data and network link monitoring data of each computing power node in the first set, it calculates the path fit of all candidate transmission paths formed by all computing power nodes in the first set. When all path fits are less than a fit threshold, a latency feedback term is calculated based on the path fit and network link monitoring data. The preset deep reinforcement learning model is then adjusted based on the latency feedback term. Reward function; adjust the decision weights of the preset deep reinforcement learning model according to the adjusted reward function; call the preset deep reinforcement learning model with adjusted decision weights to re-process cross-domain node computing power to update the first computing power node set; when there is a path fitness degree greater than or equal to the fitness degree threshold, select the target transmission path from the candidate transmission paths with path fitness degree greater than or equal to the fitness degree threshold to execute the latency-sensitive task, so that node computing power and link status can be considered at the same time when executing the latency-sensitive task, and the dynamic adaptation process of computing power node selection and network path can be realized, thereby effectively reducing the superimposed latency, so that the execution process of the latency-sensitive task meets the service level agreement constraint standards, and reducing the service level agreement default rate. Attached Figure Description

[0018] Figure 1 This is a flowchart of a method for processing a latency-sensitive task provided in an embodiment of this application; Figure 2 This is a flowchart of non-uniform federated aggregation training provided in an embodiment of this application; Figure 3 This is a flowchart illustrating the calculation of aggregate weights based on the network adaptation coefficient and data quality coefficient of each computing power node to be trained, provided in an embodiment of this application. Figure 4 This is a complete implementation flowchart of a method for processing latency-sensitive tasks provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a processing device for latency-sensitive tasks provided in an embodiment of this application; Figure 6 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0020] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0021] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0023] Before providing a detailed description of the embodiments of this application, some of the nouns and terms used in the embodiments of this application will be explained first. The nouns and terms used in the embodiments of this application shall be interpreted as follows: The D3QN (Dueling Double DQN) model is a practical and powerful deep reinforcement learning model. By combining two DQN (Deep Q-Network) models, D3QN can significantly improve learning stability and sample efficiency while maintaining the simplicity of the DQN framework.

[0024] Latency-sensitive tasks typically refer to those with strict upper limits on response time, such as obstacle avoidance control in autonomous driving, real-time grasping in industrial robots, online game AI, or high-frequency trading decision-making. These tasks place extremely high demands on the inference speed, computational overhead, and stability of the algorithms.

[0025] Reinforcement learning (RL) is a technique that teaches optimal policies through interaction between an agent and its environment, with the core objective of maximizing cumulative reward. Among these algorithms, the D3QN (Deep Double Dueling Q-Network Algorithm) is a significant branch of reinforcement learning. It employs a deep neural network to approximate the Q-value function, addressing the curse of dimensionality problem faced by traditional reinforcement learning in high-dimensional state spaces. Furthermore, it leverages a double Q-network to reduce overestimation of value, improving the stability of the learning process and providing a foundation for policy learning in decision optimization. This ensures the stability and effectiveness of policy learning, meeting the optimization needs of complex scenarios.

[0026] Federated Learning (FL) is a distributed machine learning framework that enables multiple nodes to train models locally and then aggregate parameters under the coordination of a central node, achieving cross-node collaborative model optimization. It is suitable for scenarios where data is widely distributed and difficult to collect centrally in real time. The Federated Averaging (FedAvg) algorithm serves as its baseline parameter aggregation method. Clients perform local stochastic gradient descent training, and the server then performs a weighted average of the model parameters from each client, enabling collaborative training and optimization of multi-node models. This approach reduces inter-node communication overhead while ensuring data privacy and improving model convergence speed and training stability.

[0027] Cross-domain routing nodes are routers or Layer 3 switches responsible for forwarding data packets between different Autonomous Systems (AS) or different network domains, typically running inter-domain routing protocols such as BGP (Border Gateway Protocol). Computing power nodes are physical or virtual nodes with computing, storage, or data processing capabilities, usually located at the network edge or in data centers, used to perform application-layer tasks. For example, cross-domain routing nodes are responsible for routing user requests to the nearest computing power node (such as an MEC edge server), while the computing power node announces its own status to the routing node through BGP extended attributes (such as computing power strength), achieving "network awareness of computing power, and computing power guiding the network."

[0028] In the scheduling of latency-sensitive tasks, existing methods have the following problems: First, there is insufficient coordination between computing power scheduling and network transmission. Existing computing power scheduling schemes focus on optimizing node load balancing, without fully considering the dynamic fluctuations in network path latency. In real-world deployment scenarios, link latency is easily affected by factors such as network congestion and topology changes, leading to fluctuations. When these fluctuations exceed a preset threshold, task transmission latency increases sharply, resulting in a significant rise in task timeout rates. Furthermore, traditional path planning algorithms perform path calculations based on static network topology models, failing to perceive the dynamic load changes of computing nodes in real time. In medium- to large-scale network deployment scenarios, the processing capacity of computing nodes exhibits dynamic fluctuations with changes in task queue length. Such algorithms do not coordinate the optimization of node processing latency and path transmission latency, causing the cumulative latency of tasks to exceed SLA constraints, significantly increasing the SLA default rate.

[0029] Second, the centralized scheduling architecture suffers from excessive communication overhead. The global state collection scheme employed in centralized scheduling requires scheduling nodes to aggregate the resource status of all computing nodes and all network link information in real time. The amount of state data transmitted for a single scheduling decision is large. In medium- to large-scale node deployment scenarios, data transmission time far exceeds the scheduling time limits for latency-sensitive tasks, causing delays in scheduling command issuance and making it difficult for tasks to start on time. The core flaw of this architecture lies in centralizing the entire process of "data collection - model calculation - decision issuance" into a single scheduling node. The additional latency generated during data transmission further compresses the effective scheduling window. For latency-sensitive tasks with extremely high response speed requirements, this communication overhead triggers a chain reaction leading to task timeouts.

[0030] Third, federated learning models suffer from low training efficiency. Due to geographical differences, nodes in different regions exhibit significant variations in network round-trip time (RTT). The communication latency of nodes in some regions is only a fraction of that of nodes in other regions, leading to a difference of several times in model training convergence speed. Because traditional federated learning frameworks lack training synchronization mechanisms adapted to these latency differences, the training progress among nodes is severely inconsistent. Model updates in high-latency regions consistently lag behind those in low-latency regions, resulting in a global model optimization imbalance. Ultimately, the decision-making accuracy of nodes in high-latency regions decreases significantly, making it difficult to provide precise scheduling strategies for latency-sensitive tasks.

[0031] In view of this, the present application provides a method and related equipment for processing latency-sensitive tasks, which can effectively reduce superimposed latency, so that the execution process of latency-sensitive tasks meets the service level agreement (SLA) constraint standards and reduces the SLA default rate.

[0032] This application provides a method for processing latency-sensitive tasks, relating to the field of network resource scheduling technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited thereto. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing a method for processing latency-sensitive tasks, but is not limited to the above forms.

[0033] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0034] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0035] The embodiments of this application will be described in detail below with reference to the accompanying drawings: Figure 1 This is an optional flowchart of a time-latency-sensitive task processing method provided in an embodiment of this application. Figure 1 The method may include, but is not limited to, steps S110 to S150: Step S110: When a cross-domain routing node obtains a computing power request for a latency-sensitive task, it performs computing power processing on each computing power node to be processed through a deep reinforcement learning model trained by non-uniform federated aggregation to obtain the first set of computing power nodes. Step S120: Obtain the real-time status data of each computing node in the first computing node set and the network link monitoring data in the first computing node set; Step S130: Based on real-time status data and network link monitoring data, calculate the path adaptability of all candidate transmission paths formed by all computing nodes in the first computing node set. Step S140: When the fitness of all paths is less than the fitness threshold, calculate the latency feedback term based on the path fitness and network link monitoring data; adjust the reward function of the preset deep reinforcement learning model based on the latency feedback term; adjust the decision weight of the preset deep reinforcement learning model based on the adjusted reward function; call the preset deep reinforcement learning model with adjusted decision weights to re-process cross-domain node computing power to update the first computing power node set. Step S150: When there is a path fit degree greater than or equal to the fit degree threshold, select the target transmission path from the candidate transmission paths with path fit degree greater than or equal to the fit degree threshold to perform a delay-sensitive task.

[0036] Understandably, when a user initiates a computing power request for a latency-sensitive task, the request is transmitted to the cross-domain routing node, thereby triggering the subsequent computing power and path coordination process. In this embodiment, the computing power request carries information such as task type, end-to-end latency constraint threshold, bandwidth threshold, packet loss rate threshold, and task priority.

[0037] It is understood that, in this embodiment, after receiving the computing power request for a latency-sensitive task, a deep reinforcement learning model trained with non-uniform federated aggregation will be used to process the computing power of each node to be processed, thereby obtaining a first set of computing power nodes. The deep reinforcement learning model in this embodiment can be the D3QN model. Specifically, as... Figure 2 As shown, the non-uniform federated aggregation training process in this embodiment includes, but is not limited to, the following steps: Step S210: Calculate the aggregate weight in the global controller based on the network adaptation coefficient and data quality coefficient of each computing power node to be trained; Step S220: Send model parameters and node hierarchical identifiers to all computing power nodes to be trained through the global controller; wherein, the node hierarchical identifier is used to represent the latency status of the corresponding computing power node to be trained; the model parameters include aggregate weights; Step S230: On each computing power node to be trained, after training the deep reinforcement learning model corresponding to the model parameters locally according to the node hierarchical identifier and local data, the target gradient is sent to the global controller. Step S240: In the global controller, aggregate all target gradients based on aggregate weights to update the model parameters.

[0038] Specifically, such as Figure 3 As shown, the process of calculating the aggregate weight based on the network adaptation coefficient and data quality coefficient of each computing power node to be trained in this embodiment includes, but is not limited to, the following steps: Step S310: Calculate the network adaptation coefficient based on the baseline network round-trip time and the actual network round-trip time of each computing power node to be trained; Step S320: Calculate the data quality coefficient based on the local dataset size and the global dataset size of each computing power node to be trained; Step S330: Calculate the aggregate weight of each computing power node to be trained based on the network adaptation coefficient and the data quality coefficient.

[0039] The network adaptation coefficient is calculated as: Baseline network round-trip time (RTT) / Actual network round-trip time (RTT) of the node. The baseline RTT is a preset average RTT baseline value for cross-domain nodes (the computing power nodes to be trained). The smaller the actual RTT of the computing power nodes to be trained, the closer the network adaptation coefficient is to 1, and the higher the corresponding aggregation weight. The data quality coefficient is calculated as: (Local dataset size / Total global dataset size) × Sample matching degree. The sample matching degree can be the degree of fit between the local dataset samples and the current task type, with a value range of [0,1].

[0040] It is understood that this embodiment can divide the computing power nodes to be trained into low-latency, medium-latency, and high-latency layer nodes according to the network round-trip latency. Specifically, during the local training of the deep reinforcement learning model corresponding to the model parameters based on the node layer identifier and local data, when the latency state of the computing power node to be trained corresponding to the node layer identifier is low latency, the deep reinforcement learning model corresponding to the model parameters is fully trained based on the local data to obtain the full gradient; when the latency state of the computing power node to be trained corresponding to the node layer identifier is medium latency or high latency, the deep reinforcement learning model corresponding to the model parameters is incrementally trained based on the local data to obtain the incremental gradient, that is, only the key feature layer parameters of the model are synchronized, thereby significantly reducing cross-domain communication overhead.

[0041] Based on the above, the complete implementation process of non-uniform federated aggregation training in this application embodiment is as follows: Step 1, Global Initialization: The global controller sends initial model parameters and node hierarchical identifiers to all computing power nodes to be trained; Step 2, Local Training: Each computing power node to be trained conducts model training based on the local dataset; among them, low-latency layer nodes perform full training and output full gradients, while medium- and high-latency layer nodes perform incremental training and output incremental gradients. Step 3, Non-uniform Aggregation: The global controller aggregates the target gradients uploaded by each computing power node to be trained according to the dynamically allocated aggregation weights to generate global update parameters; the global update parameters are preferentially distributed to low-latency layer nodes to update the model parameters in the low-latency layer nodes; the global update parameters are compressed and distributed to mid-to-high-latency layer nodes to update the key parameters of the model in the mid-to-high-level nodes. Step 4, Iterative Convergence: Repeat the “local training-gradient aggregation-parameter update” process until the variance of the model prediction accuracy of each computing power node to be trained is lower than the preset threshold, and then end the current round of training. This can ensure the balanced accuracy of cross-domain node scheduling decisions.

[0042] Specifically, to improve the stability of model training, this embodiment employs a dual experience pool storage mechanism for data storage. The dual experience pools include a low-latency experience pool and a global experience pool. The low-latency experience pool stores scheduling experience data for low-latency layer nodes, while the global experience pool stores scheduling experience data for all cross-domain nodes. In this embodiment, during the training process of the deep reinforcement learning model in each training epoch, data can be preferentially sampled from the low-latency layer experience pool to ensure rapid convergence of the deep reinforcement learning model. Simultaneously, data from the global experience pool is periodically fused to prevent overfitting of the deep reinforcement learning model.

[0043] In this embodiment, after completing the training of the deep reinforcement learning model for the current round, the deep reinforcement learning model trained through non-uniform federated aggregation is used to process the computing power of each computing power node to obtain a first set of computing power nodes. Then, after obtaining the real-time status data of each computing power node in the first set of computing power nodes and the network link monitoring data in the first set of computing power nodes, the path fit of all candidate transmission paths formed by all computing power nodes in the first set of computing power nodes is calculated based on the real-time status data and the network link monitoring data.

[0044] Specifically, the real-time status data in this embodiment includes node processing latency, task request success rate, node availability, CPU and memory utilization, etc. Network link monitoring data includes real-time transmission latency, bandwidth utilization, packet loss rate, bit error rate, jitter, etc. The path adaptation degree in this embodiment can be characterized by a path adaptation quality score. The formula for calculating the path adaptation quality score Q(P) is as follows: Q(P) = w1 f(NodeHealth)+w2 g(LinkQuality)+w3 h(Stability); In the formula, f(NodeHealth) represents the node health score calculated based on real-time status data; g(LinkQuality) represents the link status score calculated based on bandwidth utilization, bit error rate, and real-time transmission latency; w1 represents the historical evaluation score calculated based on jitter and historical availability; and w1, w2, and w3 represent coefficients that are dynamically adjusted according to the task type.

[0045] A higher path fit quality score indicates a higher path fit degree, and a better match between the corresponding network link and the current task. Therefore, this embodiment selects network transmission links based on path fit degree.

[0046] Specifically, when the fitness scores of all paths are less than the fitness threshold, a latency feedback term is calculated based on the path fitness scores and network link monitoring data; the reward function of the preset deep reinforcement learning model is adjusted based on the latency feedback term; the decision weights of the preset deep reinforcement learning model are adjusted based on the adjusted reward function; and the preset deep reinforcement learning model with adjusted decision weights is called to re-process cross-domain node computing power to update the first computing power node set. The fitness threshold can be set according to the actual application scenario, for example, it can be set to 80%, 90%, etc.

[0047] The formula for calculating the delay feedback term T_d in this embodiment is as follows: T_d = Path fit × Delay reward coefficient - Packet loss rate × Packet loss penalty coefficient.

[0048] Among them, the latency reward coefficient and the packet loss penalty coefficient are preset hyperparameters, and their values ​​range from [0,1].

[0049] The formula for calculating the reward function R in this embodiment is as follows: R = {Latency Coordination Coefficient × [User-preset Maximum Tolerable Latency - (Node Processing Delay + Path Transmission Delay)] / User-preset Maximum Tolerable Latency} - Resource Occupation Penalty Coefficient × (CPU Utilization + Memory Utilization) + Success Rate Reward Coefficient × Task Request Success Rate - Cross-Domain Imbalance Penalty Coefficient × Node Accuracy Deviation

[0050] Specifically, in this embodiment, the latency coordination coefficient is dynamically adjusted by the latency feedback term (the higher the path adaptability, the larger the latency coordination coefficient, with a value range of 0.5-1.0). The cross-domain imbalance penalty coefficient is derived inversely from the node aggregation weight output by the non-uniform federation aggregation module (the lower the node aggregation weight, the smaller the cross-domain imbalance penalty coefficient, avoiding excessive penalties for high-latency nodes due to low aggregation weights). The resource consumption penalty coefficient and success rate reward coefficient are preset balanced hyperparameters (both with values ​​ranging from 0-1).

[0051] This embodiment adjusts the reward function through a delay feedback term, thereby reducing the model reward value when the path fit is low, which in turn reduces the probability of selecting the computing power node in the subsequent scheduling process.

[0052] Understandably, after calculating the reward function, this embodiment adjusts the decision weights of the preset deep reinforcement learning model based on the latest reward function. This allows for a re-processing of cross-domain node computing power based on the adjusted preset deep reinforcement learning model, thus updating the first set of computing power nodes. Specifically, when all path fit in the current round is less than the fit threshold, this embodiment restarts the "computing power node selection - path fit" process. This utilizes the preset deep reinforcement learning model corresponding to the latest decision weights to re-process cross-domain node computing power, effectively avoiding the problem of optimal computing power nodes but insufficient transmission path fit. This reduces the latency of time-sensitive task execution, ensuring that the execution of latency-sensitive tasks meets the service level agreement (SLA) constraints and lowering the SLA breach rate.

[0053] Understandably, if a path fit is greater than or equal to the fit threshold in the current round, a target transmission path can be selected from the candidate transmission paths with a path fit greater than or equal to the fit threshold to execute the latency-sensitive task. Specifically, since there are multiple computing nodes in the latest first set of computing nodes, these multiple computing nodes can form multiple transmission paths as candidate transmission paths. The computing power in the computing nodes involved in each candidate transmission path is different, and the network link monitoring data on the transmission path is also different. Therefore, this embodiment calculates the path fit between each candidate transmission path and the current latency-sensitive task, filters out candidate transmission paths with a path fit less than the fit threshold, and retains candidate transmission paths with a path fit greater than or equal to the fit threshold. Then, the candidate transmission path with the highest suitability is selected from the remaining candidate transmission paths as the target transmission path to execute the current latency-sensitive task. This can effectively reduce the superimposed latency of computing power latency and link latency, so that the execution process of latency-sensitive tasks meets the service level agreement constraints and reduces the service level agreement default rate.

[0054] It is understandable that, since the link status and the remaining computing power of nodes change in real time, when executing the current latency-sensitive task through the target transmission path, the target transmission path may experience sudden congestion, causing latency fluctuations to exceed the fluctuation threshold. In this case, this embodiment can select an alternative transmission path from the candidate transmission paths retained in the current round to replace the target transmission path for executing the current latency-sensitive task, thereby effectively improving the stability of latency-sensitive task execution. Specifically, this embodiment also updates the latency feedback term in real time, thereby dynamically updating and optimizing the deep reinforcement learning model to ensure that the end-to-end latency continuously meets the service level agreement constraints.

[0055] In some embodiments, such as Figure 4 As shown, the complete implementation process of the method in this application includes the following steps: Step 1: The user initiates a latency-sensitive computing power request; Step 2: When a cross-domain routing node receives a latency-sensitive computing power request, it calls the D3QN model trained by non-uniform federation to process the computing power of all the computing power nodes to be processed, and then obtains the optimal set of computing power nodes as the first set of computing power nodes. Step 3: Based on the real-time status data of the computing nodes in the first computing node set, and combined with the network link dynamic monitoring data, calculate the path adaptability of all candidate transmission paths formed by all computing nodes in the first computing node set, so as to measure the path adaptability quality through the path adaptability; at the same time, generate a latency feedback item. Step 4: When the path adaptation quality of all candidate transmission paths fails to meet the standard, trigger full-link iterative optimization and execute Step 5; when the path adaptation quality of a candidate transmission path meets the standard, determine the target transmission path and generate a joint scheduling decision to execute latency-sensitive tasks through cross-domain routing nodes based on the joint scheduling decision. Step 5: Feedback the latency term back to the reward function of the D3QN model to calculate the latest reward function. Adjust the decision weights dynamically according to the latest reward function so that the D3QN model can be trained using the latest decision weights in the new round of non-uniform federated aggregation training. Update the first computing node set according to the retrained D3QN model.

[0056] As can be seen from the above, this embodiment utilizes a combination of non-uniform federated aggregation and latency feedback closed loop to achieve dynamic adaptation of computing node selection and network path, thereby effectively reducing superimposed latency and ensuring that the execution process of latency-sensitive tasks meets the service level agreement (SLA) constraint standards, thus reducing the SLA default rate.

[0057] Please see Figure 5This application also provides a processing apparatus for latency-sensitive tasks, the apparatus comprising: The first module is used to process the computing power of each computing power node to be processed through a deep reinforcement learning model trained by non-uniform federated aggregation when a cross-domain routing node obtains a computing power request for a latency-sensitive task, so as to obtain the first set of computing power nodes. The first module is used to acquire real-time status data of each computing node in the first computing node set and network link monitoring data in the first computing node set; The second module is used to calculate the path adaptability of all candidate transmission paths formed by all computing nodes in the third computing node set based on real-time status data and network link monitoring data. The fourth module is used to calculate a latency feedback term based on path fitness and network link monitoring data when all path fitness is less than the fitness threshold; adjust the reward function of the preset deep reinforcement learning model based on the latency feedback term; adjust the decision weight of the preset deep reinforcement learning model based on the adjusted reward function; and call the preset deep reinforcement learning model with adjusted decision weights to re-process cross-domain node computing power to update the first computing power node set. The fifth module is used to select a target transmission path from the candidate transmission paths whose path fitness is greater than or equal to the fitness threshold to perform a delay-sensitive task when there is a path fitness greater than or equal to the fitness threshold.

[0058] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0059] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0060] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0061] Please see Figure 6 , Figure 6 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 610 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 620 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 620 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 620 and is called and executed by the processor 610 using the methods described in the embodiments of this application. The input / output interface 630 is used to realize information input and output; The communication interface 640 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 650 transmits information between various components of the device (e.g., processor 610, memory 620, input / output interface 630, and communication interface 640); The processor 610, memory 620, input / output interface 630 and communication interface 640 are connected to each other within the device via bus 650.

[0062] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0063] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0064] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0065] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0066] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0067] The processing method and related device for time-sensitive tasks provided in this application have the following beneficial effects: First, this embodiment uses a non-uniform federated aggregation method to perform dynamic weight allocation and hierarchical incremental synchronization. While reducing cross-domain training communication overhead, it ensures the model accuracy balance of nodes in different latency regions, thus solving the problem of training lag of high-latency nodes caused by the traditional federated averaging algorithm. Secondly, this embodiment achieves real-time coordination between computing node selection and network path planning through the optimized design of the D3QN model and the dynamic linkage of the latency feedback loop, avoiding the SLA default risk caused by a single optimization dimension and significantly improving the end-to-end latency compliance rate of latency-sensitive tasks. Third, this embodiment ensures that scheduling decisions can dynamically adapt to network link fluctuations and node load changes through feedback mechanisms and iterative optimization processes, avoiding the disconnect between "optimal computing power nodes but insufficient path adaptation", improving scheduling stability in complex dynamic scenarios, enhancing the accuracy and completeness of scheduling decisions, and reducing the communication overhead of centralized architecture, thereby achieving the global optimization goal of "high SLA compliance rate, low cross-domain communication overhead, and balanced cross-domain accuracy".

[0068] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0069] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0070] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0071] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0072] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0073] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0074] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0075] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0076] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0077] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0078] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for processing time-latency-sensitive tasks, characterized in that, The method includes the following steps: When a cross-domain routing node receives a computing power request for a latency-sensitive task, it processes the computing power of each computing power node through a deep reinforcement learning model trained with non-uniform federated aggregation, thus obtaining the first set of computing power nodes. Obtain real-time status data of each computing node in the first computing node set and network link monitoring data in the first computing node set; Based on the real-time status data and the network link monitoring data, calculate the path adaptability of all candidate transmission paths formed by all computing nodes in the first computing node set. When the fitness of all the paths is less than the fitness threshold, a delay feedback term is calculated based on the path fitness and the network link monitoring data; the reward function of the preset deep reinforcement learning model is adjusted based on the delay feedback term; the decision weight of the preset deep reinforcement learning model is adjusted based on the adjusted reward function; the preset deep reinforcement learning model with adjusted decision weights is called to re-process cross-domain node computing power to update the first computing power node set. When there exists a path fit degree greater than or equal to the fit degree threshold, a target transmission path is selected from the candidate transmission paths whose path fit degree is greater than or equal to the fit degree threshold to execute the delay-sensitive task.

2. The method according to claim 1, characterized in that, The non-uniform federated aggregation training process includes the following steps: In the global controller, the aggregate weight is calculated based on the network adaptation coefficient and data quality coefficient of each computing power node to be trained; The global controller sends model parameters and node hierarchical identifiers to all the computing power nodes to be trained; the node hierarchical identifiers are used to indicate the latency status of the corresponding computing power node to be trained; the model parameters include the aggregate weights; Within each of the computing power nodes to be trained, the deep reinforcement learning model corresponding to the model parameters is trained locally based on the node hierarchical identifier and local data, and then the target gradient is sent to the global controller. Within the global controller, all the target gradients are aggregated and calculated based on the aggregated weights to update the model parameters.

3. The method according to claim 2, characterized in that, The calculation of aggregate weights based on the network adaptation coefficient and data quality coefficient of each computing power node to be trained includes: The network adaptation coefficient is calculated based on the baseline network round-trip time and the actual network round-trip time for each computing power node to be trained; The data quality coefficient is calculated based on the local dataset size and the global dataset size of each computing power node to be trained; The aggregate weight of each computing power node to be trained is calculated based on the network adaptation coefficient and the data quality coefficient.

4. The method according to claim 2, characterized in that, The step of locally training the deep reinforcement learning model corresponding to the model parameters based on the node hierarchical identifier and local data includes: When the latency status of the computing power node to be trained corresponding to the node hierarchical identifier is low latency, the deep reinforcement learning model corresponding to the model parameters is fully trained based on local data. When the latency status of the computing power node to be trained corresponding to the node hierarchical identifier is medium latency or high latency, the deep reinforcement learning model corresponding to the model parameters is incrementally trained based on local data.

5. The method according to claim 1, characterized in that, The calculation formula for the time delay feedback term is as follows: T_d = Path fit × Delay reward coefficient - Packet loss rate × Packet loss penalty coefficient; In the formula, T_d represents the value of the time delay feedback term.

6. The method according to claim 5, characterized in that, The formula for calculating the reward function is as follows: R = {Latency Coordination Coefficient × [User-preset Maximum Tolerable Latency - (Node Processing Delay + Path Transmission Delay)] / User-preset Maximum Tolerable Latency} - Resource Consumption Penalty Coefficient × (CPU Utilization + Memory Utilization) + Success Rate Reward Coefficient × Task Request Success Rate - Cross-Domain Imbalance Penalty Coefficient × Node Accuracy Deviation; In the formula, R represents the value of the reward function; The delay coordination coefficient is dynamically adjusted through the delay feedback term.

7. The method according to claim 1, characterized in that, The method further includes the following steps: When the target transmission path is in an abnormal state, an alternative transmission path is selected from the remaining candidate transmission paths whose path adaptability is greater than or equal to the adaptability threshold to execute the latency-sensitive task.

8. A processing device for time-delay-sensitive tasks, characterized in that, The device includes: The first module is used to process the computing power of each computing power node to be processed through a deep reinforcement learning model trained by non-uniform federated aggregation when a cross-domain routing node obtains a computing power request for a latency-sensitive task, so as to obtain the first set of computing power nodes. The first module is used to acquire real-time status data of each computing node in the first computing node set and network link monitoring data in the first computing node set. The second module is used to calculate the path adaptability of all candidate transmission paths formed by all computing nodes in the third computing node set based on the real-time status data and the network link monitoring data. The fourth module is used to calculate a latency feedback term based on the path fitness and the network link monitoring data when all the path fitness is less than the fitness threshold; adjust the reward function of the preset deep reinforcement learning model based on the latency feedback term; adjust the decision weight of the preset deep reinforcement learning model based on the adjusted reward function; and call the preset deep reinforcement learning model with adjusted decision weights to re-process cross-domain node computing power to update the first computing power node set. The fifth module is used to select a target transmission path from the candidate transmission paths whose path fitness is greater than or equal to the fitness threshold to perform the delay-sensitive task when there is a path fitness greater than or equal to the fitness threshold.

9. An electronic device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images