Dynamic load balancing method and system for optical communication device

Abstract

The present application provides a method and system for dynamic load balancing of optical communication devices, which relates to the field of optical communication equipment management. The method includes: obtaining multi-dimensional time series data of each optical communication device; for each historical time step, extracting multi-dimensional data of each optical communication device corresponding to the historical time step in the multi-dimensional time series data, and constructing a corresponding historical graph structure according to each extracted multi-dimensional data, and updating the node features of each graph node in the historical graph structure through a graph neural network; for each optical communication device, determining a time series fusion feature based on the node features of the corresponding graph nodes of each updated historical graph structure, and inputting the time series fusion feature into a traffic prediction model to determine the predicted traffic load data corresponding to a preset time period in the future; and performing dynamic load balancing operations based on each predicted traffic load data. Thus, accurate load prediction and intelligent load balancing operations are achieved in a dynamically changing network environment.

Description

Technical Field

[0001] The present application relates to the technical field of optical communication equipment management, and in particular to a dynamic load balancing method and system for an optical communication device. Background Art

[0002] As the core of modern communication systems, optical communication technology has experienced decades of development. Its advantages of fast transmission speed, large bandwidth, and strong anti-interference have made it widely used in various communication scenarios, such as metropolitan area networks, long-distance communications, and data center interconnection. In these application scenarios, optical communication technology not only improves data transmission efficiency, but also enhances the reliability and stability of communication networks.

[0003] However, with the continuous expansion of network scale and the continuous growth of communication demand, the challenges faced by optical communication equipment in actual operation are becoming increasingly severe. In particular, in current optical communication networks, load balancing mainly relies on static configuration and predefined load distribution strategies. Although these methods have solved the load distribution problem to a certain extent, their defects have gradually emerged as the scale and complexity of the network increase.

[0004] Specifically, static configuration methods usually rely on administrators to manually set load distribution parameters, which may be effective during the initial deployment of the network, but is inadequate when faced with dynamic changes in network traffic. In addition, predefined load distribution strategies are usually based on the analysis of historical traffic data, but since network traffic tends to show large instantaneous fluctuations, these strategies are difficult to fully and effectively schedule communication resources globally, resulting in some nodes being overloaded during certain time periods, while other nodes may be idle, resulting in a waste of resources.

[0005] In response to the above problems, the industry has not yet proposed a better technical solution. Summary of the invention

[0006] The present application provides a dynamic load balancing method, system, storage medium, computer program product and electronic device for an optical communication device, which are used to at least solve the problem that static configuration or predefined load distribution strategies in current related technologies cannot cope with dynamic changes in network traffic, which easily leads to waste or overload of resources.

[0007] In a first aspect, an embodiment of the present application provides a dynamic load balancing method for an optical communication device, comprising: obtaining multi-dimensional time series data of each optical communication device in an optical communication network to be managed; the multi-dimensional time series data comprises a plurality of historical time steps and corresponding multi-dimensional data, the multi-dimensional data comprises traffic load data, device status data and external environment data; the traffic load data comprises bandwidth utilization, total traffic, upload rate, download rate, packet loss rate and transmission delay; for each of the historical time steps, extracting the multi-dimensional data of each optical communication device corresponding to the historical time step in the multi-dimensional time series data, and constructing a corresponding historical graph structure according to each of the extracted multi-dimensional data, and then Then, the node features of each graph node in the historical graph structure are updated through the graph neural network; the historical graph structure includes multiple graph nodes and edge connections, the node features of each graph node are defined by the multi-dimensional data of the corresponding optical communication device, and each edge connection is used to connect the graph nodes corresponding to the optical communication device with a link transmission relationship; for each optical communication device, the timing fusion feature is determined based on the node features of the corresponding graph nodes of each updated historical graph structure, and the timing fusion feature is input into the traffic prediction model to determine the predicted traffic load data corresponding to the preset time period in the future; based on the predicted traffic load data corresponding to each of the optical communication devices, a dynamic load balancing operation is performed.

[0008] In a second aspect, an embodiment of the present application provides a dynamic load balancing system for optical communication devices, including: a data acquisition unit, used to acquire multi-dimensional time series data of each optical communication device in the optical communication network to be managed; the multi-dimensional time series data includes multiple historical time steps and corresponding multi-dimensional data, and the multi-dimensional data includes traffic load data, device status data and external environment data; the traffic load data includes bandwidth utilization, total traffic, upload rate, download rate, packet loss rate and transmission delay; a feature extraction unit, used to extract the multi-dimensional data of each optical communication device corresponding to the historical time step in the multi-dimensional time series data for each historical time step, and construct a corresponding historical graph structure according to the extracted multi-dimensional data, and then Then, the node features of each graph node in the historical graph structure are updated through the graph neural network; the historical graph structure includes multiple graph nodes and edge connections, the node features of each graph node are defined by the multi-dimensional data of the corresponding optical communication device, and each edge connection is used to connect the graph nodes corresponding to the optical communication device with a link transmission relationship; the traffic prediction unit is used to determine the timing fusion features for each optical communication device based on the node features of the corresponding graph nodes of each updated historical graph structure, and input the timing fusion features into the traffic prediction model to determine the predicted traffic load data corresponding to the preset time period in the future; the load balancing unit is used to perform dynamic load balancing operations based on the predicted traffic load data corresponding to each of the optical communication devices.

[0009] According to a third aspect, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the dynamic load balancing method for an optical communication device of any embodiment of the present application.

[0010] In a fourth aspect, an embodiment of the present application provides a storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the steps of the dynamic load balancing method for an optical communication device of any embodiment of the present application are implemented.

[0011] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the dynamic load balancing method for an optical communication device of any embodiment of the present application.

[0012] The present application provides a method and system for dynamic load balancing of an optical communication device, which can produce at least the following technical effects:

[0013] (1) By acquiring multi-dimensional time series data and constructing a historical graph structure, the status and traffic load of each device in the optical communication network at different times can be effectively captured. By constructing a historical graph structure for each historical time step and updating it using a graph neural network, the dynamic change trend of the global network traffic can be captured, and the instantaneous fluctuation of network traffic can be responded to in a timely manner. By inputting the time series fusion features of multiple historical time steps of the optical communication device into the traffic prediction model, the future traffic load can be accurately predicted based on historical data. Furthermore, dynamic load balancing operations are performed based on the predicted traffic load data, which can optimize resource allocation on a global scale and avoid overloading of some nodes and idleness of other nodes. Therefore, by analyzing historical data and predicting future traffic load conditions, planning and adjusting the resource allocation of the optical communication network in advance, the resource utilization of the optical communication network can be significantly improved, ensuring that the load of each node is within a reasonable range, and improving the overall performance and reliability of the communication network.

[0014] (2) By introducing multi-dimensional time series data, including traffic load data, equipment status data and external environment data, it is possible to more comprehensively and accurately reflect the operating status of optical communication devices in different time periods. By analyzing and predicting multi-dimensional time series data, it is possible to accurately predict the traffic load of each optical communication device in the future, so as to reasonably allocate network resources in dynamic load balancing operations, avoid the situation where some nodes are overloaded while other nodes are idle, and optimize resource utilization.

[0015] (3) Through deep learning and analysis of multi-dimensional time series data using graph neural networks, the changing trends and traffic load characteristics of each node in the network can be accurately captured. In addition, by using graph neural networks to build and update historical graph structures, the link transmission relationship between various devices in the optical communication network can be fully considered. By extracting time series fusion features and applying traffic prediction models, the future traffic load data of each node can be accurately predicted, and dynamic adjustments can be made in advance, reducing the risk of overload and failure caused by sudden traffic changes in the network, thereby enhancing the reliability and stability of the optical communication network.

[0016] Through this technical solution, by introducing multi-dimensional time series data and graph neural network technology, accurate load prediction and intelligent load balancing operations can be achieved in a dynamically changing environment, which significantly improves the resource utilization, reliability and stability of the optical communication network, reduces manual intervention and configuration complexity, and comprehensively optimizes the overall network performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 A flowchart showing an example of a dynamic load balancing method for an optical communication device according to an embodiment of the present application is shown;

[0019] Figure 2 A schematic diagram showing a structural connection of an example of a meta-learning model according to an embodiment of the present application is shown;

[0020] Figure 3 An operation flow chart of an example of determining a load optimization adjustment instruction through a load balancing model according to an example of an embodiment of the present application is shown;

[0021] Figure 4 An operation flow chart showing an example of a second-stage construction operation in the construction of a meta-learning model according to an embodiment of the present application;

[0022] Figure 5 A schematic diagram of a structural connection of an example of a traffic prediction model according to an embodiment of the present application is shown;

[0023] Figure 6 A structural block diagram of an example of a dynamic load balancing system for an optical communication device according to an embodiment of the present application is shown;

[0024] Figure 7It is a schematic structural diagram of an embodiment of an electronic device of the present application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved shall comply with the provisions of relevant laws and regulations and shall not violate public order and good morals.

[0027] Figure 1 A flowchart of an example of a dynamic load balancing method for an optical communication device according to an embodiment of the present application is shown.

[0028] Regarding the executor of the method of the embodiment of the present application, it can be any controller or processor with computing or processing capabilities. By introducing artificial intelligence technologies such as graph neural networks and traffic prediction models, dynamic load balancing operations can be implemented for each device in the optical communication network, effectively improving the resource utilization, stability and reliability of the network.

[0029] In some examples, it can be an optical communication load balancing platform, and can be integrated and configured in an electronic device, terminal or server through software, hardware or a combination of software and hardware, and the types of terminals, electronic devices or servers can be diverse, such as mobile phones, tablet computers or desktop computers, etc.

[0030] like Figure 1 As shown, in step S110, multi-dimensional time series data of each optical communication device in the optical communication network to be managed is obtained, and the multi-dimensional time series data includes multiple historical time steps and corresponding multi-dimensional data.

[0031] In some embodiments, the multi-dimensional data includes traffic load data, device status data, and external environment data. The traffic load data includes bandwidth usage, total traffic, upload rate, download rate, packet loss rate, and transmission delay. The device status data includes at least one of the following: device failure information, computing resource consumption information, and memory resource consumption information. The external environment data includes at least one of the following: temperature, weather forecast information, and voltage fluctuation information.

[0032] More specifically, by deploying multiple sensors and monitoring systems in the optical communication network, multi-dimensional time series data of each optical communication device is collected in real time, traffic load data is collected through the traffic monitoring module, device status data is collected through the device management system, and external environment data is collected through the environment monitoring module. Furthermore, all collected data are stored in time steps to form multi-dimensional time series data. In this way, by collecting and storing multi-dimensional time series data, a comprehensive and detailed historical data foundation is achieved, providing accurate data support for subsequent traffic prediction and load balancing.

[0033] It should be noted that the device status data and external environment data in the multi-dimensional data can effectively improve the accuracy of future traffic prediction results and the robustness of risk response capabilities. For device status data, by real-time monitoring of device failure information, computing resource utilization, and storage resource utilization, it can effectively identify and locate performance degradation caused by device failure or device resource pressure, and accurately analyze abnormal changes in network traffic caused by abnormal working conditions, thereby optimizing the high accuracy of traffic prediction results. In addition, for external environmental data, by monitoring ambient temperature, weather forecast information, and voltage fluctuations, the interference of abnormal environmental events on the stable operation of equipment can be quickly captured, which helps to further optimize the high accuracy of traffic prediction results.

[0034] Through this embodiment, multi-dimensional time series data is introduced, covering traffic load data, device status data and external environment data, which can more comprehensively and accurately reflect the operating status of optical communication devices in different time periods. Through the analysis and prediction of multi-dimensional time series data, the traffic load of each optical communication device in the future period can be accurately predicted.

[0035] In step S120, for each historical time step, multi-dimensional data of each optical communication device corresponding to the historical time step in the multi-dimensional time series data is extracted, and a corresponding historical graph structure is constructed based on the extracted multi-dimensional data, and then the node features of each graph node in the historical graph structure are updated through the graph neural network.

[0036] Here, the historical graph structure includes multiple graph nodes and edge connections. The node characteristics of each graph node are defined by the multi-dimensional data of the corresponding optical communication device, including the traffic load, device status and external environment data of each optical communication device at the historical time step. Each edge connection is used to connect the graph node corresponding to the optical communication device with a link transmission relationship, and the edge connection in the graph structure is constructed according to the actual link topology.

[0037] Here, the type of Graph Neural Network (GNN) can be diverse, such as GAT (Graph Attention Network) or GCN (Graph Convolutional Network), etc. More preferably, the graph neural network adopts a deep graph convolution model, and more details will be expanded in conjunction with other examples below.

[0038] In some implementations, the historical graph structure is input into a trained graph neural network, and the features of each node are updated through multi-layer propagation. The features of each node are updated through a message passing mechanism so that it can reflect the mutual influence and association between various devices in the network.

[0039] Through this embodiment, the historical graph structure of each historical time step is constructed and updated, which can capture the topological relationship and data transmission characteristics of the optical communication device at different time steps, and can fully capture the link relationship between the devices. The node features are updated using the graph neural network, which fully considers the mutual influence between the various devices, can fully explore the implicit relationship in the multi-dimensional data, and improve the accuracy and expressiveness of the feature representation.

[0040] In step S130, for each optical communication device, a timing fusion feature is determined based on the node features of the corresponding graph nodes of each updated historical graph structure, and the timing fusion feature is input into the traffic prediction model to determine the predicted traffic load data corresponding to the preset time period in the future.

[0041] In some embodiments, for each optical communication device, the corresponding graph nodes are extracted from each updated historical graph structure, and the node features and corresponding time steps of each graph node are spliced ​​and fused to obtain the corresponding time series fusion features. Here, the time series fusion features comprehensively consider multi-dimensional data, time series changes and network global relationships, so as to more accurately reflect the future traffic load trend of the device. Then, the time series fusion features are input into the pre-trained traffic prediction model to predict the traffic load data within a preset time period in the future. The traffic prediction model can adopt various types of time series models, such as LSTM, GRU, etc. By extracting time series fusion features through the time series model, the time dynamic changes of node features can be effectively integrated to improve the timeliness and accuracy of traffic prediction results. In some examples of the embodiments of the present application, the traffic prediction model can adopt an LSTM model based on a multi-head attention mechanism. More details will be expanded in conjunction with other examples below.

[0042] In step S140 , a dynamic load balancing operation is performed based on the predicted traffic load data corresponding to each optical communication device.

[0043] It should be noted that various non-restrictive dynamic load balancing operations can be used here, for example, based on the predicted traffic load data, an optimization algorithm (such as linear programming, genetic algorithm, deep reinforcement learning, etc.) is used to make dynamic load balancing decisions to determine the optimal resource allocation plan for each optical communication device. Thus, according to the optimization decision results, by adjusting the load allocation parameters of each optical communication device in real time, a specific load balancing operation is performed to ensure the balance of network traffic distribution and improve the utilization efficiency of network resources.

[0044] Regarding the implementation details of the above step S140, in some embodiments, the predicted traffic load data corresponding to each optical communication device is input into the load balancing model to determine the load optimization adjustment instruction for at least one target optical communication device. Here, the load balancing model adopts a meta-learning model, and the load optimization adjustment instruction includes at least one of the following: adjusting traffic routing, enabling standby nodes, dynamically adjusting bandwidth allocation, and adjusting transmission paths.

[0045] It should be noted that the core of the meta-learning model is to enable the model to quickly adapt to new tasks by training on multiple tasks. In the embodiment of the present application, the meta-learning model is used to implement dynamic load balancing operations, which can achieve adaptability and generalization capabilities for multiple tasks. More specifically, multiple tasks are defined as load balancing requirements under different network load scenarios or network load types. Accordingly, the data sample set of the meta-learning model includes multiple data subsets, each of which has a corresponding load type. Specifically, the multiple data subsets include a peak load data subset, a valley load data subset, a regular load data subset, and a gradual load data subset. The peak load scenario can represent a scenario with high bandwidth utilization, high latency, and rising packet loss rate, such as a network carrying a large amount of data traffic during peak hours (such as morning and evening peak hours on weekdays), and requires rapid response and load distribution. The valley load scenario can represent a scenario with low bandwidth utilization and low latency, such as a network with less traffic during valley hours (such as late at night or holidays), but the efficient operation of the equipment must still be guaranteed. A normal load scenario can represent a scenario where traffic is stable and bandwidth demand is average, such as network traffic on a normal workday. A gradual load scenario can represent a gradual change in traffic, either increasing or decreasing, such as traffic changes during rush hour or before the end of the workday.

[0046] It should be understood that the requirements for load balancing control in different load scenarios are different, and in order to achieve adaptive control for different load scenarios. Here, a meta-learning model is adopted to achieve good performance on multiple tasks, and by fine-tuning each task, the optimal load distribution strategy is generated to achieve scene-adaptive dynamic load balancing. Therefore, by training on multiple scene data sets (such as peak load, valley load, regular load, and gradient load), the meta-learning model can quickly adapt to new tasks and new environments to ensure the efficient execution of load balancing strategies. In addition, based on the ability of multi-task learning, the model can predict and respond to various complex network environments and emergencies, and improve the overall adaptability and robustness of the network.

[0047] Figure 2 A schematic diagram of the structural connection of an example of a meta-learning model according to an embodiment of the present application is shown.

[0048] like Figure 2 As shown, the meta-learning model 200 includes a random forest classifier 210, a meta-learner 220 and multiple basic model modules (231, 233...23N), each of which has a corresponding load type. For example, the basic model module 231 is used to process the predicted peak load scenario, the basic model module 233 is used to process the predicted valley load scenario, and so on.

[0049] Figure 3 An operational flow chart of an example of determining a load optimization adjustment instruction through a load balancing model according to an example of an embodiment of the present application is shown.

[0050] like Figure 3 As shown, in step S310, a target load type matching each input predicted traffic load data is determined through a random forest classifier.

[0051] In some embodiments, the random forest classifier extracts and analyzes features of the predicted traffic load data of each optical communication device in the input network through a combination of multiple decision trees, and determines the target load type that matches the input data from peak load types, valley load types, regular load types, and gradual load types.

[0052] Here, the random forest classifier can effectively improve the accuracy of load type classification and reduce misclassification by integrating the judgment results of multiple decision trees. It can also quickly process input data and determine the target load type in a timely manner.

[0053] In step S320, each predicted flow load data is processed according to a target basic model module matching the target load type among the multiple basic model modules to obtain a corresponding load optimization adjustment instruction.

[0054] Through this embodiment, the predicted traffic load data of each optical communication device in the network is analyzed by a random forest classifier to obtain the corresponding load type, and the matching basic model module is called to output the corresponding load optimization adjustment instruction, thereby ensuring that the load optimization adjustment strategy is adapted to the current network load status or type, and improving the optimization effect of the global load balancing of the network. In addition, as the network traffic changes in real time, the system can quickly identify the load type and adjust the load distribution strategy, thereby improving the dynamic response capability of the network balancing control.

[0055] In some examples of the embodiments of the present application, the construction of a meta-learning model includes consecutive first-stage construction operations and second-stage construction operations, where the first-stage construction operation is used to build a random forest classifier, and the second-stage construction operation is used to update each basic model module.

[0056] Regarding the specific implementation details of the first phase construction operation, in some implementations, the data set is divided into a training set and a test set. The random forest classifier model is trained using the training data set. Specifically, multiple samples are randomly extracted from the training set with replacement, and multiple subsets are generated. For each subset, a decision tree is constructed separately. Specifically, a feature subset is randomly selected, and the features are recursively divided until the stopping condition (such as the maximum depth or the minimum number of samples) is met. Then, all decision trees are combined into a random forest. After the independent training is completed, the random forest classifier is applied to the actual traffic prediction data to identify the corresponding load type (such as peak load, valley load, normal load, gradual load), which can accurately predict the load type corresponding to the future traffic data.

[0057] Regarding the specific implementation details of the second-stage construction operation, the corresponding basic model modules are selected and trained according to the output results of the classifier (load type), so that each basic model module is optimized for a specific load type and dynamic adjustment of the load balancing strategy is achieved.

[0058] In an embodiment of the present application, the meta-learning model is trained and constructed in stages, and after the training and construction of the random forest classifier is completed, the meta-learner and the basic model module are trained. Thus, through the prior training and optimization of the classifier, different load types can be accurately identified, providing an accurate basis for the selection of the basic model module, and the basic model module is dynamically selected according to the output results of the classifier, so that the system can flexibly respond to different load conditions and improve the overall adaptability of the system. In addition, the first stage only needs to focus on the training of the classifier, and the second stage focuses on the optimization of the basic model module, so that the tasks of each stage are clearer and more focused. Through phased construction, the coupling between the classifier and the basic model module is reduced, and the difficulty of implementation and the complexity of training are reduced.

[0059] Figure 4 An operational flowchart of an example of a second-stage construction operation in constructing a meta-learning model according to an embodiment of the present application is shown.

[0060] like Figure 4 As shown, in step S410, based on the shared parameters of the meta-learner, the model parameters of each basic model module are initialized and set.

[0061] Here, by sharing parameters Initialize each basic model module to achieve joint optimization on multiple tasks and provide good initial parameters so that the basic model can quickly adapt to different load types.

[0062] In step S420, for each data subset, multiple gradient descent optimizations are performed on a basic model module with a matching load type using the data subset to update the model parameters of the basic model module.

[0063] More specifically, each base model module Optimize on the corresponding load type data subset and update its model parameters For example, the corresponding basic model modules are trained and optimized through the peak load data subset, the valley load data subset, etc., and the parameters of the basic model modules are updated so that they can produce good performance on the data subsets of the respective tasks.

[0064] Here, by initializing the shared parameters, the basic model module has good adaptability in the initial stage and can quickly adapt to different load types. In addition, the shared parameters are jointly optimized on multiple tasks, which improves the quality of the model's initial parameters and enables the basic model to reach the optimal state in fewer iterations.

[0065] In step S430, the meta-loss for all basic model modules is calculated, and when it is detected that the meta-loss does not satisfy the meta-loss convergence condition, the shared parameters are updated based on the calculated meta-loss, and the model parameters of each basic model module are reset using the updated shared parameters to iteratively update the model parameters of each basic model module.

[0066] In some embodiments, after the training of each basic model module is completed, the model parameters and losses of each basic model module will be passed to the meta-learner update module, and the shared parameters will be updated through the meta-learning algorithm (e.g., model-agnostic meta-learning, Model-Agnostic Meta-Learning, MAML), and the optimized meta-learner parameters will be used for the next round of basic model training, so that the shared parameters Updates are made through the gradient descent of the meta-loss until the meta-loss convergence condition is met, for example, the optimization amplitude of the meta-loss corresponding to two iterations is less than a threshold.

[0067] Here, the calculation of meta-loss and gradient descent optimization ensure the optimal global performance of the model on multiple tasks. In addition, the optimization of each basic model module on its own load type significantly improves the accuracy and effectiveness of the load balancing strategy.

[0068] In some examples of the embodiments of the present application, the losses of all basic model modules can be comprehensively considered by weighted summation to obtain the meta-loss. More specifically, the meta-loss for all basic model modules is calculated by the following formula:

[0069] , formula (1);

[0070] , formula (2);

[0071] , formula (3);

[0072] In the formula, represents the meta-loss for all base model modules, Represents the total number of basic model modules contained in the meta-learning model; Indicates load type, express The corresponding weight, express The loss function of the corresponding basic model module; is The basic model module after training the corresponding data subset has the following parameters: ; express The corresponding load type task loss; express The total number of samples in the corresponding data subset; express The corresponding data subset data samples and corresponding labels; Represents the loss function of a single sample; express The gradient norm of the loss function of the corresponding basic model module, is a normalization factor, which represents the sum of the gradient norms of the loss functions of all basic model modules.

[0073] In the meta-loss function used in the embodiment of the present application, the model performance is optimized from a global perspective by comprehensively considering the losses of all basic model modules. It is dynamically adjusted to ensure that the impact of different tasks is balanced during the optimization process and improve the overall optimization effect. More specifically, the weight It is dynamically adjusted according to the gradient size of each task, so that the model can adaptively focus on important tasks during training and improve the performance of the model under different load types.

[0074] In some examples of the embodiments of the present application, the basic model module adopts a reinforcement learning model. At this time, the loss function of the reinforcement learning model can be a corresponding policy gradient loss function (Policy Gradient Loss), that is, a function used to optimize the parameters of the policy network so that the actions taken can maximize the expected reward, so as to support the iterative calculation of the meta-loss.

[0075] More specifically, the state of the reinforcement learning model is defined by the predicted traffic load data of each optical communication device in the optical communication network and the overall load balancing index, overall network delay and overall bandwidth utilization of the optical communication network; the action of the reinforcement learning model is defined by load optimization adjustment instructions regarding one or more optical communication devices; and the reward of the reinforcement learning model is defined by the degree of optimization of the overall load balancing index, overall network delay and overall bandwidth utilization of the optical communication network after the action is executed.

[0076] Through this embodiment, the load balancing control scheme is integrated into the reinforcement learning model. Through interactive learning with the environment, the load balancing strategy can be adjusted in real time to adapt to the rapidly changing network load conditions and improve the dynamic response capability of the system. Through continuous interactive learning and optimization, the basic model module can show good robustness in the face of different load changes and network conditions. Through the selection and execution of the optimal strategy, the bandwidth utilization rate can be improved, the network delay can be reduced, the resource allocation can be optimized, and the overall network performance can be improved. For different load types, each corresponding basic model module can adaptively learn the optimal strategy to ensure the optimization of system performance under various load conditions. Therefore, through detailed state, action and reward design, the adaptive reinforcement learning model is used as the basic model module, and efficient and stable dynamic load balancing of optical communication devices can be achieved through real-time learning and dynamic optimization.

[0077] In some examples of the embodiments of the present application, the reward function of the reinforcement learning model is:

[0078] , formula (4);

[0079] , formula (5);

[0080] , formula (6);

[0081] , formula (7);

[0082] , formula (8);

[0083] , formula (9);

[0084] , formula (10);

[0085] , formula (11);

[0086] In the formula, Indicates the previous state After the action is executed, the new state is reached the rewards generated; The state before the action is executed The corresponding overall load balancing index, The new state after the action is executed Overall load balancing indicator; The state before the action is executed The overall network latency, The new state after the action is executed Overall network latency The state before the action is executed The overall bandwidth utilization of The new state after the action is executed Overall bandwidth utilization; For the previous state After the action is executed, the new state is reached Dynamic weight coefficient of is the initial weight, is the adjustment factor, They are the overall load balancing index reference value, network delay reference value and bandwidth utilization reference value respectively; represents the total number of optical communication devices in the optical communication network, Status Time The traffic load of an optical communication device, Status Average traffic load at: Status Time Network latency of optical communication devices; Status Time The bandwidth utilization of an optical communication device.

[0087] In the reward function provided in this embodiment, the reward function comprehensively considers the improvement degree of the overall load balancing index, network delay and bandwidth utilization, so that the load balancing strategy can more effectively optimize various network performance indicators and avoid excessive fluctuations in the reward value. The reward function reflects the changes in the system state after the action is executed in real time, guides the reinforcement learning model to learn the optimal strategy, improves the dynamic response capability of the system, enables the system to quickly adapt to load changes, and adjusts the load balancing strategy in time to avoid network congestion and waste of resources. In addition, through the design of each dynamic weight coefficient, the reward function can adaptively adjust the optimization target so that the system can perform well under different load conditions. For example, under high load conditions, the system will pay more attention to the improvement of load balancing indicators and network delay; while under low load conditions, the system will pay more attention to the improvement of bandwidth utilization and optimize the dynamic load balancing capability of the network system.

[0088] In some examples of the embodiments of the present application, the graph neural network adopts a deep graph convolution model, which includes multiple graph convolution layers to update the node features of each graph node in the historical graph structure in the following way:

[0089] , formula (12);

[0090] , formula (13);

[0091] In the formula, Indicates The node feature matrix of the graph convolution layer, represents the activation function, Representing a graph node The set of neighbor graph nodes of ; and Graph nodes and degree; and Respectively The weights and biases of the graph convolutional layers; Indicates Layer Graph Node The node characteristics of Represents the mean aggregation function.

[0092] In the embodiment of the present application, the multi-dimensional data is integrated through the graph convolution layer, which can fully reflect the workload of each optical communication device in the optical communication network, so that each updated node feature not only contains its own load information, but also integrates the features of neighboring nodes. Specifically, through the mean aggregation function, the graph convolution layer can effectively aggregate the information of neighboring nodes, so that the features of each node not only depend on its own data, but also are affected by its neighboring nodes, which can better reflect the correlation between nodes in the network. By considering the load of neighboring nodes, the load balancing strategy is more holistic and global, avoiding the local optimal phenomenon, performing load balancing from a global perspective, and improving the utilization efficiency of network resources.

[0093] In some examples of the embodiments of the present application, the traffic prediction model adopts an LSTM model based on a multi-head attention mechanism. Figure 5 A schematic diagram of the structural connection of an example of a traffic prediction model according to an embodiment of the present application is shown.

[0094] like Figure 5 As shown, the traffic prediction model 500 includes an input layer 510, an LSTM layer 520, a multi-head attention layer 530 and an output layer 540.

[0095] The input layer 510 is used to receive the time series fusion features.

[0096] Here, the amount of information contained in the timing fusion feature is huge, which can reflect the historical traffic characteristics, equipment status characteristics, external environment characteristics and corresponding timing feature change information of the optical communication device, as well as the characteristics of other network-related optical communication devices, and can comprehensively reflect the network status.

[0097] The LSTM layer 520 is used to process the time series fusion features and capture the dependencies between time steps:

[0098] , formula (14);

[0099] In the formula, Represents the fusion features of the corresponding time step in the time series fusion features, is the time step The hidden state of is the time step The hidden state of .

[0100] The LSTM layer can capture dependencies in long time series, ensure the effective extraction and utilization of time series features, and improve the ability to predict future traffic trends.

[0101] The multi-head attention layer 530 is used to capture temporal features from different angles and levels:

[0102] , formula (15);

[0103] , formula (16);

[0104] , formula (17;

[0105] , formula (18);

[0106] , formula (19);

[0107] , formula (20);

[0108] In the formula, The multi-head attention features output by the multi-head attention layer; and Respectively represent and The output features of the attention head are represents the total number of attention heads, represents the feature concatenation operation, Represents the multi-head attention mechanism function; represents the attention mechanism function, is the output weight matrix of multi-head attention; For the The query matrix weights of the attention heads, For the The key matrix weights of the attention heads, For the The value matrix weights of the attention heads; , and denote the query matrix, key matrix and value matrix respectively, Represented as the dimension of the key vector; Represents the softmax activation function.

[0109] Through the multi-head attention mechanism, time series features can be captured from different angles and levels, enhancing the model's expressiveness. As a result, the multi-head attention mechanism enables the model to understand and process time series data from different perspectives. The model can more finely capture features that have an important impact on the prediction results, improving the comprehensiveness of feature extraction and prediction reliability.

[0110] The output layer 540 is used to map the output of the multi-head attention mechanism through the fully connected layer to obtain the final predicted traffic load data:

[0111] ;

[0112] In the formula, is the time step The traffic prediction results are and are the weights and biases of the fully connected layer, Represents the Leaky ReLU activation function.

[0113] It should be noted that, in combination with the characteristics of traffic prediction, in order to realize the processing of non-negative data, the Leaky ReLU activation function is used here. The Leaky ReLU activation function allows a small negative slope, thereby alleviating the problem that the traditional ReLU outputs zero in the negative area, ensuring that the neurons can remain active when negative inputs are input, improving the stability of the model, and effectively alleviating the "dead neuron" problem and improving the stability of the model. In addition, since neurons will not be completely inactivated due to negative inputs, the model can better adapt to different input data and improve the overall robustness. On the other hand, the LeakyReLU activation function introduces nonlinear characteristics, which enables the model to better fit complex data distributions, and the model can more accurately predict complex traffic load conditions and optimize load balancing strategies. As a result, accurate traffic prediction is achieved, allowing the system to quickly respond to changes in network loads, and by dynamically adjusting the load balancing strategy, it ensures that the system maintains efficient operation under different load conditions and improves overall network performance.

[0114] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of actions combined, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application. In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0115] Figure 6 A structural block diagram of an example of a dynamic load balancing system for an optical communication device according to an embodiment of the present application is shown.

[0116] like Figure 6 As shown, the dynamic load balancing system 600 of the optical communication device includes a data acquisition unit 610 , a feature extraction unit 620 , a traffic prediction unit 630 and a load balancing unit 640 .

[0117] The data acquisition unit 610 is used to acquire multi-dimensional time series data of each optical communication device in the optical communication network to be managed; the multi-dimensional time series data includes multiple historical time steps and corresponding multi-dimensional data, and the multi-dimensional data includes traffic load data, equipment status data and external environment data; the traffic load data includes bandwidth utilization, total traffic, upload rate, download rate, packet loss rate and transmission delay.

[0118] The feature extraction unit 620 is used to extract the multi-dimensional data of each optical communication device corresponding to the historical time step in the multi-dimensional time series data, and construct a corresponding historical graph structure based on the extracted multi-dimensional data, and then update the node features of each graph node in the historical graph structure through a graph neural network; the historical graph structure includes multiple graph nodes and edge connections, and the node features of each graph node are defined by the multi-dimensional data of the corresponding optical communication device, and each edge connection is used to connect the graph nodes corresponding to the optical communication devices having a link transmission relationship.

[0119] The traffic prediction unit 630 is used to determine the timing fusion features for each optical communication device based on the node features of the corresponding graph nodes of each updated historical graph structure, and input the timing fusion features into the traffic prediction model to determine the predicted traffic load data corresponding to the preset time period in the future.

[0120] The load balancing unit 640 is used to perform a dynamic load balancing operation based on the predicted traffic load data corresponding to each of the optical communication devices.

[0121] In some embodiments, an embodiment of the present application provides a non-volatile computer-readable storage medium, in which one or more programs including execution instructions are stored, and the execution instructions can be read and executed by an electronic device (including but not limited to a computer, a server, or a network device, etc.) to execute the steps of the dynamic load balancing method of any of the above-mentioned optical communication devices of the present application.

[0122] In some embodiments, the embodiments of the present application also provide a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the steps of any of the above-mentioned dynamic load balancing methods for optical communication devices.

[0123] In some embodiments, the embodiments of the present application also provide an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the dynamic load balancing method of the optical communication device.

[0124] Figure 7 is a schematic diagram of the hardware structure of an electronic device for executing a dynamic load balancing method for an optical communication device provided by another embodiment of the present application, such as Figure 7 As shown, the device includes:

[0125] One or more processors 710 and memory 720, Figure 7 A processor 710 is taken as an example.

[0126] The apparatus for executing the dynamic load balancing method for an optical communication device may further include: an input device 730 and an output device 740 .

[0127] The processor 710, the memory 720, the input device 730 and the output device 740 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0128] The memory 720, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the dynamic load balancing method of the optical communication device in the embodiment of the present application. The processor 710 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 720, that is, the dynamic load balancing method of the optical communication device in the above method embodiment is implemented.

[0129] The memory 720 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 720 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 720 may optionally include a memory remotely arranged relative to the processor 710, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0130] The input device 730 may receive input digital or character information and generate signals related to user settings and function control of the electronic device. The output device 740 may include a display device such as a display screen.

[0131] The one or more modules are stored in the memory 720 , and when executed by the one or more processors 710 , the dynamic load balancing method for an optical communication device in any of the above method embodiments is executed.

[0132] The above-mentioned product can execute the method provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of the present application.

[0133] The electronic devices of the embodiments of the present application exist in various forms, including but not limited to:

[0134] (1) Mobile communication equipment: This type of equipment is characterized by having mobile communication functions and its main purpose is to provide voice and data communications. This type of terminal includes: smart phones, multimedia phones, functional phones, and low-end phones.

[0135] (2) Ultra-mobile personal computer devices: These devices belong to the category of personal computers, have computing and processing functions, and generally also have mobile Internet access features. These terminals include: PDA, MID and UMPC devices, etc.

[0136] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players, handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0137] (4) Other onboard electronic devices with data interaction functions, such as on-board devices installed in vehicles.

[0138] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0139] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A dynamic load balancing method for an optical communication device, comprising: Acquire multi-dimensional time series data of each optical communication device in the optical communication network to be managed; The multi-dimensional time series data includes multiple historical time steps and corresponding multi-dimensional data, and the multi-dimensional data includes traffic load data, device status data and external environment data; the traffic load data includes bandwidth utilization, total traffic, upload rate, download rate, packet loss rate and transmission delay; For each of the historical time steps, extract the multi-dimensional data of each optical communication device corresponding to the historical time step in the multi-dimensional time series data, and construct a corresponding historical graph structure according to each of the extracted multi-dimensional data, and then update the node features of each graph node in the historical graph structure through a graph neural network; the historical graph structure includes a plurality of graph nodes and edge connections, the node features of each of the graph nodes are respectively defined by the multi-dimensional data of the corresponding optical communication device, and each of the edge connections is used to connect the graph nodes corresponding to the optical communication devices having a link transmission relationship; For each optical communication device, determining a time series fusion feature based on the node features of the corresponding graph nodes of each updated historical graph structure, and inputting the time series fusion feature into a traffic prediction model to determine predicted traffic load data corresponding to a preset time period in the future; Performing a dynamic load balancing operation based on the predicted traffic load data corresponding to each of the optical communication devices; The graph neural network adopts a deep graph convolution model, which includes multiple graph convolution layers to update the node features of each graph node in the historical graph structure in the following way: In the formula, Indicates The node feature matrix of the graph convolution layer, represents the activation function, Representing a graph node The set of neighbor graph nodes of ; and Graph nodes and degree; and Respectively The weights and biases of the graph convolutional layers; Indicates Layer Graph Node The node characteristics of represents the mean aggregation function; The traffic prediction model adopts an LSTM model based on a multi-head attention mechanism, which includes an input layer, an LSTM layer, a multi-head attention layer and an output layer; The input layer is used to receive time series fusion features; The LSTM layer is used to process the time series fusion features and capture the dependencies between time steps: In the formula, Indicates the corresponding time step in the time series fusion feature The fusion characteristics of is the time step The hidden state of is the time step The hidden state of The multi-head attention layer is used to capture temporal features from different angles and levels: In the formula, The multi-head attention features output by the multi-head attention layer; and Respectively represent and The output features of the attention head are represents the total number of attention heads, represents the feature concatenation operation, Represents the multi-head attention mechanism function; represents the attention mechanism function, is the output weight matrix of multi-head attention; For the The query matrix weights of the attention heads, For the The key matrix weights of the attention heads, For the The value matrix weights of the attention heads; , and denote the query matrix, key matrix and value matrix respectively, Represented as the dimension of the key vector; represents the softmax activation function; The output layer is used to map the output of the multi-head attention mechanism through the fully connected layer to obtain the final predicted traffic load data: In the formula, is the time step The traffic prediction results are and are the weights and biases of the fully connected layer, Represents the Leaky ReLU activation function.

2. The method according to claim 1, wherein: The performing of a dynamic load balancing operation based on the predicted traffic load data corresponding to each of the optical communication devices comprises: The predicted traffic load data corresponding to each of the optical communication devices is input into a load balancing model to determine a load optimization adjustment instruction for at least one target optical communication device; the load balancing model adopts a meta-learning model, and the load optimization adjustment instruction includes at least one of the following: adjusting traffic routing, enabling spare nodes, dynamically adjusting bandwidth allocation, and adjusting transmission paths; the data sample set of the meta-learning model includes multiple data subsets, each of the data subsets has a corresponding load type; the multiple data subsets include a peak load data subset, a valley load data subset, a normal load data subset, and a gradual load data subset.

3. The method according to claim 2, wherein the meta-learning model comprises a random forest classifier, a meta-learner and a plurality of basic model modules, each of which has a corresponding load type; The step of inputting the predicted traffic load data corresponding to each of the optical communication devices into a load balancing model to determine a load optimization adjustment instruction for at least one target optical communication device includes: Determining, by means of the random forest classifier, a target load type matching each input predicted flow load data; Each of the predicted traffic load data is processed according to a target basic model module that matches the target load type among the multiple basic model modules to obtain a corresponding load optimization adjustment instruction.

4. The method according to claim 3, wherein: The construction of the meta-learning model includes a continuous first-stage construction operation and a second-stage construction operation, wherein the first-stage construction operation is used to construct the random forest classifier, and the second-stage construction operation is used to update each basic model module; The second stage construction operation includes: Based on the shared parameters of the meta-learner, the model parameters of each basic model module are initialized and set; For each of the data subsets, using the data subset to perform multiple gradient descent optimizations on a basic model module with a matching load type, so as to update model parameters of the basic model module; The meta-loss for all base model modules is calculated, and when it is detected that the meta-loss does not satisfy the meta-loss convergence condition, the shared parameters are updated based on the calculated meta-loss, and the model parameters of each base model module are reset using the updated shared parameters to iteratively update the model parameters of each base model module.

5. The method according to claim 4, wherein: The calculation is for the meta-loss of all base model modules, including: In the formula, represents the meta-loss for all base model modules, Represents the total number of basic model modules contained in the meta-learning model; Indicates load type, express The corresponding weight, express The loss function of the corresponding basic model module; is The basic model module after training the corresponding data subset has the following parameters: ; express The corresponding load type task loss; express The total number of samples in the corresponding data subset; express The corresponding data subset data samples and corresponding labels; Represents the loss function of a single sample; express The gradient norm of the loss function of the corresponding basic model module, is a normalization factor, which represents the sum of the gradient norms of the loss functions of all basic model modules.

6. The method according to claim 4, wherein: The basic model module adopts a reinforcement learning model; The state of the reinforcement learning model is defined by the predicted traffic load data of each optical communication device in the optical communication network and the overall load balancing index, overall network delay and overall bandwidth utilization of the optical communication network. The action of the reinforcement learning model is defined by the load optimization adjustment instructions for one or more optical communication devices. The reward of the reinforcement learning model is defined by the degree of optimization of the overall load balancing index, overall network delay and overall bandwidth utilization of the optical communication network after the action is executed.

7. The method according to claim 6, wherein: The reward function of the reinforcement learning model is: In the formula, Indicates the previous state After the action is executed, the new state is reached the rewards generated; The state before the action is executed The corresponding overall load balancing index, The new state after the action is executed Overall load balancing indicator; The state before the action is executed The overall network latency, The new state after the action is executed Overall network latency The state before the action is executed The overall bandwidth utilization of The new state after the action is executed Overall bandwidth utilization; For the previous state After the action is executed, the new state is reached Dynamic weight coefficient of is the initial weight, is the adjustment factor, They are the overall load balancing index reference value, network delay reference value and bandwidth utilization reference value respectively; represents the total number of optical communication devices in the optical communication network, Status Time The traffic load of an optical communication device, Status Average traffic load at: Status Time Network latency of optical communication devices; Status Time The bandwidth utilization of an optical communication device.

8. A dynamic load balancing system for an optical communication device, comprising: A data acquisition unit, used to acquire multi-dimensional time series data of each optical communication device in the optical communication network to be managed; The multi-dimensional time series data includes multiple historical time steps and corresponding multi-dimensional data, and the multi-dimensional data includes traffic load data, device status data and external environment data; the traffic load data includes bandwidth utilization, total traffic, upload rate, download rate, packet loss rate and transmission delay; A feature extraction unit is used to extract, for each of the historical time steps, the multi-dimensional data of each optical communication device corresponding to the historical time step in the multi-dimensional time series data, and construct a corresponding historical graph structure according to each of the extracted multi-dimensional data, and then update the node features of each graph node in the historical graph structure through a graph neural network; the historical graph structure includes a plurality of graph nodes and edge connections, the node features of each of the graph nodes are respectively defined by the multi-dimensional data of the corresponding optical communication device, and each of the edge connections is used to connect the graph nodes corresponding to the optical communication devices having a link transmission relationship; A traffic prediction unit, for determining, for each optical communication device, a time series fusion feature based on the node features of the corresponding graph nodes of each updated historical graph structure, and inputting the time series fusion feature into a traffic prediction model to determine predicted traffic load data corresponding to a preset time period in the future; A load balancing unit, configured to perform a dynamic load balancing operation based on the predicted traffic load data corresponding to each of the optical communication devices; The graph neural network adopts a deep graph convolution model, which includes multiple graph convolution layers to update the node features of each graph node in the historical graph structure in the following way: In the formula, Indicates The node feature matrix of the graph convolution layer, represents the activation function, Representing a graph node The set of neighbor graph nodes of ; and Graph nodes and degree; and Respectively The weights and biases of the graph convolutional layers; Indicates Layer Graph Node The node characteristics of represents the mean aggregation function; The traffic prediction model adopts an LSTM model based on a multi-head attention mechanism, which includes an input layer, an LSTM layer, a multi-head attention layer and an output layer; The input layer is used to receive time series fusion features; The LSTM layer is used to process the time series fusion features and capture the dependencies between time steps: In the formula, Indicates the corresponding time step in the time series fusion feature The fusion characteristics of is the time step The hidden state of is the time step The hidden state of The multi-head attention layer is used to capture temporal features from different angles and levels: In the formula, The multi-head attention features output by the multi-head attention layer; and Respectively represent and The output features of the attention head are represents the total number of attention heads, represents the feature concatenation operation, Represents the multi-head attention mechanism function; represents the attention mechanism function, is the output weight matrix of multi-head attention; For the The query matrix weights of the attention heads, For the The key matrix weights of the attention heads, For the The value matrix weights of the attention heads; , and denote the query matrix, key matrix and value matrix respectively, Represented as the dimension of the key vector; represents the softmax activation function; The output layer is used to map the output of the multi-head attention mechanism through the fully connected layer to obtain the final predicted traffic load data: In the formula, is the time step The traffic prediction results are and are the weights and biases of the fully connected layer, Represents the Leaky ReLU activation function.

Images