Network resource calling optimization method and device

Abstract

The invention discloses a network resource calling optimization method and device. The method comprises the following steps: acquiring an atlas change event set and network service demand information of a target network in a preset time period; updating the network time sequence atlas of the target network according to the atlas change event set; judging whether the target network is abnormal or not according to the updated network time sequence graph; and under the condition that the target network is abnormal, analyzing the updated network time sequence atlas and the network service demand information by using a pre-trained strategy decision model to obtain a corresponding target network resource scheduling strategy, and optimizing the network resource scheduling of the target network according to the target network resource scheduling strategy. According to the method and the device, the technical problem that the real-time service requirement of a user is difficult to meet due to poor accuracy and timeliness of a scheduling result when network resource scheduling is carried out based on a regularly updated network static graph in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the technical field of network management, and in particular, to a method and device for optimizing network resource invocation. Background Art

[0002] In modern target networks, especially against the backdrop of 5G, Internet of Things (IoT), and cloud-network convergence, the dynamics and complexity of network resources have increased significantly, posing new challenges to network management and optimization. Among them, network resources include, but are not limited to, data communication devices, mobile communication devices, transmission devices, data center facilities, and virtual resources, etc. These pieces of information form a network topology through complex connection relationships.

[0003] Traditional network resource scheduling relies on a network map updated regularly. Such scheduling decisions based on a static map cannot reflect the real-time status of current network resources, and it is difficult to comprehensively consider the complexity and dynamics of the network, thus affecting the accuracy and efficiency of resource scheduling. At the same time, static network map analysis is difficult to predict network failures in advance, and it is impossible to take preventive measures or make a quick response, which causes huge losses in terms of business continuity and user experience.

[0004] In view of the above problems, no effective solution has been proposed yet. Summary of the Invention

[0005] Embodiments of the present application provide a method and device for optimizing network resource invocation, so as to at least solve the technical problem that when network resource scheduling is performed based on a periodically updated network static map in related technologies, the accuracy and timeliness of the scheduling result are poor, resulting in difficulty in meeting the real-time service requirements of users.

[0006] According to one aspect of the embodiments of the present application, a method for optimizing network resource invocation is provided, including: obtaining a set of graph change events and network service requirement information of a target network within a preset time period; updating the network time-series graph of the target network according to the set of graph change events, where multiple nodes in the network time-series graph are respectively used to store attribute information of a resource entity in the target network, multiple directed edges in the network time-series graph are respectively used to represent connection relationships between multiple nodes, and the attribute information includes: static attribute information for uniquely identifying the resource entity, and dynamic attribute information for representing the resource status time-series information of the resource entity; judging whether the target network has an abnormality according to the updated network time-series graph; in the case where the target network has an abnormality, analyzing the updated network time-series graph and network service requirement information by using a pre-trained policy decision model to obtain a corresponding target network resource scheduling policy, and optimizing the network resource scheduling of the target network according to the target network resource scheduling policy.

[0007] Optionally, the process of constructing a network timing graph includes: obtaining multiple resource entities within a target network, where the resource entities include at least one of the following: physical devices, virtual resources; determining the attribute information corresponding to each resource entity, and determining the resource scheduling relationship and resource transmission quality between every two resource entities; using each resource entity as a node, using the resource scheduling relationship between every two resource entities as a directed edge, and using the resource transmission quality between every two resource entities as the weight of the edge to construct a network timing graph.

[0008] Optionally, updating the network timing graph of the target network according to the graph change event set includes: traversing each graph change event in the graph change event set, parsing the graph change event, and determining the event type and event content corresponding to the graph change event, where the event type includes at least one of the following: addition, update, deletion; performing an operation corresponding to the event type on the network timing graph according to the event content to obtain an updated network timing graph.

[0009] Optionally, determining whether there is a network anomaly in the target network according to the updated network timing graph includes: analyzing the updated network timing graph using a preset anomaly detection model to determine the anomaly probability of each node, where the anomaly detection model consists of at least a graph attention network layer and a graph isomorphism network layer, and the graph isomorphism network layer includes a multi-layer perceptron; determining that there is an anomaly in the target network when the anomaly probability of any node in the updated network timing graph is higher than a preset probability threshold; determining that there is no anomaly in the target network when the anomaly probabilities of all nodes in the updated network timing graph are not higher than the preset probability threshold.

[0010] Optionally, analyzing the updated network timing graph using a preset anomaly detection model to determine the anomaly probability of each node includes: for each node in the updated network timing graph, determining a corresponding feature vector according to the attribute information of the current node extracted from the updated network timing graph, and constructing an adjacency matrix of the current node according to all the directed edges with the current node as an endpoint, where the adjacency matrix is used to record the feature vectors of all neighbor nodes of the node; analyzing the feature vector using the graph attention network layer to obtain an attention weight vector of the node, and updating the feature vector with the attention weight vector and the adjacency matrix, where the attention weight vector includes the attention weights between the node and each neighbor node; for each layer of perceptron in the graph isomorphism network layer, scaling the updated feature vector output by the previous layer of perceptron using a preset parameter, and superimposing the updated feature vectors of all neighbor nodes of the current node to obtain an updated feature vector; analyzing the updated feature vector output by the last layer of perceptron in the multi-layer perceptron using an activation function layer to obtain the anomaly probability of the node.

[0011] Optionally, the training process of the policy decision model includes: constructing an online Q-network and a policy network, and initializing the network weight parameters of the online Q-network and the policy network, where the online Q-network is used to solve the network resource scheduling policy, and the policy network includes multiple network resource scheduling policies; using the network weight parameters of the initialized online Q-network as the network parameters of the target Q-network; setting an experience pool and determining the capacity of the experience pool; determining a first preset number of iteration cycles, and iteratively solving the network resource scheduling policy and the network weight parameters through the following steps: in each iteration cycle, initializing the network state and determining a second preset number of calculation cycles, where the network state includes: the updated network timing map and network service demand information; in each calculation cycle, inputting the current network state into the policy network to obtain a policy probability distribution, sampling a network resource scheduling policy according to the policy probability distribution, and inputting the policy adjustment and the current network state into the online Q-network to calculate the predicted Q-value corresponding to the network resource scheduling policy; executing the network resource scheduling policy and calculating the reward, obtaining a new network state, and using the current network state, the network resource scheduling policy, the reward, and the new network state as a sample, storing the sample in the experience pool; inputting multiple samples randomly sampled from the experience pool into a neural network containing a bidirectional long short-term memory network, calculating the probability of each sample being sampled, the mean square error loss function, and the loss function weight, determining the target Q-value according to the calculation results, and updating the network weight parameters of the online Q-network according to the target Q-value and the predicted Q-value; updating the network weight parameters of the target Q-network according to the network weight parameters of the online Q-network after a third preset number of calculation cycles, where the third preset number is less than the second preset number; after the iteration is completed, using the obtained target Q-network as the policy decision model.

[0012] Optionally, the target network resource scheduling policy includes at least one of the following: a path network resource scheduling policy, a virtual machine migration policy, and a bandwidth expansion policy.

[0013] According to another aspect of the embodiments of the present application, there is also provided a network resource call optimization device, including: an acquisition module, configured to acquire a set of graph change events and network service demand information of a target network within a preset time period; an update module, configured to update the network timing graph of the target network according to the set of graph change events, wherein multiple nodes in the network timing graph are respectively used to store attribute information of a resource entity in the target network, and multiple directed edges in the network timing graph are respectively used to represent connection relationships between multiple nodes, and the attribute information includes: static attribute information for uniquely identifying the resource entity, and dynamic attribute information for representing the resource status timing information of the resource entity; a judgment module, configured to judge whether the target network has an abnormality according to the updated network timing graph; an optimization module, configured to, when the target network has an abnormality, analyze the updated network timing graph and network service demand information by using a pre-trained policy decision model to obtain a corresponding target network resource scheduling policy, and optimize the network resource scheduling of the target network according to the target network resource scheduling policy.

[0014] According to another aspect of the embodiments of the present application, there is also provided a computer program product, which includes: a computer program, wherein when the computer program is executed by a processor, the above-mentioned network resource call optimization method is implemented.

[0015] According to another aspect of the embodiments of the present application, there is also provided an electronic device, which includes: a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to execute the above-mentioned network resource call optimization method through the computer program.

[0016] In the embodiments of the present application, the system adopts a real-time event-driven network timing graph update mechanism and a dynamic resource optimization algorithm based on a graph neural network, realizes efficient management of the resource topology of the target network and accurate and rapid detection of abnormal states, and further automatically generates an optimal resource scheduling policy through a reinforcement learning algorithm, significantly improving the response speed of network faults and the accuracy of resource scheduling. Therefore, the embodiments of the present application can effectively solve the problems that traditional relational databases are difficult to cope with complex network associations, static graph modeling cannot reflect resource changes in a timely manner, and threshold-based fault diagnosis methods are prone to false alarms in complex topology environments, achieving the purpose of improving network performance, enhancing network stability, and optimizing resource utilization efficiency, and further solving the technical problem that when network resource scheduling is performed based on a periodically updated network static graph in related technologies, the accuracy and timeliness of the scheduling result are poor, resulting in difficulty in meeting the real-time service requirements of users. Description of the Drawings

[0017] The accompanying drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0018] Figure 1 is a schematic flowchart of an optional method for optimizing network resource invocation according to an embodiment of the present application;

[0019] Figure 2 is a schematic structural diagram of an optional device for optimizing network resource invocation according to an embodiment of the present application;

[0020] Figure 3 is a schematic structural diagram of another optional device for optimizing network resource invocation according to an embodiment of the present application;

[0021] Figure 4 is a schematic structural diagram of an optional electronic device according to an embodiment of the present application. Detailed implementation manners

[0022] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0023] It should be noted that the terms "first", "second", etc. in the specification, claims and drawings of the present application are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those clearly listed steps or units, but may include other steps or units not clearly listed or inherent to these process, method, product or device.

[0024] To better understand the embodiments of the present application, some nouns or terms that appear in the description process of the embodiments of the present application are translated and explained as follows:

[0025] SNMP (Simple Network Management Protocol) Trap: It is an asynchronous notification mechanism. When a network device (such as a router, switch, etc.) has a specific event (such as a port status change, device failure, etc.), it will generate a Trap message and send it to the configured management station. This mechanism allows the device to actively report important events to the management system without the management system constantly querying the device status.

[0026] Netconf (Network Configuration) protocol: It is a network management protocol mainly used for configuring network devices and querying device status. Therefore, Netconf event notification provides a standardized method to achieve real-time monitoring of device status, and usually uses the XML format to describe data, which has better scalability and readability.

[0027] Embodiment 1

[0028] According to the embodiments of the present application, a method for optimizing network resource invocation is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0029] Figure 1 It is a schematic flowchart of a method for optimizing network resource invocation provided according to the embodiments of the present application. As Figure 1 shown, the method includes the following steps:

[0030] Step S102, obtain the set of device change events and network service demand information of the target network within a preset time period.

[0031] In the technical solution provided in the above step S102, the above target network includes but is not limited to: IP network, access network, core network, or cloud resource network, etc. Therefore, the set of device change events covers the status change information of all resource entities in the network, including but not limited to events such as device addition, link update, and node deletion. And the network service demand information is a matrix containing parameters such as service ID, source node, target node, and required bandwidth, which reflects the specific demands of various services in the target network for network resources.

[0032] Step S104, update the network timing graph of the target network according to the network resource topology information.

[0033] In the technical solution provided in step S104 above, the above network timing map is a graph data structure used to store and characterize the state of target network resources and their changes over time. It contains multiple nodes and multiple directed edges. Among them, each node is used to store the attribute information of a resource entity in the target network, including: static attribute information for uniquely identifying the resource entity, and dynamic attribute information for characterizing the resource state timing information of the resource entity; and the edges are used to characterize the connection relationships between multiple nodes.

[0034] Step S106: According to the updated network timing map, determine whether there is an abnormality in the target network.

[0035] In the technical solution provided in step S106 above, the above updated network timing map refers to a graph data structure that has been updated by a set of graph change events and can reflect the real-time state of the target network. Therefore, by only analyzing the updated network timing map, abnormal topologies within the map (i.e., resource entities or connection relationships that do not conform to the normal operating state) can be identified, thereby determining whether there is an abnormality in the target network within a preset time period.

[0036] Step S108: When there is an abnormality in the target network, use the pre-trained policy decision model to analyze the updated network timing map and network service demand information to obtain the target network resource scheduling policy for the target network, and optimize the network resource scheduling of the target network according to the target network resource scheduling policy.

[0037] In the technical solution provided in step S108 above, the above policy decision model is an intelligent model based on machine learning. It adopts a reinforcement learning algorithm and can generate effective network resource scheduling policies for the optimization of the network resource topology while considering network service demands. Therefore, when there is an abnormality in the target network, the system can use the updated network timing map and network service demand information as inputs and send them into the pre-trained policy decision model for in-depth analysis. The policy decision model can, based on the current network state and service demands, determine the optimal target network resource scheduling policy by evaluating the costs and benefits of different resource adjustment actions. Finally, the system executes the target network resource scheduling policy to optimize the network resource scheduling of the target network, thereby achieving the purpose of improving network resource utilization efficiency and ensuring service quality.

[0038] Based on the solution defined in the above steps S102 to S108, it can be known that in the embodiment of the present application, the system adopts a real-time event-driven network timing graph update mechanism and a dynamic resource optimization algorithm based on a graph neural network, realizing efficient management of the resource topology of the target network and accurate and rapid detection of abnormal states. Further, an optimal resource scheduling strategy is automatically generated through a reinforcement learning algorithm, significantly improving the response speed of network faults and the accuracy of resource scheduling. Therefore, the embodiment of the present application can effectively solve the problems that traditional relational databases are difficult to handle complex network associations, static graph modeling cannot reflect resource changes in a timely manner, and threshold-based fault diagnosis methods are prone to false alarms in complex topology environments, achieving the purpose of improving network performance, enhancing network stability, and optimizing resource utilization efficiency.

[0039] The following explains each step of the network resource call optimization method in combination with a specific implementation process.

[0040] As an optional implementation manner, in the technical solution provided in the above step S102, the resource collection layer in the system can monitor graph change events in the target network. For example, the network element acquisition and control module is used to monitor SNMP Trap or Netconf events of physical devices, and capture changes in the operating status of physical devices in real time, such as device online, device offline, link status change, etc.; the virtualization collector listens to lifecycle events of virtual resources (such as VMs, containers), such as virtual resource migration. At the same time, the resource collection layer in the system can also monitor the network service demand information of the target network. For example, the service awareness module parses service opening, adjustment, or cancellation requirements from BMO domain order data.

[0041] Next, the system formats the captured graph change event set and service demand information into a unified event stream, and efficiently transmits it to the dynamic graph engine layer in the system through the Kafka cluster. Among them, Kafka, as an event middleware, ensures the reliable transmission and real-time processing of events.

[0042] Then, after receiving the event stream, the dynamic graph engine layer in the system can parse the event stream and update the network timing graph of the target network according to the obtained graph change event set.

[0043] Among them, the construction method of the network timing graph of the target network is as follows:

[0044] The first step: Obtain multiple resource entities in the target network, where the resource entities include but are not limited to: physical devices (such as routers, switches), virtual resources (such as virtual machines, containers), etc.

[0045] Step 2: Determine the attribute information corresponding to each resource entity, and determine the resource scheduling relationship and resource transmission quality between every two resource entities. Among them, the above-mentioned attribute information includes: static attribute information (such as device type, geographical location, fixed capacity index, etc.) and dynamic attribute information (such as CPU utilization rate, memory usage, bandwidth consumption, etc.). In addition, the above-mentioned resource scheduling relationship reflects whether there is a need or possibility of resource scheduling between every two resource entities, and the resource transmission quality between every two resource entities reflects the real-time transmission status, which can be obtained by calculating transmission quality indicators (such as latency, packet loss rate, bandwidth utilization rate).

[0046] Step 3: Use each resource entity as a node, use the resource scheduling relationship between every two resource entities as a directed edge, and use the resource transmission quality between any two resource entities as the weight of the edge to construct a network time-series graph.

[0047] Therefore, each resource entity in the target network exists in the form of a node, carrying its static and dynamic attribute information, and the scheduling relationship between entities is represented by directed edges, and the weight of the edge is dynamically updated according to the resource transmission quality. The network time-series graph formed in this way can comprehensively and real-time reflect the resource connection status and performance of the target network.

[0048] Furthermore, the system can traverse each graph change event in the graph change event set, parse the graph change event, and determine the event type and event content corresponding to the graph change event. Among them, the event type includes at least one of the following: addition, update, deletion; perform an operation corresponding to the event type on the network time-series graph according to the event content to obtain an updated network time-series graph.

[0049] That is to say, for each graph change event, the system can generate a corresponding graph operation instruction. For example, if the graph change event is a device online event, the system will add a new node to the network time-series graph and assign the corresponding static attribute information (such as device type, location) and dynamic attribute information (such as current CPU utilization rate, memory occupancy, bandwidth consumption, etc.) to this node; if the graph change event is a link state change event, the system will update the weight of the relevant edge in the network time-series graph to reflect the latest resource transmission quality; if the graph change event is a device offline event, the system will trigger a node deletion operation to delete the relevant node in the network time-series graph. Thus, the network time-series graph of the target network is updated in real time to keep the graph consistent with the actual network state, providing the most accurate data basis for subsequent network resource management and optimization.

[0050] Therefore, updating the network time-series graph through the above incremental graph construction mechanism means that each time a change event is received, it only updates the relevant part of the graph instead of reconstructing the entire graph. This mechanism significantly reduces resource consumption and improves the update efficiency. At the same time, the Kafka event-driven architecture ensures that events can be processed quickly and reliably, guaranteeing the real-time nature of the graph even under a large number of concurrent events.

[0051] In addition, the updated network time-series graph can be persistently stored and a version control mechanism is adopted to record each change. This can not only save the latest state of the network but also track the change process of resources through historical versions, providing a basis for fault diagnosis and rollback operations.

[0052] Furthermore, in the technical solution provided in the above step S106, the system can determine whether there is a network anomaly in the target network according to the following method, including:

[0053] Step S1061, analyze the updated network time-series graph using a preset anomaly detection model to determine the anomaly probability of each node.

[0054] Step S1062, when the anomaly probability of any node in the updated network time-series graph is higher than the preset probability threshold, determine that there is an anomaly in the target network;

[0055] Step S1063, when the anomaly probabilities of all nodes in the updated network time-series graph are not higher than the preset probability threshold, determine that there is no anomaly in the target network.

[0056] Optionally, in the technical solution provided in the above step S1061, the anomaly detection model is at least composed of a Graph Attention (GAT) layer and a Graph Isomorphism Network (GIN) layer. Among them, the GAT layer assigns different weights to different neighbors of each node by learning the attention mechanism, and these weights reflect the influence degree of neighbor nodes on the central node; the GIN layer updates the feature representation of the node by aggregating the neighbor information of the node and applying a multi-layer perceptron (MLP). Therefore, the anomaly detection model can determine the anomaly probability of each node in the network time-series graph according to the following steps, including:

[0057] The first step: For each node in the updated network time-series graph, determine the corresponding feature vector based on the attribute information of the current node extracted from the updated network time-series graph, and construct the adjacency matrix of the current node based on all the directed edges with the current node as the endpoint, where the adjacency matrix is used to record the feature vectors of all neighbor nodes of the node.

[0058] Step 2: Use the graph attention network layer to analyze the feature vectors, obtain the attention weight vector of the nodes, and update the feature vectors with the attention weight vector and the adjacency matrix. Among them, the attention weight vector includes the attention weights between the nodes and their respective neighbor nodes.

[0059] Step 3: For each perceptron in the graph isomorphism network layer, use the preset parameters to scale the updated feature vectors output by the previous perceptron, and superimpose the updated feature vectors of all neighbor nodes of the current node to obtain the updated feature vectors.

[0060] Step 4: Use the activation function layer to analyze the updated feature vectors output by the last perceptron in the multi-layer perceptron to obtain the anomaly probability of the nodes.

[0061] Therefore, if the neighbor matrix of node i in the updated network time series graph is denoted as A i ∈R n×n , and the node features (such as device type, geographical location, resource utilization rate, etc.) are denoted as X i ∈R n×d , where n represents the number of nodes in the subgraph where node i is located, and d represents the number of feature dimensions of node i. Then, the anomaly detection model can calculate the anomaly probability of this node according to the following steps:

[0062] Step 1: First, calculate the attention coefficient e between node i and its neighbor node j according to the following formula ij :

[0063] e ij = LeakyReLU(a T [Wh i ||Wh j )

[0064] In the formula, h i represents the feature vector obtained by extracting the features of the node feature X i of node i, h j represents the feature vector obtained by extracting the features of the node feature X j of neighbor node j, W represents the weight matrix of the graph attention network, and a represents the attention vector.

[0065] Step 2: Use the softmax function to normalize the attention coefficients e ij of all neighbor nodes of node i to obtain the attention weight α ij :

[0066]

[0067] Wherein, N(i) represents the set of neighbor nodes of node i.

[0068] Step 3: Use the attention weight α ij Aggregate the features of all neighbor nodes of node i to update the feature vector of node i:

[0069] h i′ = σ(∑ j∈N(i) α ij Wh j )

[0070] Step 4: Scale the updated feature vector of node i using the learnable parameters set by the multi-layer perceptron, specifically as follows:

[0071]

[0072] Wherein, p represents the number of layers of the perceptron, represents the feature vector of node i after scaling processing by the (p - 1)-th layer perceptron, represents the feature vector of the adjacent node i after scaling processing by the (p - 1)-th layer perceptron, represents the feature vector of node i after scaling processing by the p-th layer perceptron.

[0073] Step 5: Analyze the updated feature vector output by the last layer perceptron in the multi-layer perceptron using the following activation function to obtain the anomaly probability of the node:

[0074]

[0075] Wherein, W o represents the weight matrix of the Sigmoid function, p i represents the anomaly probability of node i.

[0076] Therefore, through the method provided in the above Step 1 to Step 5, the anomaly probability of each node in the network resource graph can be obtained.

[0077] It should be noted that the loss function of the above anomaly detection model can adopt Focal Loss. Therefore, the loss function of the anomaly detection model can be written as:

[0078] FocalLoss(p, y) = -α(1 - p) γ y log(p)

[0079] Among them, γ represents an adjustable factor, p represents the degree of proximity to class y, and the larger p is, the closer it is to class y, that is, the more accurate the classification is.

[0080] As an alternative implementation, the system can train the policy decision model according to the following steps, including:

[0081] Construct an online Q-network and a policy network, and initialize the network weight parameters of the online Q-network and the policy network. Among them, the online Q-network is used to solve the network resource scheduling policy, and the policy network includes multiple network resource scheduling policies, such as path network resource scheduling policy, virtual machine migration policy, and bandwidth expansion policy.

[0082] Take the network weight parameters of the initialized online Q-network as the network parameters of the target Q-network.

[0083] Set up an experience pool and determine the capacity of the experience pool.

[0084] Determine the first preset number of iteration cycles, and iteratively solve the network resource scheduling policy and network weight parameters through the following steps:

[0085] In each iteration cycle, initialize the network state and determine the second preset number of calculation cycles. Among them, the network state includes: network timing graph and network service demand information, and the network service demand information can be in matrix form, and each row in the network service demand matrix includes at least: service ID, source node, target node, and required bandwidth;

[0086] In each calculation cycle, input the current network state into the policy network to obtain the policy probability distribution, sample a network resource scheduling policy according to the policy probability distribution, input the policy adjustment and the current network state into the online Q-network, and calculate the predicted Q-value corresponding to the network resource scheduling policy; execute the network resource scheduling policy and calculate the reward, obtain the new network state, and use the current network state, network resource scheduling policy, reward, and new network state as a sample, and store the sample in the experience pool; input multiple samples randomly sampled from the experience pool into a neural network containing a bidirectional long short-term memory network, calculate the probability of each sample being sampled, the mean square error loss function, and the loss function weight, determine the target Q-value according to the calculation results, and update the network weight parameters of the online Q-network according to the target Q-value and the predicted Q-value;

[0087] After the third preset number of calculation cycles, update the network weight parameters of the target Q-network according to the network weight parameters of the online Q-network, where the third preset number is less than the second preset number;

[0088] After the iteration is completed, use the obtained target Q-network as the policy decision model.

[0089] In the above training process, the relevant function for calculating the reward corresponding to the network resource scheduling policy is as follows:

[0090] r t= w1·SLA satisfaction rate - w2·resource overhead - w3·handover cost. In the formula, the SLA (Service Level Agreement) satisfaction rate represents the degree to which network services meet the SLA within a preset time period. The SLA stipulates the service quality standards provided by the network, such as packet loss rate, latency, bandwidth, etc. Therefore, the higher the SLA satisfaction rate, the better the network can support business requirements and the better the user experience. Thus, it is considered positively in the reward function, that is, increasing the SLA satisfaction rate will increase the reward value; resource overhead represents the usage cost of network resources (such as computing resources, network bandwidth, storage, etc.). Therefore, resource overhead is regarded as a negative factor in the reward function, and reducing resource overhead will increase the overall reward, encouraging the scheduling strategy to optimize resource usage efficiency while meeting service quality; handover cost represents the additional latency, bandwidth consumption, and potential wear on hardware caused by frequent resource handovers (such as path rerouting, virtual machine migration, etc.). Therefore, the handover cost aims to measure the negative impacts brought by these operations. Reducing the handover frequency and cost can also improve network efficiency and user experience, so it is also considered as a negative factor in the reward function; w1, w2, w3 represent the weight coefficients in the reward function, which are used to adjust the relative importance of different objectives in the total reward. Usually, w1 is set relatively large because the satisfaction rate of the SLA (Service Level Agreement) is directly related to user experience and business success, while w2 and w3 are set according to the resource cost sensitivity and tolerance for handover frequency.

[0091] Meanwhile, GAE (Generalized Advantage Estimation) is used to calculate the advantage estimation corresponding to the execution of the network resource scheduling strategy:

[0092]

[0093] In the formula, γ represents the discount factor, which is used to control the importance of future rewards at the current time point t. Its value range is between 0 and 1. When γ is closer to 1, it means the algorithm attaches more importance to long-term rewards. On the contrary, when it is closer to 0, it means the algorithm attaches more importance to short-term rewards; λ represents a hyperparameter, which is used to control the balance between bias and variance. Its value range is also between 0 and 1. When λ = 0, GAE degenerates into the traditional single-step Temporal Difference (TD) method. On the contrary, when λ = 1, GAE approaches the complete Monte Carlo estimation method; δ t represents the temporal difference error, and its calculation formula is: δ t = r t + γV(s t+1 ) - V(s t ), where, r tdenotes the reward obtained by the network resource scheduling policy at time step t, and V(s t ) represents the estimate of the state value function.

[0094] Therefore, in a policy optimization algorithm (such as the Proximal Policy Optimization, PPO algorithm), the advantage estimate calculated by the above GAE can be used to update the policy space. Generally, the update objective of the policy space is to maximize the following objective function:

[0095]

[0096] where r t (θ) represents the probability ratio of the current network resource scheduling policy to the old network resource scheduling policy.

[0097] By performing the advantage estimate corresponding to the network resource scheduling policy through the above GAE calculation, more stable and efficient policy updates can be achieved.

[0098] Optionally, the above experience pool can be a SumTree structure, and the samples in the experience pool are the leaf nodes in the SumTree. When storing a sample in the experience pool, it can first be determined whether the capacity of the experience pool is full; if the capacity of the experience pool is not full, the sample is directly stored in the experience pool; if the capacity of the experience pool is full, based on the first-in, first-out principle, the new sample is used to replace the oldest stored old sample in the experience pool.

[0099] It should be noted that during the training process of the above policy decision-making model, it can be divided into the following three training stages based on the Curriculum Learning strategy in machine learning, aiming to improve the learning efficiency and final performance of the model by gradually transitioning from simple tasks to complex tasks. Among them:

[0100] In the first stage (such as before 1k steps), a relatively small 10-node topology is adopted, and only the path network resource scheduling policy is allowed within this stage, thus providing a relatively simple learning starting point for the model;

[0101] As the model gradually matures, in the second stage (such as before 5k steps), a 100-node topology is adopted, and the virtual machine migration policy is opened within this stage, increasing the dimension of network management and resource scheduling;

[0102] Finally, in the third stage (such as before 10k steps), a larger-scale 1k-node topology is adopted, and all network resource scheduling policies are enabled at this time, thus coordinating a wider range of resource types and more complex network structures.

[0103] Through staged training, allowing the model to focus on learning basic rules and patterns in the early stage can prevent the model from falling into local optimal solutions, or the learned policies being too general to handle specific complex situations.

[0104] Finally, the system can call the policy decision model obtained from the above training to analyze the updated network timing graph and network service demand information, obtain the corresponding target network resource scheduling policy, and optimize the network resource scheduling of the target network according to the target network resource scheduling policy.

[0105] Specifically, the target network resource scheduling policy includes but is not limited to the following several policies:

[0106] (1) Path network resource scheduling policy (including parameters related to the new path node sequence): Call the SDN controller to issue OpenFlow flow tables;

[0107] (2) Virtual machine migration policy (including parameters such as source host ID and target host ID): Trigger Live Migration through the Kubernetes API;

[0108] (3) Bandwidth expansion policy, etc. (including related IDs such as link ID and new bandwidth value): Send Cross-Connect configurations to the transmission network management system.

[0109] In addition, the embodiments of the present application can also present the results of anomaly detection on the target network and the network resource scheduling policy for resource topology optimization of the target network in the form of a decision tree, which is convenient for operation and maintenance personnel to understand and execute. This not only improves the transparency of decision-making, but also reduces the professional skill requirements, enabling non-expert personnel to quickly understand and operate complex resource scheduling decisions, and improving the operation and maintenance efficiency and decision-making quality.

[0110] In the network resource topology optimization method introduced in the embodiments of the present application, an anomaly detection model based on a graph neural network (GNN) is introduced, which can more accurately identify and locate topological anomalies based on the deep features of the network topology structure, especially the accurate positioning ability of the fault propagation path in a complex topology environment; at the same time, a policy decision model based on a reinforcement learning algorithm is also introduced, which can generate optimal resource scheduling policies in real time under multi-objective constraints such as meeting service requirements, quality of service (QoS) requirements, and cost control. In addition, the embodiments of the present application adopt a graph computing accelerator with GPU parallelization technology, which greatly improves the processing speed of graph computing tasks and can support real-time optimization decisions for large-scale network resources. Under high-concurrency requests, compared with traditional solutions, the response time and CPU occupancy rate of the embodiments of the present application have been significantly improved.

[0111] Embodiment 2

[0112] According to the embodiments of the present application, there is also provided a network resource call optimization device for implementing the network resource call optimization method in Embodiment 1, as Figure 2As shown in the figure, the network resource call optimization device at least includes an acquisition module 22, an update module 24, a judgment module 26, and an optimization module 28, where:

[0113] The acquisition module 22 is used to acquire a set of graph change events and network service requirement information of the target network within a preset time period;

[0114] The update module 24 is used to update the network timing graph of the target network according to the set of graph change events. Among them, multiple nodes in the network timing graph are respectively used to store the attribute information of a resource entity in the target network, and multiple directed edges in the network timing graph are respectively used to represent the connection relationships between multiple nodes, and the attribute information includes: static attribute information for uniquely identifying the resource entity, and dynamic attribute information for representing the resource state timing information of the resource entity;

[0115] The judgment module 26 is used to judge whether the target network has abnormalities according to the updated network timing graph;

[0116] The optimization module 28 is used to, when the target network has abnormalities, analyze the updated network timing graph and network service requirement information by using a pre-trained policy decision model, obtain a corresponding target network resource scheduling policy, and optimize the network resource scheduling of the target network according to the target network resource scheduling policy.

[0117] The following will be based on Figure 3 the more detailed device structure diagram shown in the figure, and in combination with the specific implementation process, illustrate the functions of each module of the network resource call optimization device.

[0118] As an optional implementation manner, in the technical solution provided in the above step S102, the acquisition module 22 (that is, Figure 3 the resource collection layer inside) can monitor the graph change events in the target network. For example, the network element acquisition and control module monitors SNMP Trap or Netconf events of physical devices, and captures the running state changes of physical devices in real time, such as device online, device offline, link state change, etc.; the virtualization collector listens to the life cycle events of virtual resources (such as VMs, containers), such as virtual resource migration. At the same time, the acquisition module 22 can also monitor the network service requirement information of the target network. For example, the service awareness module parses the service opening, adjustment, or cancellation requirements from the BMO domain order data.

[0119] Then, the acquisition module 22 formats the captured set of graph change events and service requirement information into a unified event stream, and efficiently transmits it to the update module 24 (that is, Figure 3within the dynamic graph engine layer). Among them, Kafka, as an event middleware, ensures the reliable transmission and real-time processing of events.

[0120] Then, after receiving the event stream, the update module 24 can parse the event stream and update the network timing graph of the target network according to the set of graph change events obtained from the parsing.

[0121] Among them, the method for constructing the network timing graph of the target network is as follows:

[0122] The first step: Obtain multiple resource entities within the target network, where the resource entities include but are not limited to: physical devices (such as routers, switches), virtual resources (such as virtual machines, containers), etc.

[0123] The second step: Determine the attribute information corresponding to each resource entity, and determine the resource scheduling relationship and resource transmission quality between every two resource entities. Among them, the above-mentioned attribute information includes: static attribute information (such as device type, geographical location, fixed capacity index, etc.) and dynamic attribute information (such as CPU utilization rate, memory usage, bandwidth consumption, etc.). In addition, the above-mentioned resource scheduling relationship reflects whether there is a need or possibility of resource scheduling between every two resource entities, and the resource transmission quality between every two resource entities reflects the real-time transmission state, which can be obtained by calculating transmission quality indicators (such as latency, packet loss rate, bandwidth utilization rate).

[0124] The third step: Use each resource entity as a node, use the resource scheduling relationship between every two resource entities as a directed edge, and use the resource transmission quality between any two resource entities as the weight of the edge to construct a network timing graph.

[0125] Therefore, each resource entity within the target network exists in the form of a node, carrying its static and dynamic attribute information, and the scheduling relationship between entities is represented by directed edges, and the weights of the edges are dynamically updated according to the resource transmission quality. The network timing graph formed in this way can comprehensively and real-time reflect the resource connection status and performance of the target network.

[0126] Furthermore, the update module 24 (i.e., Figure 3 the incremental graph construction module within) can traverse each graph change event within the set of graph change events, parse the graph change events, determine the event type and event content corresponding to the graph change events, where the event type includes at least one of the following: addition, update, deletion; perform operations corresponding to the event type on the network timing graph according to the event content to obtain the updated network timing graph.

[0127] That is to say, for each graph change event, the system can generate corresponding graph operation instructions. For example, if the graph change event is a device online event, the system will add a new node in the network time series graph and assign corresponding static attribute information (such as device type, location) and dynamic attribute information (such as current CPU usage, memory occupancy, bandwidth consumption, etc.) to this node; if the graph change event is a link state change event, the system will update the weights of relevant edges in the network time series graph to reflect the latest resource transmission quality; if the graph change event is a device offline event, the system will trigger a node deletion operation to delete relevant nodes in the network time series graph. Thus, the network time series graph of the target network is updated in real time to maintain the consistency between the graph and the actual network state, providing the most accurate data basis for subsequent network resource management and optimization.

[0128] Therefore, updating the network time series graph through the above incremental graph construction mechanism means that each time a change event is received, it only updates the part of the graph involved, rather than reconstructing the entire graph. This mechanism significantly reduces resource consumption and improves the update efficiency. At the same time, the Kafka event-driven architecture ensures that events can be processed quickly and reliably, guaranteeing the real-time nature of the graph even under a large number of concurrent events.

[0129] In addition, the update module 24 can perform persistent storage on the updated network time series graph and adopt a version control mechanism to record each change. This can not only save the latest state of the network but also track the change process of resources through historical versions, providing a basis for fault diagnosis and rollback operations.

[0130] Furthermore, the judgment module 26 (i.e., Figure 3 the topology anomaly detection module inside) can judge whether there is a network anomaly in the target network according to the following method, including:

[0131] Step S1, analyze the updated network time series graph using a preset anomaly detection model to determine the anomaly probability of each node.

[0132] Step S2, when the anomaly probability of any node in the updated network time series graph is higher than the preset probability threshold, it is determined that there is an anomaly in the target network;

[0133] Step S3, when the anomaly probabilities of all nodes in the updated network time series graph are not higher than the preset probability threshold, it is determined that there is no anomaly in the target network.

[0134] Optionally, in the technical solution provided in the above step S1, the anomaly detection model is at least composed of a graph attention network layer and a graph isomorphism network layer. Among them, the GAT layer assigns different weights to different neighbors of each node by learning the attention mechanism, and these weights reflect the influence degree of neighbor nodes on the central node; the GIN layer updates the feature representation of the node by aggregating the neighbor information of the node and applying a multi-layer perceptron. Therefore, the anomaly detection model can determine the anomaly probability of each node in the network time series graph according to the following steps, including:

[0135] The first step: For each node in the updated network time series graph, determine the corresponding feature vector according to the attribute information of the current node extracted from the updated network time series graph, and construct the adjacency matrix of the current node based on all the directed edges with the current node as the endpoint, where the adjacency matrix is used to record the feature vectors of all neighbor nodes of the node.

[0136] The second step: Use the graph attention network layer to analyze the feature vector to obtain the attention weight vector of the node, and update the feature vector with the attention weight vector and the adjacency matrix, where the attention weight vector includes the attention weights between the node and each neighbor node.

[0137] The third step: For each layer of perceptron in the graph isomorphism network layer, scale the updated feature vector output by the previous layer of perceptron using the preset parameters, and superimpose the updated feature vectors of all neighbor nodes of the current node to obtain the updated feature vector;

[0138] The fourth step: Use the activation function layer to analyze the updated feature vector output by the last layer of perceptron in the multi-layer perceptron to obtain the anomaly probability of the node.

[0139] Therefore, if the neighbor matrix of node i in the updated network time series graph is denoted as A i ∈R n×n , and the node features (such as device type, geographical location, resource utilization rate, etc.) are denoted as X i ∈R n×d , where n represents the number of nodes in the subgraph where node i is located, and d represents the number of feature dimensions of node i. Then, the anomaly detection model can calculate the anomaly probability of this node according to the following steps:

[0140] The first step: First calculate the attention coefficient e between node i and its neighbor node j according to the following formula ij :

[0141] e ij =LeakyReLU(a T [Wh i ||Wh j )

[0142] Wherein, h i represents the feature vector obtained by performing feature extraction on the node feature X i of node i, and h j represents the feature vector obtained by performing feature extraction on the node feature X j of neighbor node j. W represents the weight matrix of the graph attention network, and a represents the attention vector.

[0143] Step 2: Use the softmax function to normalize the attention coefficients e ij of all neighbor nodes of node i to obtain the attention weight α ij :

[0144]

[0145] Wherein, N(i) represents the set of neighbor nodes of node i.

[0146] Step 3: Aggregate the features of all neighbor nodes of node i using the attention weight α ij to update the feature vector of node i:

[0147] h i′ = σ(∑ j∈N(i) α ij Wh j )

[0148] Step 4: Scale the updated feature vector of node i using the learnable parameters set by the multi-layer perceptron as follows:

[0149]

[0150] Wherein, p represents the number of layers of the perceptron, represents the feature vector of node i after scaling processing by the (p-1)-th layer perceptron, represents the feature vector of the adjacent node i after scaling processing by the (p-1)-th layer perceptron, represents the feature vector of node i after scaling processing by the p-th layer perceptron.

[0151] Step 5: Analyze the updated feature vector output by the last layer perceptron in the multi-layer perceptron using the following activation function to obtain the anomaly probability of the node:

[0152]

[0153] Wherein, W o represents the weight matrix of the Sigmoid function, and p i represents the anomaly probability of node i.

[0154] Therefore, through the methods provided in the above first step to fifth step, the anomaly probabilities of each node in the network resource graph can be obtained.

[0155] It should be noted that the loss function of the above anomaly detection model can adopt Focal Loss. Therefore, the loss function of the anomaly detection model can be written as:

[0156] FocalLoss(p,y)=-α(1 - p) γ ylog(p)

[0157] Among them, γ represents an adjustable factor, p represents the degree of proximity to class y. The larger p is, the closer it is to class y, that is, the more accurate the classification is.

[0158] As an alternative implementation, Figure 3 the resource scheduling optimization module within can train the policy decision model according to the following steps, including:

[0159] Construct an online Q-network and a policy network, and initialize the network weight parameters of the online Q-network and the policy network. Among them, the online Q-network is used to solve the network resource scheduling policy, and the policy network includes multiple network resource scheduling policies, such as path network resource scheduling policies, virtual machine migration policies, and bandwidth expansion policies.

[0160] Take the network weight parameters of the initialized online Q-network as the network parameters of the target Q-network.

[0161] Set up an experience pool and determine the capacity of the experience pool.

[0162] Determine the first preset number of iteration cycles, and iteratively solve the network resource scheduling policy and network weight parameters through the following steps:

[0163] In each iteration cycle, initialize the network state and determine the second preset number of calculation cycles. Among them, the network state includes: network time series graph and network service demand information. And the network service demand information can be in matrix form, and each row in the network service demand matrix includes at least: service ID, source node, target node, and required bandwidth;

[0164] In each computing cycle, the current network state is input into the policy network to obtain a policy probability distribution. Then, a network resource scheduling policy is sampled according to the policy probability distribution. Next, the policy adjustment and the current network state are input into the online Q-network to calculate the predicted Q-value corresponding to the network resource scheduling policy. Execute the network resource scheduling policy and calculate the reward to obtain a new network state. Then, use the current network state, the network resource scheduling policy, the reward, and the new network state as a sample, and store the sample in the experience pool. Input multiple samples randomly sampled from the experience pool into a neural network containing a bidirectional long short-term memory network, calculate the probability of each sample being sampled, the mean squared error loss function, and the loss function weight. Determine the target Q-value according to the calculation results, and update the network weight parameters of the online Q-network according to the target Q-value and the predicted Q-value.

[0165] After a third preset number of computing cycles, update the network weight parameters of the target Q-network according to the network weight parameters of the online Q-network, where the third preset number is less than the second preset number.

[0166] After the iteration is completed, use the obtained target Q-network as the policy decision model.

[0167] Finally, the optimization module 28 (i.e., Figure 3 the resource scheduling optimization module within) can call the above-mentioned trained policy decision model to analyze the updated network timing graph and network service demand information, obtain the corresponding target network resource scheduling policy, and optimize the network resource scheduling of the target network according to the target network resource scheduling policy.

[0168] Specifically, the target network resource scheduling policy includes but is not limited to the following several policies: path network resource scheduling policy, virtual machine migration policy, bandwidth expansion policy, etc.

[0169] In addition, the optimization module 28 (i.e., Figure 3 the visualization parser within) can also present the results of anomaly detection on the target network and the network resource scheduling policy for resource topology optimization of the target network in the form of a decision tree, which is convenient for operation and maintenance personnel to understand and execute. This not only improves the transparency of decision-making but also reduces the professional skill requirements, enabling non-expert personnel to quickly understand and operate complex resource scheduling decisions, thus improving the operation and maintenance efficiency and decision-making quality.

[0170] It should be noted that each module in the network resource call optimization device in the embodiment of the present application corresponds to each implementation step of the network resource call optimization method in Embodiment 1. Since the description in Embodiment 1 is already detailed, some details not shown in this embodiment can be referred to Embodiment 1 and will not be elaborated here.

[0171] Embodiment 3

[0172] According to an embodiment of the present application, there is also provided a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the network resource call optimization method in Embodiment 1.

[0173] According to an embodiment of the present application, there is also provided a non-volatile storage medium, which includes a stored computer program. When the device where the non-volatile storage medium is located runs the computer program, it executes the network resource call optimization method in Embodiment 1.

[0174] According to an embodiment of the present application, there is also provided a processor, which is used to run a computer program. When the computer program runs, it executes the network resource call optimization method in Embodiment 1.

[0175] According to an embodiment of the present application, there is also provided an electronic device, which includes a memory and a processor. A computer program is stored in the memory, and the processor is configured to execute the network resource call optimization method in Embodiment 1 through the computer program.

[0176] Specifically, when the computer program runs, it executes the following steps: obtaining a set of graph change events and network service requirement information of the target network within a preset time period; updating the network time-series graph of the target network according to the set of graph change events, where multiple nodes in the network time-series graph are respectively used to store the attribute information of a resource entity in the target network, and multiple directed edges in the network time-series graph are respectively used to represent the connection relationships between multiple nodes, and the attribute information includes: static attribute information for uniquely identifying the resource entity and dynamic attribute information for characterizing the resource status time-series information of the resource entity; judging whether the target network has an abnormality according to the updated network time-series graph; in the case where the target network has an abnormality, analyzing the updated network time-series graph and network service requirement information by using a pre-trained policy decision model to obtain a corresponding target network resource scheduling policy, and optimizing the network resource scheduling of the target network according to the target network resource scheduling policy.

[0177] As an optional implementation manner, the above-mentioned electronic device may exist in the form of a mobile terminal, a computer terminal or a similar computing device. Figure 4 Shows a hardware structure block diagram of an electronic device for implementing the network resource call optimization method. As Figure 4As shown, the electronic device 40 may include one or more processors 402 (shown as 402a, 402b, ……, 402n in the figure) (the processor 402 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 404 for storing data, and a transmission device 406 for communication functions. In addition, it may further include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply, and / or a camera. Those of ordinary skill in the art can understand that Figure 4 the structure shown is only schematic and does not limit the structure of the above-mentioned electronic device. For example, the electronic device 40 may further include more or fewer components than Figure 4 shown in Figure 4 the figure, or have a different configuration from

[0178] It should be noted that the above one or more processors 402 and / or other data processing circuits are generally referred to as "data processing circuits" in this article. The data processing circuit may be embodied in software, hardware, firmware, or any combination thereof, in whole or in part. In addition, the data processing circuit may be a single independent processing module, or be incorporated in whole or in part into any one of the other elements in the electronic device 40. As involved in the embodiments of the present application, the data processing circuit is a processor control (such as the selection of a variable resistance terminal path connected to an interface).

[0179] The memory 404 may be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the network resource call optimization method in the embodiments of the present application. The processor 402 executes various functional applications and data processing by running the software programs and modules stored in the memory 404, that is, implements the vulnerability detection method of the above application program. The memory 404 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 404 may further include a memory remotely provided with respect to the processor 402, and these remote memories may be connected to the electronic device 40 through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0180] The transmission device 406 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by a communication provider of the electronic device 40. In one example, the transmission device 406 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a Radio Frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0181] The display can be, for example, a touch-screen liquid crystal display (LCD), which enables the user to interact with the user interface of the electronic device 40.

[0182] The above-mentioned embodiment numbers are only for description and do not represent the advantages or disadvantages of the embodiments.

[0183] In the above embodiments of the present application, the descriptions of each embodiment have their own focuses. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0184] In several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of units can be a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of units or modules can be in electrical or other forms.

[0185] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0186] In addition, each functional unit in the various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0187] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), mobile hard disks, magnetic disks, or optical discs.

[0188] The above are only the preferred embodiments of this application. It should be noted that for those of ordinary skill in the art, without departing from the principle of this application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of this application.

Claims

1. A method for optimizing network resource calls, characterized in that, Including: Obtain a set of graph change events and network service demand information of the target network within a preset time period; Update the network time-series graph of the target network according to the set of graph change events, wherein multiple nodes in the network time-series graph are respectively used to store attribute information of a resource entity in the target network, and multiple directed edges in the network time-series graph are respectively used to represent connection relationships between the multiple nodes, and the attribute information includes: static attribute information for uniquely identifying the resource entity, and dynamic attribute information for representing resource status time-series information of the resource entity; Judge whether the target network has an anomaly according to the updated network time-series graph; When the target network has an anomaly, analyze the updated network time-series graph and the network service demand information by using a pre-trained policy decision model to obtain a corresponding target network resource scheduling policy, and optimize the network resource scheduling of the target network according to the target network resource scheduling policy.

2. The method according to claim 1, wherein The construction process of the network time-series graph includes: Obtain multiple resource entities in the target network, wherein the resource entity includes at least one of the following: physical device, virtual resource; Determine the attribute information corresponding to each resource entity, and determine the resource scheduling relationship and resource transmission quality between every two resource entities; Construct the network time-series graph by using each resource entity as a node, the resource scheduling relationship between every two resource entities as a directed edge, and the resource transmission quality between every two resource entities as the weight of the edge.

3. The method according to claim 1, wherein Updating the network time-series graph of the target network according to the set of graph change events includes: Traverse each graph change event in the set of graph change events, parse the graph change event to determine the event type and event content corresponding to the graph change event, wherein the event type includes at least one of the following: addition, update, deletion; perform an operation corresponding to the event type on the network time-series graph according to the event content to obtain the updated network time-series graph.

4. The method according to claim 1, characterized in that, Judging whether the target network has a network anomaly according to the updated network time-series graph includes: Analyze the updated network time-series graph by using a preset anomaly detection model to determine the anomaly probability of each node, wherein the anomaly detection model is at least composed of a graph attention network layer and a graph isomorphism network layer, and the graph isomorphism network layer includes a multi-layer perceptron; When the anomaly probability of any node in the updated network time-series graph is higher than a preset probability threshold, determine that the target network has an anomaly; When the anomaly probabilities of all nodes in the updated network time-series graph are not higher than the preset probability threshold, determine that the target network has no anomaly.

5. The method according to claim 4, wherein Analyzing the updated network time-series graph by using a preset anomaly detection model to determine the anomaly probability of each node includes: For each of the nodes in the updated network time series graph, determine the corresponding feature vector based on the attribute information of the current node extracted from the updated network time series graph, and construct the adjacency matrix of the current node based on all the directed edges with the current node as an endpoint, where the adjacency matrix is used to record the feature vectors of all the neighbor nodes of the node; Use the graph attention network layer to analyze the feature vector to obtain the attention weight vector of the node, and update the feature vector with the attention weight vector and the adjacency matrix, where the attention weight vector includes the attention weights between the node and each of the neighbor nodes; For each perceptron in the graph isomorphism network layer, scale the updated feature vector output by the previous perceptron using preset parameters, and stack the updated feature vectors of all the neighbor nodes of the current node to obtain the updated feature vector; Use the activation function layer to analyze the updated feature vector output by the last perceptron in the multi-layer perceptron to obtain the anomaly probability of the node.

6. The method according to claim 1, characterized in that, The training process of the policy decision model includes: Construct an online Q network and a policy network, and initialize the network weight parameters of the online Q network and the policy network, where the online Q network is used to solve the network resource scheduling policy, and the policy network includes multiple network resource scheduling policies; Use the network weight parameters of the initialized online Q network as the network parameters of the target Q network; Set up an experience pool and determine the capacity of the experience pool; Determine the first preset number of iteration cycles, and iteratively solve the network resource scheduling policy and the network weight parameters through the following steps: In each iteration cycle, initialize the network state, and determine the second preset number of calculation cycles, where the network state includes: the updated network time series graph and the network service demand information; In each calculation cycle, input the current network state into the policy network to obtain the policy probability distribution, sample a network resource scheduling policy according to the policy probability distribution, input the policy adjustment and the current network state into the online Q network, and calculate the predicted Q value corresponding to the network resource scheduling policy; execute the network resource scheduling policy and calculate the reward, obtain the new network state, and use the current network state, the network resource scheduling policy, the reward and the new network state as a sample, store the sample in the experience pool; input multiple samples randomly sampled from the experience pool into a neural network containing a bidirectional long short-term memory network, calculate the probability of each sample being sampled, the mean square error loss function and the loss function weight, determine the target Q value according to the calculation results, and update the network weight parameters of the online Q network according to the target Q value and the predicted Q value; After the third preset number of calculation cycles, update the network weight parameters of the target Q network according to the network weight parameters of the online Q network, where the third preset number is less than the second preset number; After the iteration is completed, the obtained target Q-network is used as the policy decision model.

7. The method according to claim 1, wherein The target network resource scheduling policy includes at least one of the following: path network resource scheduling policy, virtual machine migration policy, bandwidth expansion policy.

8. An apparatus for optimizing network resource calls, characterized in that, It includes: An acquisition module, configured to acquire a set of graph change events and network service demand information of a target network within a preset time period; An update module, configured to update the network time-series graph of the target network according to the set of graph change events, wherein multiple nodes in the network time-series graph are respectively used to store attribute information of a resource entity in the target network, and multiple directed edges in the network time-series graph are respectively used to represent the connection relationship between the multiple nodes, and the attribute information includes: static attribute information for uniquely identifying the resource entity, and dynamic attribute information for representing the resource state time-series information of the resource entity; A judgment module, configured to judge whether the target network has an abnormality according to the updated network time-series graph; An optimization module, configured to, when the target network has an abnormality, analyze the updated network time-series graph and the network service demand information by using a pre-trained policy decision model, obtain a corresponding target network resource scheduling policy, and optimize the network resource scheduling of the target network according to the target network resource scheduling policy.

9. A computer program product, characterized in that, It includes: A computer program, wherein when the computer program is executed by a processor, it implements the network resource call optimization method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes: A memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to execute the network resource call optimization method according to any one of claims 1 to 7 through the computer program.

Images