A satellite-ground fusion network node resource deployment method, system and electronic equipment

Abstract

The application provides a kind of star-ground fusion network node resource deployment method, system and electronic equipment, related to star-ground fusion network communication technical field.The method includes based on node resource and link resource consumption, with resource loss minimization as objective function, the multi-modal service resource dynamic deployment optimization problem under the scene of space-ground integration network is constructed as Markov decision process framework;Under the Markov decision process framework, the network state and action reward training multi-modal encoder are calculated using the proximal optimization strategy, and the optimal node resource deployment scheme is obtained and resource deployment is carried out.The application constructs the star-ground fusion network node resource deployment problem as Markov decision process framework and determines the optimal node resource deployment scheme using the proximal optimization strategy, which can improve the accuracy and rationality of star-ground fusion network node resource deployment.

Description

Technical Field

[0001] This invention relates to the field of space-ground converged network communication technology, and in particular to a method, system and electronic device for deploying space-ground converged network node resources. Background Technology

[0002] In recent years, converged space-ground networks have received significant attention from academia and industry, as they can extend network coverage to areas difficult to reach using traditional terrestrial networks. With the rapid expansion of space-ground distributed applications, the need for Service Function Chain (SFC) technology to provide differentiated services is increasingly valued. While converged space-ground network applications have experienced significant exponential growth, meeting the growing demands for diverse service resource deployments and heterogeneous Quality of Service (QoS) requirements, and achieving multimodal resource deployment and differentiated service needs, has become a challenge. Meeting these requirements is crucial for ensuring optimal system performance and user satisfaction. Multimodal service resource deployment requires effective modeling and orchestration of different resources such as computing power, storage resources, and bandwidth of satellite network nodes. However, deploying multimodal service resources in converged space-ground networks presents significant challenges. It requires effective orchestration of different resources while considering factors such as node selection, link capacity, and bandwidth. Furthermore, the dynamic nature of converged space-ground networks, with frequent additions, deletions, or capacity changes of network nodes and links, increases the complexity of the resource deployment process. Summary of the Invention

[0003] The purpose of this invention is to provide a method, system, and electronic device for deploying node resources in a space-ground converged network, which can improve the accuracy and rationality of the deployment of node resources in a space-ground converged network.

[0004] To achieve the above objectives, the present invention provides the following solution:

[0005] A method for deploying node resources in a space-ground integrated network includes:

[0006] Establish a multimodal service resource deployment architecture in a space-ground converged network;

[0007] The problem of dynamic deployment and optimization of multimodal service resources in the context of building a space-ground integrated network;

[0008] Based on node and link resource consumption, and with the goal of minimizing resource loss, the dynamic deployment optimization problem of multimodal service resources in the space-ground integrated network scenario is constructed as a Markov decision process framework.

[0009] Within the Markov decision process framework, a proximal optimization strategy is employed to compute network state and action rewards to train a multimodal encoder, thereby obtaining the optimal node resource deployment scheme. The multimodal encoder includes a request encoder, a network state encoder, and a global encoder.

[0010] Resource deployment for the space-ground converged network is carried out based on the optimal node resource deployment scheme.

[0011] Optionally, the establishment of a multimodal service resource deployment architecture in a space-ground converged network includes:

[0012] Define a directed graph G(N,E) to represent a physical network; where N is the number of satellite nodes in the network and E is the number of links between satellite network nodes;

[0013] definition This represents the CPU computing power of a satellite node; where t represents a single time slice. The CPU computing power of the nth node when the time slice is t. The CPU computing power of the nth node when the time slice is 0.

[0014] definition Indicates the remaining RAM capacity of the satellite node; The remaining RAM capacity of the nth node when the time slice is t. The remaining RAM capacity of the nth node when the time slice is 0.

[0015] definition This indicates that the remaining effective time of a ground network node in any time slice is positive infinity; The remaining effective time for the nth ground network node when the time slice is t. N represents the remaining effective time of the nth ground network node when the one-time time slice is 0; T This is a set of ground nodes; the nodes are either ground network nodes or satellite nodes.

[0016] definition Indicates the remaining valid time of the satellite node; Let N be the remaining valid time of the nth satellite node at time k×P; k is a natural number, P is the time for switching satellite coverage service areas; N s For satellite node integration; It is the set of natural numbers;

[0017] definition Indicates link E i,j bandwidth capacity; For a one-time time slice, link E is at time t. i,j bandwidth capacity; Link E when the one-time time slice is 0 i,j bandwidth capability; E i,j This represents the link between the i-th node and the j-th node.

[0018] Optionally, the objective function is:

[0019]

[0020] Where rev(t) represents the resource consumption function of the system in the t-th one-time time slice; T is the number of one-time time slices; C1 indicates that the latency of the link traversed by the data stream of the service request is less than the latency requirement D. u C2 indicates that the remaining bandwidth of any physical link carrying the data stream mapped by the service request is greater than the bandwidth resource BW required by the request. u C3 and C4 both indicate that the node embedding the VNFs provides the necessary resources for the VNFs; C5 indicates that each VNF can only be embedded on one node. This represents the u-th virtual network function that is activated in a specific order within the l-th service request. The (u+1)th virtual network function is activated in a specific order in the l-th service request. Does the data flow between them pass through link E? i,j ; Indicates link E i,j The sum of processing latency and transmission latency; The physical link E represents the data stream that carries the mapping of business requests. i,j The remaining bandwidth; This represents the u-th virtual network function that is activated in a specific order within the l-th service request. Whether to embed it into the nth node.

[0021] Optionally, within the Markov decision process framework, the step of using a proximal optimization strategy to calculate network state and action rewards to train a multimodal encoder and obtain the optimal node resource deployment scheme includes:

[0022] Obtain the first, second, and third observations;

[0023] The first observation is input into the request encoder to obtain the first query value;

[0024] The second observation is input into the network state encoder to obtain the key quantity;

[0025] The third observation is input into the global encoder to obtain the second query value;

[0026] The first query volume, the key volume, and the second query volume are all input into the actor network to obtain the node resource deployment scheme.

[0027] The first query volume, the key volume, the second query volume, and the node resource deployment scheme are input into the commentator network to obtain the reward function value;

[0028] Adjust the parameters of the request encoder, the network state encoder, and the global encoder, and return to the step "input the first observation into the request encoder to obtain the first query quantity" until the reward function value is less than the reward function value threshold to obtain the optimal node resource deployment scheme.

[0029] Optionally, the first observation is an observation of the heterogeneous quality of service requirements of the service request; the first observation is used to describe the source node and destination node identifiers of the service request, the VNF to be embedded, the required CPU computing resources, RAM resources, duration, bandwidth request, and latency requirements; the second observation is an observation of global network information extraction; the second observation is used to describe the availability of CPU, RAM, and link bandwidth in the LEO satellite network; the third observation is a graph-based network resource distribution observation; the third observation is used to describe the resource distribution of multiple nodes and links.

[0030] A satellite-ground integrated network node resource deployment system includes:

[0031] The multimodal service resource deployment architecture module is used to establish the multimodal service resource deployment architecture in the space-ground converged network.

[0032] The module for constructing dynamic deployment optimization problems of service resources is used to construct dynamic deployment optimization problems of multimodal service resources in the context of space-ground integrated network scenarios.

[0033] The Markov Decision Process Framework Building Module is used to construct a Markov Decision Process Framework for the dynamic deployment optimization problem of multimodal service resources in the space-ground integrated network scenario, based on node resource and link resource consumption and with the goal of minimizing resource loss.

[0034] The optimal node resource deployment scheme construction module is used to calculate the network state and action reward to train a multimodal encoder under the Markov decision process framework, and obtain the optimal node resource deployment scheme; the multimodal encoder includes a request encoder, a network state encoder and a global encoder.

[0035] The resource deployment module is used to deploy resources for the space-ground converged network based on the optimal node resource deployment scheme.

[0036] An electronic device includes a memory and a processor, the memory storing a computer program and the processor running the computer program to enable the electronic device to perform the aforementioned method for deploying node resources in a space-ground integrated network.

[0037] Optionally, the memory is a readable storage medium.

[0038] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0039] This invention provides a method, system, and electronic device for deploying node resources in a space-ground converged network. It establishes a multimodal service resource deployment architecture in the space-ground converged network; constructs a dynamic deployment optimization problem for multimodal service resources in a space-ground converged network scenario; based on node and link resource consumption, and with the objective function of minimizing resource loss, the dynamic deployment optimization problem for multimodal service resources in a space-ground converged network scenario is constructed as a Markov decision process framework; under the Markov decision process framework, a proximal optimization strategy is used to calculate network state and action rewards to train a multimodal encoder, obtaining the optimal node resource deployment scheme; the multimodal encoder includes a request encoder, a network state encoder, and a global encoder; and resource deployment is performed on the space-ground converged network based on the optimal node resource deployment scheme. This invention constructs the node resource deployment problem of the space-ground converged network as a Markov decision process framework and uses a proximal optimization strategy to determine the optimal node resource deployment scheme, which can improve the accuracy and rationality of node resource deployment in the space-ground converged network. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a flowchart of the satellite-ground integrated network node resource deployment method in Embodiment 1 of the present invention;

[0042] Figure 2 This is a schematic diagram of the multimodal proximal strategy optimization framework in Embodiment 1 of the present invention;

[0043] Figure 3 This is a schematic diagram of the application of the graphics converter in Embodiment 1 of the present invention;

[0044] Figure 4 This is a schematic diagram of the graphics converter in Embodiment 1 of the present invention. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] The purpose of this invention is to provide a method, system, and electronic device for deploying node resources in a space-ground converged network, which can improve the accuracy and rationality of the deployment of node resources in a space-ground converged network.

[0047] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0048] Example 1

[0049] like Figure 1 As shown, this embodiment provides a method for deploying node resources in a space-ground converged network, including:

[0050] Step 101: Establish a multimodal service resource deployment architecture in the space-ground converged network.

[0051] Step 102: Optimize the dynamic deployment of multimodal service resources in a space-ground integrated network scenario.

[0052] Step 103: Based on node resource and link resource consumption, and with the goal of minimizing resource loss, the dynamic deployment optimization problem of multimodal service resources in the space-ground integrated network scenario is constructed into a Markov decision process framework.

[0053] Step 104: Within the Markov decision process framework, a proximal optimization strategy is used to calculate network state and action rewards to train a multimodal encoder, obtaining the optimal node resource deployment scheme. The multimodal encoder includes a request encoder, a network state encoder, and a global encoder.

[0054] Step 105: Deploy resources for the space-ground converged network based on the optimal node resource deployment scheme.

[0055] Step 101 includes:

[0056] Step 101-1: Define a directed graph G(N,E) to represent the physical network. Here, N is the number of satellite nodes in the network, and E represents the number of links between satellite network nodes.

[0057] Step 101-2: Definition This represents the CPU computing power of a satellite node. Here, t represents a single time slice. This represents the CPU computing power of the nth node when the time slice is t. This represents the CPU computing power of the nth node when the time slice is 0.

[0058] Step 101-3: Definition This indicates the remaining RAM capacity of the satellite node. The remaining RAM capacity of the nth node when the time slice is t. This represents the remaining RAM capacity of the nth node when the time slice is 0.

[0059] Step 101-4: Definition This indicates that the remaining effective time of a ground network node in any time slice is positive infinity. The remaining effective time for the nth ground network node at time t is a one-time time slice. N represents the remaining effective time of the nth ground network node when the one-time time slice is 0. T This is a set of ground nodes. These nodes can be ground network nodes or satellite nodes.

[0060] Step 101-5: Definition Indicates the remaining valid time of the satellite node; N represents the remaining valid time of the nth satellite node at time k×P. k is a natural number, and P is the time for switching satellite coverage service areas. s For satellite node integration. It is the set of natural numbers.

[0061] Step 101-6: Definition Indicates link E i,j bandwidth capacity. For a one-time time slice, link E is at time t. i,j bandwidth capacity. Link E when the one-time time slice is 0 i,j bandwidth capability. E i,j This represents the link between the i-th node and the j-th node.

[0062] Specifically, the objective function is:

[0063]

[0064] Where rev(t) represents the resource consumption function of the system in the t-th one-time time slice. T is the number of one-time time slices. C1 indicates that the latency of the link traversed by the data stream of the service request is less than the latency requirement D. u C2 indicates that the remaining bandwidth of any physical link carrying the data stream mapped by the service request is greater than the bandwidth resource BW required by the request. uC3 and C4 both indicate that the node embedding the VNFs provides the necessary resources for the VNFs. C5 indicates that each VNF can only be embedded on one node. This represents the u-th virtual network function that is activated in a specific order within the l-th service request. The (u+1)th virtual network function is activated in a specific order in the l-th service request. Does the data flow between them pass through link E? i,j . Indicates link E i,j The sum of processing latency and transmission latency. The physical link E represents the data stream that carries the mapping of business requests. i,j The remaining bandwidth. This represents the u-th virtual network function that is activated in a specific order within the l-th service request. Whether to embed it into the nth node.

[0065] like Figure 2 Step 104 includes:

[0066] Step 104-1: Obtain the first observation, the second observation, and the third observation.

[0067] Step 104-2: Input the first observation into the request encoder to obtain the first query.

[0068] Step 104-3: Input the second observation into the network state encoder to obtain the key quantity.

[0069] Step 104-4: Input the third observation into the global encoder to obtain the second query.

[0070] Step 104-5: Input the first query volume, key volume and second query volume into the actor network to obtain the node resource deployment scheme.

[0071] Step 104-6: Input the first query volume, key volume, second query volume, and node resource deployment plan into the commentator network to obtain the reward function value.

[0072] Step 104-7: Adjust the parameters of the request encoder, network state encoder, and global encoder, and return to step 104-2: until the reward function value is less than the reward function value threshold, to obtain the optimal node resource deployment scheme.

[0073] Specifically, the first observation describes the source and destination node identifiers of the service request, the VNF to be embedded, the required CPU computing resources, RAM resources, duration, bandwidth request, and latency requirements. The second observation is an observation extracted from global network information. It describes the availability of CPU, RAM, and link bandwidth in the LEO satellite network. The third observation is a graph-based observation of network resource distribution. It describes the resource distribution of multiple nodes and links.

[0074] The multimodal proximal policy optimization method is a framework that combines reinforcement learning with multimodal input data. In this method, different modalities of the input data, such as grid-like and graph-structured data, are processed by separate neural networks, called modality-specific networks (MSNs). The outputs of these networks are then combined and fed into a common policy or value network, which learns to make decisions based on multimodal information. Furthermore, a graph structure converter is designed to handle graph-structured data. Additionally, two independent multilayer perceptrons (MLPs) are used to handle QoS requests and global network information during multimodal service resource deployment. Subsequently, two multimodal representation methods are designed, representing multimodal information as VNF embedding decisions and state values, respectively.

[0075] Specifically, the technical solution adopted in this invention is as follows: establishing a multimodal service resource deployment architecture in a space-ground integrated network; establishing a dynamic deployment optimization problem for multimodal service resources in a space-ground integrated network scenario, comprehensively considering node resource and link resource consumption, with the goal of minimizing resource loss; establishing a Markov decision process (MDP) framework; and designing a dynamic deployment algorithm for service resources based on a near-end policy optimization method.

[0076] Furthermore, this invention uses a directed graph G(N,E) to represent the physical network, where N represents the nodes in the network and E represents the links between satellite network nodes. Since this invention aims to solve the problem of multimodal service resource deployment optimization in satellite-ground integrated networks, firstly, [the following is a partial translation of the first paragraph, which is not directly related to the previous sentence]. Defined as the CPU computing power of a node. Where t represents a single slice, and each node n has a fixed amount of positive computational resources in the initial time slice. Similarly... Defined as the remaining RAM capacity of node n.

[0077] In addition to considering the common attributes of all nodes, it is also necessary to consider the situation where coverage of satellite nodes is switched due to the mobility of satellite nodes. Therefore, this invention defines the effective remaining time of ground nodes and satellite nodes. This indicates that the remaining effective time of a ground network node in any time slice is positive infinity. This represents the remaining effective time for a satellite node, where k is a natural number and P is the time for switching satellite coverage service areas. For each satellite node, at the start of the handover, the effective duration of the first time slice t is a fixed positive number. Similarly, this invention defines the properties of links in the network. Indicates link E i,j The bandwidth capacity. Here, i and j represent the two nodes connected through this link. Indicates link E i,j The time delay.

[0078]

[0079] The latency of each link is determined by the processing latency. and transmission delay It consists of two parts. Here... This includes Virtual Network Function (VNF) embedding latency and queuing latency. After defining the physical network attributes, the relevant attributes for dynamic deployment requests of service resources will be given. The i-th service request in time slice t can be represented as Q. l (t). This can be further expressed as: This means that for the l-th service request, u VNFs need to be started in a specific order, and each VNF needs to be embedded in a physical network node. Furthermore, each VNF can be represented as: Each VNF has different requirements for computing resources (CPU), storage resources (RAM), service duration, bandwidth, and latency. Then, a series of binary variables are defined to represent the deployment status of business requests.

[0080]

[0081] As shown in equation (2), using express Whether to embed it into node n.

[0082]

[0083] Then we get equation (3), where express and Does the data flow between them pass through link E? i,j .

[0084]

[0085] Finally, we obtain equation (4), z l (t) indicates whether the business request has been successfully served.

[0086] Based on the above, the resource depletion model can be represented as follows:

[0087]

[0088]

[0089]

[0090]

[0091] A resource consumption model is established from two aspects: node consumption and link consumption. Both models are based on time slice t.

[0092] As shown in equation (5), Indicates satellite node N S The overhead of a node. Node overhead includes the overhead of all VNFs embedded within the node. and So, p represents the resources consumed per unit.

[0093] Equation (6) considers VNF re-embedding The cost. When the duration of the business request library exceeds the remaining validity period of the satellite node library, the VNF needs to be re-embedded.

[0094] Equation (7) considers the link The consumption includes the bandwidth BW consumed by all VNFs u during time slice t. u Finally, equation (8), This indicates that the total consumption is the sum of these three types of consumption.

[0095] Furthermore, an optimization problem is formulated. First, the resource consumption function of the system in each time slice is given.

[0096]

[0097] in, Let represent the resource consumption of the l-th VNF at time slice t. The goal of this invention is to optimize the resource consumption function for the entire time slice, minimizing the overall system resource consumption, thereby improving the resource utilization of nodes and links and minimizing latency. The optimization problem can be expressed as:

[0098]

[0099] Here, C1 indicates that the latency of the links traversed by the data stream of the service request should be less than the latency requirement. C2 indicates that the remaining bandwidth of any physical link carrying the data stream mapped by the service request needs to be greater than the bandwidth resources required by the request. C3 and C4 indicate that the nodes embedding VNFs need to provide the necessary resources for the VNFs. C5 indicates that each VNF can only be embedded on one node. Then, the sum of the revenue functions for all time slices is defined.

[0100] like Figure 2 Furthermore, a Markov Decision Process (MDP) framework is established, and relevant notation used in deep reinforcement learning is defined. A Markov Decision Process is a mathematical framework for simulating sequential decision-making problems involving agent-environment interactions. It is defined by a tuple (S, A, R, P, γ), where S represents the set of multimodal states in the space-ground fusion network within the environment; A represents the set of actions available to the agent; P is the state transition probability function, defining the probability of transitioning from one state to another when a specific action is taken; R is the reward function obtained after taking an action, associating a scalar reward with each state-action pair; and γ is the discount factor that determines the importance of future rewards relative to immediate rewards. Agent Design: A more comprehensive definition and interpretation of the information contained in multimodal observations is established.

[0101]

[0102] As shown in equation (11), S req This represents an observation of the heterogeneous service quality requirements of a business request. It includes the source and destination node identifiers of the business request, the VNF to be embedded, the required CPU computing resources, RAM resources, duration, bandwidth request, and latency requirements.

[0103]

[0104] As shown in equation (12), S glo This represents the observations extracted from global network information. Resource utilization is used as an abstraction of the global state. These represent the availability of CPU, RAM, and link bandwidth in the LEO satellite network, respectively.

[0105]

[0106] As shown in equation (13), S ns This is a graph-based observation of network resource distribution. Each VNF represents the location record of each VNF embedded within the service resource deployment, encoded using one-hot encoding as node attribute labels. Furthermore, the agents in this modality observation also include the resource distribution of each node and link.

[0107] action = (Node n (14)

[0108] The agent's output action (representing the node identifier of the current VNF ​​deployment) is a discrete operation.

[0109]

[0110] The proxy's reward design has two cases. When the embedded VNF is not the last VNF ​​in the business request, the first part of the reward is Δd, where Δd represents the difference between the total latency of the current chain and the latency along the shortest path. Furthermore, the available resources on the embedded node are considered. and The utilization rate. If the VNF is the last one in the business request, this invention needs to add the reward (rev) to the reward.

[0111] Furthermore, establish a multi-modal working framework. For example... Figure 2 As shown, the working framework involves transforming multimodal information into VNF embeddings and state values ​​based on the proximal policy optimization method. This framework comprises three components: multimodal observation, a multimodal encoder, and a policy architecture based on the proximal policy optimization method.

[0112] 1) Multimodal Observation: In the service resource deployment task within a space-ground integrated network, the agent needs to make VNF embedding decisions based on observation information with different attributes. This invention processes this observation information with different attributes as multimodal observation. This invention divides the information observable by the agent into a multimodal observation space, represented as QoS requirements for service resource deployment requests, graph-based network resource allocation, and global network information extraction. The service resource deployment request observation information mainly includes source nodes, destination nodes, bandwidth, latency, and the network resources required by each VNF. For graph-based network resource allocation, this invention directly inputs the entire space-ground integrated network into the model as a directed graph. This invention uses the attributes of nodes and links to represent the resource distribution of nodes and links. For global network information extraction, this invention needs to extract global resource information from the resource distribution of the terrestrial network and the LEO satellite network respectively. This can provide decision information for the agent to embed VNFs in different types of networks.

[0113] 2) Multimodal Encoder: For each modal observation space, an independent neural network model is used for modal representation processing, which is called a multimodal encoder. In the multimodal encoder, this invention uses different types of models to adapt to the multiple features of the observation information. For the request encoder and global encoder, due to the low dimension of their observation modes and the weak correlation between information, this invention uses a multilayer perceptron (MLP) to encode this information separately. For the network state encoder, this invention needs to use a model that supports graph structure representation oriented towards the input graph. This invention aims for the model to learn the potential relationship between node connectivity and resource distribution and VNF embeddings. Furthermore, since VNF embeddings have a prescribed order, this invention also aims for the model to learn the mutual influence between each VNF embedding. To meet the above requirements, such as Figure 3This invention employs a graph converter as a network state encoder. A graph converter is a machine learning model that applies the concept of a converter architecture to graph-structured data. A graph is a data structure composed of nodes (or vertices) and edges, which can represent complex relationships between different entities. In a graph converter, each node in the graph is associated with a feature vector, and the goal is to learn a new representation for each node based on its features and the features of its neighbors. This is achieved through a series of layers, each applying a self-attention mechanism to measure the importance of a node's neighbors.

[0114] 3) Policy Architecture Based on Proximity Policy Optimization: Regarding the policy architecture, this invention employs an actor-critic framework based on the proximity policy optimization algorithm. It should be noted that the vectors output by different multimodal encoders are referred to as keys and queries. This invention treats the output of the network state encoder as the key, and the outputs of the request encoder and the global encoder as queries. This invention designs multimodal representation modules for actors and critics respectively.

[0115] MMR actor =sum[Key·(Query 1+Query 2)] (16)

[0116] As shown in (16), the present invention uses MMR actor Representing a multimodal representation of participants, this invention calculates the product of each key with the sums of Query1 and Query2, and then sums these products. Afterward, it undergoes a softmax process to output a discrete action.

[0117] MMR critic =avg(Key)+Query 1+Query 2 (17)

[0118] Similarly, the present invention uses MMR critic To represent the commentator's multimodal representation, this invention calculates the average of the keys and then adds it to Query1 and Query2. The commentator then uses an MLP called a value header to output a one-dimensional state value.

[0119] A graphics converter is essentially a network state encoder. For example... Figure 4 As shown, this illustrates the operating principles of the graph converter. Node features and link features serve as inputs to the graph converter. Each node and its neighboring nodes are fed through a linear layer and then input into a scaled dot product attention layer.

[0120]

[0121] Equation (18) calculates the attention coefficient of each node and its neighboring nodes by comparing the compatibility of each node's query and key vector with edge features.

[0122] Here, a set of node features is defined as This represents the query vector of node i at level l, expressed using a weight matrix. and bias vector Hidden state of node i It is obtained by performing a linear transformation. The key vector of node j at layer l is represented by the weight matrix. and bias vector Hidden state of node j It is obtained by performing a linear transformation. c,ij Represents the weight matrix W c,e and bias vector b c,e Features e of edge (i,j) ij Edge features obtained by performing a linear transformation. This represents the attention weight between node i and node j in layer 1, indicating the importance of node i relative to node j. It is obtained by taking the query vector. and key vector The inner product, plus the edge feature e c,ij Then, it is calculated by normalizing using a normalization factor. Let i represent the set of neighboring nodes of node i. Furthermore, the computation performed to update the hidden representation of a node in the graphics converter can be represented as:

[0123]

[0124] in, This represents the value vector of the first-level adjacent nodes. Similar to the query and key vectors, it is generated by analyzing the values ​​of adjacent nodes. Linear transformation of the hidden representation And add a bias term Obtained. This represents the hidden representation of node i after updating at the next level (l+1). Its calculation method is based on the set... Sum the neighbors j in the neighborhood of node i, and sum the attention coefficients. The value vector corresponding to each attention head c and edge features e c,ij Multiply the sum of the terms.

[0125]

[0126]

[0127]

[0128] First, consider the hidden state of node i. Perform a linear transformation to obtain the reset gate. Then the updated hidden state Reset gate The difference between the two values ​​is subjected to a series linear transformation, and then the update gate is obtained through the sigmoid function. Finally, using the update gate Updated hidden state and reset gate Perform a weighted sum, apply layer normalization and ReLU nonlinear transformation, and calculate the updated hidden state.

[0129] The policy architecture based on proximal policy optimization utilizes the proximal policy optimization algorithm to train and update the parameters of a multimodal model. The proximal policy optimization method ensures a stable and efficient learning process by restricting policy updates to avoid significant deviations from the current policy.

[0130] In proximal policy optimization methods, the objective function L CLIP (θ) aims to punish the new strategy π θ (a t |s t ) and old strategy The divergence between them. Ratio ρ t (θ) is defined as:

[0131]

[0132] Cutting substitute target L CLIP (θ) blocking will cause π θ (a t |s t Overly variable updates, which disrupt the stability of the training process, are defined as follows:

[0133]

[0134] in Let r represent the expected value. t (θ) represents the probability of the model output. This might represent the action selected in time slice t during near-end policy optimization. clip(r) t (θ), 1-∈, 1+∈) represent the condition for r t (θ) performs a truncation operation, restricting the value to the range [1-∈, 1+∈], used to control the range of fractions or probabilities to prevent values ​​that are too large or too small, where ∈ is the smallest possible number between 0 and 1.

[0135] The loss function in the near-end policy optimization method includes the policy loss L. CLIP (θ) and value function loss L VF (θ), that is, the estimated state value V(s) t The squared error loss between the actual return Gt and the actual return Gt:

[0136] L VF (θ)=(V θ (s t )-G t ) 2

[0137] The total loss L(θ) combines these terms with the entropy reward H(θ) that promotes exploration:

[0138]

[0139] Here, c1 and c2 are coefficients that balance these terms. Then, the policy parameters θ are updated using gradient ascent to maximize the total loss L(θ):

[0140]

[0141] Here, α is the learning rate. This is the gradient of the loss function with respect to the policy parameters. This process is repeated until convergence or the maximum number of iterations is reached. Entropy, denoted as H(θ), can be defined as:

[0142] H(θ) = -∑π θ (A t |S t logπ θ (A t |S t )

[0143] The generalized advantage estimate (GAE) of the advantage function at time t is:

[0144]

[0145] The equation includes a discount factor γ and a GAE parameter λ. It is summed over all future time steps. The time difference (TD) error of the value function.

[0146] Example 2

[0147] In order to execute the method corresponding to Embodiment 1 above and achieve the corresponding functions and technical effects, a satellite-ground converged network node resource deployment system is provided below, including:

[0148] The multimodal service resource deployment architecture module is used to establish the multimodal service resource deployment architecture in the space-ground converged network.

[0149] The module for constructing dynamic deployment optimization problems of business resources is used to construct dynamic deployment optimization problems of multimodal business resources in the context of space-ground integrated network scenarios.

[0150] The Markov Decision Process Framework Construction Module is used to construct a Markov Decision Process framework for the dynamic deployment optimization problem of multimodal service resources in the space-ground integrated network scenario, based on node resource and link resource consumption and with the goal of minimizing resource loss.

[0151] The optimal node resource deployment scheme construction module is used to calculate network state and action rewards and train a multimodal encoder within the Markov decision process framework, thereby obtaining the optimal node resource deployment scheme. The multimodal encoder includes a request encoder, a network state encoder, and a global encoder.

[0152] The resource deployment module is used to deploy resources for the space-ground converged network based on the optimal node resource deployment scheme.

[0153] Example 3

[0154] This embodiment provides an electronic device, including a memory and a processor. The memory is used to store computer programs, and the processor runs the computer programs to enable the electronic device to execute the satellite-ground fusion network node resource deployment method described in Embodiment 1.

[0155] The memory is a readable storage medium.

[0156] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0157] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for deploying node resources in a space-ground integrated network, characterized in that, include: Establish a multimodal service resource deployment architecture in a space-ground converged network; The problem of dynamic deployment and optimization of multimodal service resources in the context of building a space-ground integrated network; Based on node and link resource consumption, and with the goal of minimizing resource loss, the dynamic deployment optimization problem of multimodal service resources in the satellite-ground converged network scenario is constructed as a Markov decision process framework. Within the Markov decision process framework, a proximal optimization strategy is employed to compute network state and action rewards to train a multimodal encoder, thereby obtaining the optimal node resource deployment scheme. The multimodal encoder includes a request encoder, a network state encoder, and a global encoder. Deploy resources for the space-ground converged network based on the optimal node resource deployment scheme; Within the Markov decision process framework, a proximal optimization strategy is employed to calculate network state and action rewards to train a multimodal encoder, resulting in an optimal node resource deployment scheme. This includes: Obtain the first, second, and third observations; The first observation is input into the request encoder to obtain the first query value; The second observation is input into the network state encoder to obtain the key quantity; The third observation is input into the global encoder to obtain the second query. The first query volume, the key volume, and the second query volume are all input into the actor network to obtain the node resource deployment scheme. The first query volume, the key volume, the second query volume, and the node resource deployment scheme are input into the commentator network to obtain the reward function value; Adjust the parameters of the request encoder, the network state encoder, and the global encoder, and return to the step "input the first observation into the request encoder to obtain the first query value" until the reward function value is less than the reward function value threshold to obtain the optimal node resource deployment scheme; The first observation is an observation of the heterogeneous quality of service requirements of the service request; the first observation is used to describe the source node and destination node identifiers of the service request, the VNF to be embedded, the required CPU computing resources, RAM resources, duration, bandwidth request, and latency requirements; the second observation is an observation of global network information extraction; the second observation is used to describe the availability of CPU, RAM, and link bandwidth in the LEO satellite network; the third observation is a graph-based observation of network resource distribution; the third observation is used to describe the resource distribution of multiple nodes and links.

2. The method for deploying node resources in a space-ground integrated network according to claim 1, characterized in that, The establishment of a multimodal service resource deployment architecture in a space-ground converged network includes: Define a directed graph This represents the physical network; where N is the number of satellite nodes in the network, and E is the number of links between satellite network nodes. definition This represents the CPU computing power of a satellite node; where t represents a single time slice. The CPU computing power of the nth node when the time slice is t. The CPU computing power of the nth node when the time slice is 0. definition Indicates the remaining RAM capacity of the satellite node; The remaining RAM capacity of the nth node when the time slice is t. The remaining RAM capacity of the nth node when the time slice is 0. definition This indicates that the remaining effective time of a ground network node in any time slice is positive infinity; The remaining effective time for the nth ground network node when the time slice is t. The remaining effective time for the nth ground network node when the one-time time slice is 0; This is a set of ground nodes; the nodes are either ground network nodes or satellite nodes. definition Indicates the remaining valid time of the satellite node; Let P be the remaining valid time of the nth satellite node at time k×P; k is a natural number, and P is the time for switching satellite coverage service areas. For satellite node integration; It is the set of natural numbers; definition Indicates link bandwidth capacity; For a one-time time slice, the link is at time t. bandwidth capacity; Link when time slice is 0 bandwidth capacity; This represents the link between the i-th node and the j-th node.

3. The method for deploying satellite-ground integrated network node resources according to claim 2, characterized in that, The objective function is: ； ； in, This represents the resource consumption function of the system in the t-th one-time time slice; T is the number of one-time time slices; C1 indicates that the latency of the links traversed by the data stream of the business request is less than the latency requirement. C2 indicates that the remaining bandwidth of any physical link carrying the data stream mapped by the service request is greater than the bandwidth resources required by the request. C3 and C4 both indicate that the node embedding the VNFs provides the necessary resources for the VNFs; C5 indicates that each VNF can only be embedded on one node. Indicates the first The uth virtual network function that is activated in a specific order from a series of service requests With the The (u+1)th virtual network function activated in a specific order from a series of service requests. Does the data flow between them pass through a link? ; Indicates link The sum of processing latency and transmission latency; The physical link that carries the data stream mapped by the business request. The remaining bandwidth; Indicates the first The uth virtual network function that is activated in a specific order from a series of service requests Whether to embed it into the nth node.

4. A satellite-ground integrated network node resource deployment system, characterized in that, The system applies the satellite-ground integrated network node resource deployment method as described in any one of claims 1-3, and the system includes: The multimodal service resource deployment architecture module is used to establish the multimodal service resource deployment architecture in the space-ground converged network. The module for constructing dynamic deployment optimization problems of service resources is used to construct dynamic deployment optimization problems of multimodal service resources in the scenario of space-ground converged network. The Markov Decision Process Framework Building Module is used to construct a Markov Decision Process Framework for the dynamic deployment optimization problem of multimodal service resources in the satellite-ground converged network scenario, based on node resource and link resource consumption and with the goal of minimizing resource loss. The optimal node resource deployment scheme construction module is used to calculate the network state and action reward to train a multimodal encoder under the Markov decision process framework, and obtain the optimal node resource deployment scheme; the multimodal encoder includes a request encoder, a network state encoder and a global encoder. The resource deployment module is used to deploy resources for the space-ground converged network based on the optimal node resource deployment scheme.

5. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program and the processor runs the computer program to enable the electronic device to perform a method for deploying satellite-ground integrated network node resources according to any one of claims 1 to 3.

6. An electronic device according to claim 5, characterized in that, The memory is a readable storage medium.

Images