Routing, routing path, multicast routing path decision method and electronic device

Abstract

This application provides a routing, routing path, and multicast routing path decision-making method and electronic device. The routing decision-making method includes acquiring first information of a source node satellite and second information of a destination node satellite in a routing path decision-making task; determining a satellite to be decided upon, and designating neighboring node satellites of the satellite to be decided upon as candidate routing node satellites; acquiring the real-time traffic load of the candidate routing node satellites and the predicted traffic load within a preset future time step; and, based on the real-time traffic load, the predicted traffic load, the first information, and the second information, determining the next routing node satellite from the candidate routing node satellites using a routing decision-making model pre-constructed based on a Markov decision process. The routing, routing path, and multicast routing path decision-making method and electronic device of this application can determine both real-time performance and accuracy, thereby effectively improving data transmission efficiency.

Description

Technical Field

[0001] This application relates to the field of satellite communication technology, and in particular to a routing, routing path, multicast routing path decision method and electronic device. Background Technology

[0002] Current satellite network routing decisions typically involve calculating a complete optimal path after obtaining global satellite network environment information, and then transmitting data from the source node to the destination node according to the calculated optimal path. This decision-making method has a certain lag in practical applications. The status of each link in the satellite network changes in real time. The link status at the time of data transmission may have changed compared to the time of decision-making. Some links on the path may be congested, or some previously unselected links may have become unobstructed. These changes mean that the path at the time of data transmission is no longer the optimal path, thus affecting the data transmission efficiency of the satellite network. Summary of the Invention

[0003] In view of this, the purpose of this application is to propose a routing, routing path, multicast routing path decision method and electronic device to improve the data transmission efficiency of satellite networks.

[0004] To achieve the above objectives, this application provides a routing decision method, comprising:

[0005] Acquire the first information of the source node satellite and the second information of the destination node satellite in the routing path decision task;

[0006] Identify the satellite to be decided and use its neighboring node satellites as candidate routing node satellites;

[0007] Obtain the real-time traffic load of the candidate routing node satellites, as well as the predicted traffic load within a preset time step in the future;

[0008] Based on the real-time traffic load, the predicted traffic load, the first information, and the second information, the next routing node satellite is determined from the candidate routing node satellites using a routing decision model pre-built based on a Markov decision process.

[0009] Further, obtaining the predicted traffic load of the candidate routing node satellite within a future preset time step includes:

[0010] Obtain network topology information of the satellite network and link status information of each node satellite;

[0011] Based on the link status information of each node satellite and the network topology information, traffic prediction is performed using a pre-built traffic prediction model to obtain the predicted traffic load of each node satellite.

[0012] Furthermore, the traffic prediction model includes a graph convolutional neural network, a gated recurrent neural network, and a fully connected layer; the step of predicting traffic based on the link state information of each node satellite and the network topology information, using a pre-built traffic prediction model to obtain the predicted traffic load of each node satellite, includes:

[0013] Based on the link status information of each node satellite and the network topology information, spatial features are extracted using the graph convolutional neural network.

[0014] Based on the spatial features, the gated recurrent neural network is used to extract the temporal features, and the gated recurrent neural network is used to fuse the spatial features and the temporal features to obtain the spatiotemporal features.

[0015] The weights of the spatiotemporal features are calculated using an attention mechanism, and the predicted traffic load of each node satellite is output through the fully connected layer based on the spatiotemporal features and their corresponding weights.

[0016] Furthermore, the routing decision model is constructed through the following steps:

[0017] Define the agent, state space, action space, and reward policy of the Markov decision process to obtain the initial routing decision model;

[0018] A reinforcement learning network is constructed based on the dual-deep Q-learning algorithm;

[0019] The initial routing decision model is trained using the reinforcement learning network until a preset condition is met, thus obtaining the routing decision model.

[0020] The state space consists of the real-time traffic load, the predicted traffic load, the first information, and the second information.

[0021] The action space is the set of all possible actions obtained by the agent based on the state space;

[0022] The reward strategy is determined by the link transmission cost and link status of the satellite network.

[0023] Furthermore, the reinforcement learning network includes a value function network and a target network;

[0024] The step of training the initial routing decision model using the reinforcement learning network includes:

[0025] Based on the current state of the agent, the value function Q value corresponding to all possible actions to be executed in the current state is calculated in the value function network;

[0026] Based on the value function Q value corresponding to all possible actions to be executed in the current state, a greedy strategy is used to determine the first target action that the agent plans to execute in the current state, and the value function Q value corresponding to the first target action is recorded.

[0027] Instruct the agent to perform the first target action to obtain the state at the next moment;

[0028] Based on the state at the next moment, the value function Q value corresponding to all possible actions in the state at the next moment is calculated in the value function network, and the action corresponding to the maximum value of the value function Q value among all possible actions in the state at the next moment is taken as the second target action.

[0029] Based on the state at the next moment and the second target action, the target Q value corresponding to the second target action is calculated in the target network;

[0030] The value function value is calculated based on the target Q value;

[0031] Using the value function value as the actual value and the value function Q value corresponding to the first target action as the predicted value, backpropagation of the error is performed to complete one update of the network parameters of the value function network.

[0032] Specifically, at predetermined intervals, the network parameters of the target network are adjusted to be the same as those of the value function network.

[0033] Furthermore, the link status information includes link connectivity, bandwidth capacity, latency, and packet loss rate;

[0034] The neighboring node satellites of the satellite to be decided include adjacent satellites in the orbital plane of the satellite to be decided and satellites with the same number as the satellite to be decided in the adjacent orbital plane of the satellite to be decided.

[0035] Based on the same inventive concept, this application also provides a routing path decision method utilizing a routing decision method, comprising:

[0036] The source node satellite and the destination node satellite are determined;

[0037] The source node satellite is used as the satellite to be decided. The next routing node satellite of the satellite to be decided is determined using the routing decision method. It is also determined whether the next routing node satellite of the satellite to be decided is the destination node satellite.

[0038] In response to determining that the next routing node satellite of the satellite to be decided is not the destination node satellite, the next routing node is taken as a new satellite to be decided, until the next routing node determined by the routing decision method is the destination node satellite, thus obtaining a routing path including the source node satellite, all routing node satellites and the destination node satellite.

[0039] Based on the same inventive concept, this application also provides a multicast routing path decision method utilizing routing decision methods. The satellite network includes multiple satellite regions, each satellite region includes multiple satellites, and each satellite region includes a master satellite node. The multicast routing path includes a first multicast routing path and a second multicast routing path. The method includes:

[0040] The source node satellite and multiple destination node satellites of the routing path decision task are determined, wherein the multiple destination node satellites include a first destination node satellite belonging to the same satellite region as the source node satellite and / or a second destination node satellite not belonging to the same satellite region as the source node satellite. The primary satellite node of the satellite region to which the source node satellite belongs is the first primary satellite node, and the primary satellite node of the satellite region to which the second destination node satellite belongs is the second primary satellite node.

[0041] In response to determining that the endpoint satellite is the first endpoint satellite and the source satellite is not the first master satellite, the source satellite is designated as the satellite to be decided, the routing decision method is used to determine the next routing node satellite of the satellite to be decided, and it is determined whether the next routing node satellite of the satellite to be decided is the first master satellite.

[0042] In response to determining that the next routing node satellite of the satellite to be decided is not the first primary satellite node, the next routing node is taken as a new satellite to be decided, until the next routing node determined by the routing decision method is the first primary satellite node, thus obtaining a first routing path including the source node satellite, all routing node satellites and the first primary satellite node.

[0043] The first primary satellite node is used as the satellite to be decided. The routing decision method is used to determine the next routing node satellite of the first primary satellite node, and to determine whether the next routing node satellite of the first primary satellite node is the first destination node satellite.

[0044] In response to determining that the next routing node satellite of the first primary satellite node is not the first destination satellite, the next routing node is taken as a new satellite to be decided, until the next routing node determined by the routing decision method is the first destination satellite, thus obtaining a second routing path including the first primary satellite node, all routing node satellites and the first destination satellite.

[0045] The first multicast routing path, consisting of the first routing path and the second routing path, is the first multicast routing path from the source node satellite to the first destination node satellite; and / or,

[0046] In response to determining that the endpoint satellite is the second endpoint satellite, the source satellite is not the first primary satellite, and the second endpoint satellite is not the second primary satellite, the source satellite is designated as the satellite to be decided, and the first routing path is obtained.

[0047] The first primary satellite node is used as the satellite to be decided. The routing decision method is used to determine the next routing node satellite of the first primary satellite node, and to determine whether the next routing node satellite of the first primary satellite node is the second primary satellite node.

[0048] In response to determining that the next routing node satellite of the first primary satellite node is not the second primary satellite node, the next routing node is taken as a new satellite to be decided, until the next routing node determined by the routing decision method is the second primary satellite node, thus obtaining a third routing path including the first primary satellite node, all routing node satellites and the second primary satellite node.

[0049] The second primary satellite node is used as the satellite to be decided. The routing decision method is used to determine the next routing node satellite of the second primary satellite node, and to determine whether the next routing node satellite of the second primary satellite node is the second destination node satellite.

[0050] In response to determining that the next routing node satellite of the second primary satellite node is not the second destination satellite, the next routing node is taken as a new satellite to be decided, until the next routing node determined by the routing decision method is the second destination satellite, thus obtaining a fourth routing path including the second primary satellite node, all routing node satellites and the second destination satellite.

[0051] The second multicast routing path, consisting of the first routing path, the third routing path, and the fourth routing path, runs from the source node satellite to the second destination node satellite.

[0052] Furthermore, it also includes:

[0053] Determine the data to be transmitted, and transmit the data to be transmitted from the source node satellite to the first master satellite node according to the first routing path;

[0054] The first primary satellite node performs a first copy operation on the data to be transmitted, and distributes the data to be transmitted after the first copy operation to each of the first destination satellites and the second primary satellites according to the second routing path and the third routing path, wherein the number of data to be transmitted after the first copy operation is equal to the sum of the number of the first destination satellites and the second primary satellites;

[0055] The second primary satellite node performs a second copy operation on the data to be transmitted, and distributes the data to be transmitted after the second copy operation to each of the second destination satellites according to the fourth routing path, wherein the number of data to be transmitted after the second copy operation is equal to the number of the second destination satellites.

[0056] Based on the same inventive concept, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the program, it implements a routing decision method and / or a routing path decision method and / or a multicast routing path decision method.

[0057] As can be seen from the above, the routing, routing path, and multicast routing path decision-making method and electronic device provided in this application replace the process of determining the complete path in one go in the routing path decision-making task with single-step decision-making by the routing node satellite. That is, the decision is made by the routing node satellite, and only the next routing node satellite is decided. In the single-step decision-making process, the routing decision model needs real-time traffic load when performing decision calculations. Therefore, when the traffic load of the satellite network changes, the routing decision model can obtain the change in a timely manner and make a decision to determine the next routing node satellite based on the new information. This ensures that the routing node satellite determined each time is the optimal link, effectively improving the lag problem of determining the complete path in one go, and effectively ensuring timeliness, thereby effectively improving data transmission efficiency. At the same time, the routing decision model of this application is built based on the Markov decision process. When solving the routing decision process, it can obtain feedback through interaction with the environment, thereby optimizing the decision. It does not require obtaining global environmental information of the satellite network, and can be applied to scenarios where the satellite network environment information is incomplete or unknown, thus having a wider range of applications. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a schematic diagram of the routing decision method according to an embodiment of this application;

[0060] Figure 2 This is a flowchart illustrating the graph convolutional neural network and gated recurrent neural network in an embodiment of this application.

[0061] Figure 3 This is a flowchart illustrating the multicast routing path decision method according to an embodiment of this application.

[0062] Figure 4 This is a schematic diagram of a routing decision device according to an embodiment of this application;

[0063] Figure 5 This is a schematic diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0065] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0066] Satellite networks are microwave communication systems that use artificial Earth communication satellites as relay stations. They represent the development and extension of terrestrial microwave relay communication into space. Through the expansion of space networks, integrated space-ground information networks now possess large-scale coverage and abundant spectrum resources. They are capable of accurately acquiring, rapidly processing, and efficiently transmitting massive amounts of business information. Combining the advantages of satellite and terrestrial networks, they hold the promise of providing global broadband access and realizing ubiquitous network services.

[0067] Existing network communication modes are generally classified into unicast, multicast, and broadcast. Unicast refers to data transmission to a specific host, with information reception and transmission occurring only between two nodes. Multicast, also known as groupcast, refers to sending data to a specific group of hosts (multicast group), with multiple nodes receiving the information. Broadcast can be seen as a special type of multicast, referring to sending data to all hosts on a network. Multicast transmission technology supports applications involving multiple parties, enabling data distribution to be delivered cost-effectively. Advanced multicast routing technology can reduce the large amount of traffic in the network. Satellite multicast routing is the technological foundation for data distribution and information interconnection in satellite networks.

[0068] Regardless of whether multicast, unicast, or broadcast transmission technologies are used, existing satellite network routing decisions are typically made after obtaining global satellite network environment information, and a complete optimal path is calculated in one go. Data is then transmitted from the source node to the destination node according to this calculated optimal path. While traditional optimization-based satellite routing algorithms can obtain relatively accurate solutions after large-scale computation, the following problems still exist:

[0069] (1) The prerequisite for decision-making is that all accurate environmental information of the satellite network must be known. When environmental information is obtained incorrectly or missing, it will seriously affect the accuracy of decision-making. However, it is actually very difficult to obtain real and accurate environmental scene information.

[0070] (2) It has a certain lag in practical application. Specifically, the status of each link in the satellite network changes in real time. The status of the link at the time of data transmission may have changed from that at the time of decision-making. Some links on the path may be congested, and some unselected links may change from congested to smooth. As a result, the path at the time of data transmission is no longer the optimal path, thus affecting the data transmission efficiency of the satellite network.

[0071] Therefore, improving the timeliness and accuracy of routing decision algorithms in satellite networks is an important technical means to ensure the efficiency of satellite data transmission.

[0072] To address the aforementioned problems, this application provides a routing decision method, such as... Figure 1 As shown, the routing path decision-making method used in satellite networks can effectively improve the timeliness and accuracy of decision-making and effectively improve data transmission efficiency. The specific methods include:

[0073] S11: Obtain the first information of the source node satellite and the second information of the destination node satellite in the routing path decision task;

[0074] S12: Determine the satellite to be decided and use the neighboring node satellites of the satellite to be decided as candidate routing node satellites;

[0075] S13: Obtain the real-time traffic load of the candidate routing node satellites, and the predicted traffic load within a preset time step in the future;

[0076] S14: Based on the real-time traffic load, the predicted traffic load, the first information, and the second information, determine the next routing node satellite from the candidate routing node satellites using a routing decision model pre-built based on a Markov decision process.

[0077] In this application, the process of determining the complete path in one go in the routing path decision-making task is replaced by single-step decision-making by the routing node satellite. That is, the routing node satellite makes the decision, and only the next routing node satellite is determined. In the single-step decision-making process, the routing decision model needs real-time traffic load to perform decision calculations. Therefore, when the traffic load of the satellite network changes, the routing decision model can promptly obtain the change and make a decision to determine the next routing node satellite based on the actual traffic load. This ensures that the routing node satellite determined each time is the optimal link, effectively improving the lag problem of determining the complete path in one go, and effectively ensuring timeliness, thereby effectively improving data transmission efficiency. At the same time, the routing decision model of this application is built based on the Markov decision process. When solving the routing decision, it obtains feedback through interaction with the environment, thereby optimizing the decision. It does not require obtaining global environmental information of the satellite network, and can be applied to scenarios where the satellite network environment information is incomplete or unknown, thus having a wider range of applications.

[0078] In some embodiments, the preset time step can be 30 seconds to 15 minutes, specifically, it can be 30 seconds, 1 minute, 3 minutes, 5 minutes, 8 minutes, 10 minutes, 15 minutes, etc.; the preset time step can also be set to a longer one, even more than 15 minutes, or a shorter one, even less than 30 seconds, depending on the actual situation. Users can adjust it according to the actual application scenario and needs, and there are no specific restrictions.

[0079] In some embodiments, the first information is location information indicating the location of the source node satellite; the second information is location information indicating the location of the destination node satellite. Specifically, it can be the satellite number of the source node satellite or the destination node satellite, or other information that can indicate its location, without any specific limitation.

[0080] In some embodiments, the satellite network includes the Iridium constellation network. This application can be applied to routing decisions within the Iridium constellation network. The Iridium constellation network comprises six orbital planes with a fixed orbital inclination of 90°. Eleven satellites are deployed at equal intervals in each orbital plane, and the entire constellation consists of 66 satellites at an orbital altitude of approximately 780 km.

[0081] In some embodiments, obtaining the predicted traffic load of the candidate routing node satellite within a future preset time step includes:

[0082] S131: Obtain network topology information of the satellite network and link status information of each node satellite;

[0083] S132: Based on the link status information of each node satellite and the network topology information, traffic prediction is performed using a pre-built traffic prediction model to obtain the predicted traffic load of each node satellite.

[0084] In a satellite network, each node satellite connects with two adjacent node satellites in the same orbital plane to form an intra-orbit inter-satellite link, and connects with two node satellites of the same number in adjacent orbital planes to form an inter-orbit inter-satellite link. During the periodic operation of node satellites, when a node satellite enters a high-latitude polar region, such as the Antarctic or Arctic region, the inter-orbit inter-satellite link will experience a temporary disconnection. The connection will be restored after leaving the high-latitude polar region. Current traffic prediction technologies can only detect the link disconnection after a node satellite actually enters a high-latitude polar region, and cannot predict in advance whether a satellite is about to enter or leave such a region.

[0085] In this application, traffic prediction is performed by integrating network topology information with the link status information of each node satellite. Network topology information is used to obtain information on node satellites about to enter or leave high-latitude polar regions. This allows the traffic prediction model to capture this information during traffic prediction, ensuring that the predicted traffic load of each node satellite within a preset time step more closely reflects the actual situation. Therefore, during routing decisions, node satellites about to enter high-latitude polar regions can be effectively avoided, guaranteeing normal data transmission. Optionally, traffic prediction can be performed using a pre-built traffic prediction model according to a preset period, which can be consistent with a preset time step.

[0086] In some embodiments, the traffic prediction model includes a graph convolutional neural network, a gated recurrent neural network, and a fully connected layer; step S132 includes:

[0087] S1321: Based on the link status information of each node satellite and the network topology information, spatial features are extracted using the graph convolutional neural network;

[0088] S1322: Based on the spatial features, the time features are extracted using the gated recurrent neural network, and the spatial features and the time features are fused using the gated recurrent neural network to obtain spatiotemporal features;

[0089] S1323: The attention mechanism is used to calculate the weights of the spatiotemporal features, and the predicted traffic load of each node satellite is output through the fully connected layer based on the spatiotemporal features and their corresponding weights.

[0090] In this application, a Graph Convolutional Neural Network (GCN) is used to fully extract the spatial features between node satellites. These spatial features are then input into a Gated Recurrent Neural Network (GRU). Several hidden layers within the GRU extract features from the input spatial features to obtain temporal features. Finally, the spatial and temporal features are fused to obtain spatiotemporal features. An attention mechanism is used to calculate the weights of the spatiotemporal features, filtering out irrelevant information and emphasizing the varying importance of historical information. This results in a more accurate predicted traffic load output by the fully connected layer.

[0091] In some embodiments, the traffic prediction model is a multi-layered structure connected sequentially, each layer including a graph convolutional neural network and a gated recurrent neural network, with the last layer being a fully connected layer. The graph convolutional neural network and gated recurrent neural network in each layer are represented as GCN-GRU. The layer settings and workflow of the graph convolutional neural network and gated recurrent neural network can be configured as follows: Figure 2 As shown, adjustments can be made according to the actual situation and specific application scenarios, without any restrictions.

[0092] It should be noted that, Figure 2 In this context, t represents time, and Y... t-2 Y t-1 Y t Indicates the spatiotemporal characteristics at the corresponding moment; h t-2 h t-1 h t Indicates the temporal characteristics at the corresponding moment; X t-2 X t-1 X t Represents the spatial characteristics at the corresponding time point, X′ t This represents the new spatial features obtained through feature extraction using a graph convolutional neural network (GCN), u t r t c t The parameters of the gated recurrent recurrent neural network (GRU) are input into the GRU for feature extraction, yielding a new temporal feature h. t And the spatiotemporal features Y are obtained by fusing the temporal and spatial features. t .

[0093] In some embodiments, the network topology information is obtained through the following steps:

[0094] Obtain the spatial topology of satellite networks;

[0095] The spatial topology is divided into multiple static topologies within preset time intervals, and all static topologies are combined to obtain the network topology information.

[0096] The spatial topology of satellite networks is actually a dynamic topology. To improve computational efficiency, the spatial topology of satellite networks is converted into a static topology, and then the static topology is input into the traffic prediction model for calculation. This makes the calculation simpler and more efficient.

[0097] In some embodiments, the routing decision model is constructed through the following steps:

[0098] S21: Define the agent, state space, action space, and reward policy of the Markov decision process to obtain the initial routing decision model;

[0099] S22: Construct a reinforcement learning network based on the dual-deep Q-learning algorithm;

[0100] S23: Train the initial routing decision model using the reinforcement learning network until the preset conditions are met to obtain the routing decision model;

[0101] The state space consists of the real-time traffic load, the predicted traffic load, the first information, and the second information.

[0102] The action space is the set of all possible actions obtained by the agent based on the state space;

[0103] The reward strategy is determined by the link transmission cost and link status of the satellite network.

[0104] The initial routing decision model is trained using a reinforcement learning network. An agent makes dynamic decisions, and the quality of these decisions is evaluated based on environmental feedback. During the learning process, the decision strategy is progressively adjusted until optimal performance is achieved. Solving the routing problem using reinforcement learning algorithms allows for feedback-based optimization and the attainment of the best routing solution even without prior environmental information, solely through interaction with the environment.

[0105] In some embodiments, the reward function is defined as:

[0106]

[0107] r d The positive reward for reaching the terminal node satellite is P, where P is the link transmission cost and ω1 and ω2 are different weighting coefficients.

[0108] In some embodiments, the reinforcement learning network includes a value function network and a target network;

[0109] The step of training the initial routing decision model using the reinforcement learning network includes:

[0110] Based on the current state of the agent, the value function Q value corresponding to all possible actions to be executed in the current state is calculated in the value function network;

[0111] Based on the value function Q value corresponding to all possible actions to be executed in the current state, a greedy strategy is used to determine the first target action that the agent plans to execute in the current state, and the value function Q value corresponding to the first target action is recorded.

[0112] Instruct the agent to perform the first target action to obtain the state at the next moment;

[0113] Based on the state at the next moment, the value function Q value corresponding to all possible actions in the state at the next moment is calculated in the value function network, and the action corresponding to the maximum value of the value function Q value among all possible actions in the state at the next moment is taken as the second target action.

[0114] Based on the state at the next moment and the second target action, the target Q value corresponding to the second target action is calculated in the target network;

[0115] The value function value is calculated based on the target Q value;

[0116] Using the value function value as the actual value and the value function Q value corresponding to the first target action as the predicted value, backpropagation of the error is performed to complete one update of the network parameters of the value function network.

[0117] Specifically, at predetermined intervals, the network parameters of the target network are adjusted to be the same as those of the value function network.

[0118] During the training of the initial routing decision model, a greedy strategy (ε-greedy) is used in the value function network for action selection to explore new network environments. In the target network, a Q-value evaluation is given based on historical experience, i.e., the target Q-value is calculated. This mitigates the over-dependency that can occur when using only one network for simultaneous exploration and evaluation, thus affecting model convergence and effectively improving training efficiency and model accuracy. Simultaneously, the network parameters of the target network are adjusted to be identical to those of the value function network at predetermined intervals, mitigating the over-evaluation problem of the target network and further improving model training efficiency and accuracy.

[0119] In some embodiments, the value function value is calculated using the following formula:

[0120] y = r + γQ′ (s ′ ,argmaxQ(s ′ ,,), ′ )

[0121] Where r is the reward value, γ is the decay factor, and ω is the network parameter of the value function network. ′ Here are the network parameters for the objective function network, where 'a' represents the action, 'Q' represents the value function Q-value, and Q' represents the network parameters for the objective function network. ′ For the target Q value, s ′ For the state at the next moment, argmaxQ(s) ′ ,,) represents the state s at the next time step. ′ The action corresponding to the maximum value of the value function Q among all possible actions a (i.e., the second target action).

[0122] In some embodiments, a loss function is calculated using the value function value and the value function Q value, and error backpropagation is performed using the loss function to complete one update of the network parameters of the value function network; the loss function is calculated using the following formula:

[0123] Loss=E[yQ(s,a,ω) 2 ]

[0124] Where s represents the current state.

[0125] In some embodiments, the link state information includes link connectivity, bandwidth capacity, latency, and packet loss rate;

[0126] The neighboring node satellites of the satellite to be decided include adjacent satellites in the orbital plane of the satellite to be decided and satellites with the same number as the satellite to be decided in the adjacent orbital plane of the satellite to be decided.

[0127] In some embodiments, the traffic prediction model is a pre-trained model. Specifically, the traffic prediction model can be trained using historical traffic data from different historical moments of satellites at various nodes in the satellite network.

[0128] Based on the same inventive concept, corresponding to any of the above embodiments, this application also provides a routing path decision method utilizing a routing decision method, including:

[0129] The source node satellite and the destination node satellite are determined;

[0130] The source node satellite is used as the satellite to be decided. The next routing node satellite of the satellite to be decided is determined using the routing decision method. It is also determined whether the next routing node satellite of the satellite to be decided is the destination node satellite.

[0131] In response to determining that the next routing node satellite of the satellite to be decided is not the destination node satellite, the next routing node is taken as a new satellite to be decided, until the next routing node determined by the routing decision method is the destination node satellite, thus obtaining a routing path including the source node satellite, all routing node satellites and the destination node satellite.

[0132] In existing routing path decision-making processes, a complete routing path is typically determined at once, and then data transmission is carried out based on the determined routing path. Since the link status of each satellite node changes in real time, the routing paths obtained by existing decision-making methods are usually lagging. For example, when data transmission is carried out along the routing path, if a link located later in the path becomes congested, the routing path cannot be changed, and transmission can only proceed along the original path through the congested link. Obviously, this data transmission efficiency is low.

[0133] In this application, the process of determining a complete path in one go is transformed into a single-step decision-making process by routing node satellites. Specifically, the source node satellite uses a routing decision-making method to determine the next routing node satellite, and so on, until the destination node satellite is reached. In this process, each decision made using the routing decision-making method is based on the current real-time traffic load. Therefore, when the link status changes, the routing decision-making model can promptly detect this change, ensuring that each determined routing node satellite represents the optimal link. This results in the optimal routing path composed of all routing node satellites, effectively mitigating the lag problem of determining a complete path in one go and ensuring timely delivery, thereby significantly improving data transmission efficiency. Furthermore, the routing decision-making model in this application is based on a Markov decision process. During the calculation of the routing decision process, feedback can be obtained through interaction with the environment, thus optimizing the decision. It does not require access to the global environment information of the satellite network, making it applicable to scenarios where satellite network environment information is incomplete or unknown, thus broadening its application scope.

[0134] Based on the same inventive concept, corresponding to any of the above embodiments, this application also provides a multicast routing path decision method utilizing a routing decision method. The satellite network includes multiple satellite regions, each satellite region includes multiple satellites, and each satellite region includes a master satellite node. The multicast routing path includes a first multicast routing path and a second multicast routing path. The method includes:

[0135] The source node satellite and multiple destination node satellites of the routing path decision task are determined, wherein the multiple destination node satellites include a first destination node satellite belonging to the same satellite region as the source node satellite and / or a second destination node satellite not belonging to the same satellite region as the source node satellite. The primary satellite node of the satellite region to which the source node satellite belongs is the first primary satellite node, and the primary satellite node of the satellite region to which the second destination node satellite belongs is the second primary satellite node.

[0136] In response to determining that the endpoint satellite is the first endpoint satellite and the source satellite is not the first master satellite, the source satellite is designated as the satellite to be decided, the routing decision method is used to determine the next routing node satellite of the satellite to be decided, and it is determined whether the next routing node satellite of the satellite to be decided is the first master satellite.

[0137] In response to determining that the next routing node satellite of the satellite to be decided is not the first primary satellite node, the next routing node is taken as a new satellite to be decided, until the next routing node determined by the routing decision method is the first primary satellite node, thus obtaining a first routing path including the source node satellite, all routing node satellites and the first primary satellite node.

[0138] The first primary satellite node is used as the satellite to be decided. The routing decision method is used to determine the next routing node satellite of the first primary satellite node, and to determine whether the next routing node satellite of the first primary satellite node is the first destination node satellite.

[0139] In response to determining that the next routing node satellite of the first primary satellite node is not the first destination satellite, the next routing node is taken as a new satellite to be decided, until the next routing node determined by the routing decision method is the first destination satellite, thus obtaining a second routing path including the first primary satellite node, all routing node satellites and the first destination satellite.

[0140] The first multicast routing path, consisting of the first routing path and the second routing path, is the first multicast routing path from the source node satellite to the first destination node satellite; and / or,

[0141] In response to determining that the endpoint satellite is the second endpoint satellite, the source satellite is not the first primary satellite, and the second endpoint satellite is not the second primary satellite, the source satellite is designated as the satellite to be decided, and the first routing path is obtained.

[0142] The first primary satellite node is used as the satellite to be decided. The routing decision method is used to determine the next routing node satellite of the first primary satellite node, and to determine whether the next routing node satellite of the first primary satellite node is the second primary satellite node.

[0143] In response to determining that the next routing node satellite of the first primary satellite node is not the second primary satellite node, the next routing node is taken as a new satellite to be decided, until the next routing node determined by the routing decision method is the second primary satellite node, thus obtaining a third routing path including the first primary satellite node, all routing node satellites and the second primary satellite node.

[0144] The second primary satellite node is used as the satellite to be decided. The routing decision method is used to determine the next routing node satellite of the second primary satellite node, and to determine whether the next routing node satellite of the second primary satellite node is the second destination node satellite.

[0145] In response to determining that the next routing node satellite of the second primary satellite node is not the second destination satellite, the next routing node is taken as a new satellite to be decided, until the next routing node determined by the routing decision method is the second destination satellite, thus obtaining a fourth routing path including the second primary satellite node, all routing node satellites and the second destination satellite.

[0146] The second multicast routing path, consisting of the first routing path, the third routing path, and the fourth routing path, runs from the source node satellite to the second destination node satellite.

[0147] In some embodiments, it also includes:

[0148] Determine the data to be transmitted, and transmit the data to be transmitted from the source node satellite to the first master satellite node according to the first routing path;

[0149] The first primary satellite node performs a first copy operation on the data to be transmitted, and distributes the data to be transmitted after the first copy operation to each of the first destination satellites and the second primary satellites according to the second routing path and the third routing path, wherein the number of data to be transmitted after the first copy operation is equal to the sum of the number of the first destination satellites and the second primary satellites;

[0150] The second primary satellite node performs a second copy operation on the data to be transmitted and distributes the data to be transmitted after the second copy operation to each of the second destination satellites according to the fourth routing path, wherein the number of the data to be transmitted after the second copy operation is equal to the number of the second destination satellites.

[0151] In this application, the satellite network is pre-divided into multiple satellite regions. Specifically, this division can be achieved by combining the reverse seams and polar regions of the satellite network. For example, the satellite network can be divided once based on the reverse seams, and then further divided based on the North and South Poles to obtain multiple satellite regions. Each satellite region includes multiple satellites. The number of satellites in each satellite region can be equal or unequal. Then, a master satellite node is determined in each satellite region. Specifically, the node satellite located at the center of the satellite region can be determined as the master satellite node.

[0152] Based on the regional division of the satellite network, multiple endpoint satellites are divided into first endpoint satellites and second endpoint satellites according to their locations, resulting in first and second multicast routing paths. The second endpoint satellites are not located in the same satellite region as the source satellites, indicating that they are relatively far from the source satellites. Traditional routing paths typically run directly from the source satellite to the endpoint satellite; however, this method requires individual decision-making for each endpoint satellite when dealing with multicast routing path decisions, leading to tediousness and a large decision-making workload. In this application, for multiple second endpoint satellites that are far from the source satellite, data is uniformly transmitted to the corresponding second primary satellite node, and then transmitted from the second primary satellite node to multiple second endpoint satellites within the same satellite region. This effectively reduces the number of tasks involved in long-distance path decision-making and significantly improves data transmission efficiency.

[0153] For example, suppose the source node satellite is satellite number 4, the first primary satellite node is satellite number 6, and the second endpoint node satellites include satellite number 43, satellite number 45, satellite number 55, and satellite number 57. Among them, satellite number 43 and satellite number 45 are located in the same satellite region, and their corresponding second primary satellite node is satellite number 44; satellite number 55 and satellite number 57 are located in the same satellite region, and their corresponding second primary satellite node is satellite number 56. In the multicast routing path decision-making process for the second endpoint satellite, the routing decision method is first used to reach the 6th satellite (i.e., the first primary satellite node) starting from the 4th satellite, obtaining the first routing path. Then, the first replication operation is performed through the 6th satellite, copying the data to be transmitted into two copies, and transmitting the data to the 44th and 56th satellites (i.e., the two second primary satellite nodes) respectively using the routing decision method. Then, the second replication operation is performed through the 44th satellite, copying the data to be transmitted into two copies, and transmitting the data to the 43rd and 45th satellites respectively using the routing decision method. The second replication operation is performed through the 56th satellite, copying the data to be transmitted into two copies, and transmitting the data to the 55th and 57th satellites respectively using the routing decision method. In this process, traditional path decision-making requires separate decisions for each of the four secondary endpoint satellites (i.e., satellites 43, 44, 55, and 57), resulting in four decision-making tasks, all of which are long-distance decisions. This process is cumbersome and computationally intensive. In this application, the data first reaches the secondary primary satellite nodes (i.e., satellites 44 and 56), reducing the number of long-distance decision-making tasks from four to two. Then, each of the two secondary primary satellite nodes uses its own routing decision method to distribute the data to be transmitted to the secondary endpoint satellites within its respective satellite region. Since the secondary primary satellite nodes and their corresponding secondary endpoint satellites belong to the same satellite region, this constitutes a short-distance decision. Clearly, this application reduces the number of cumbersome and computationally intensive long-distance decision-making tasks, effectively reducing the computational load in the multicast routing decision-making process, resulting in more efficient computation and higher data transmission efficiency.

[0154] The specific process of using the multicast routing path decision method of this application for multicast data transmission can provide... Figure 3 The process shown can also be designed into other processes according to actual application scenarios and specific needs. For example, it can first determine whether the source node satellite is the first primary satellite node. After reaching the first primary satellite node, it can further distinguish between the first destination node satellite and the second destination node satellite. Then, based on the different destination node satellites, the corresponding second, third, or fourth routing paths can be obtained. There are no specific restrictions.

[0155] The following is based on Figure 3Taking the process shown as an example, this paper specifically illustrates the process of multicast data transmission using the multicast routing path decision method of this application.

[0156] like Figure 3 As shown, the source node satellite and multiple destination node satellites for the path decision task are first determined. Then, it is determined whether each destination node satellite belongs to the same satellite region as the source node satellite. If they belong to the same satellite region, they are determined as the first destination node satellite. If they do not belong to the same satellite region, they are determined as the second destination node satellite.

[0157] For the first destination satellite, it is further determined whether the source satellite is the first primary satellite. If so, the first primary satellite performs the first replication operation on the data to be transmitted. If not, a first routing path is first obtained using a routing decision method. Once the data arrives at the first primary satellite, the first primary satellite then performs the first replication operation on the data to be transmitted. Then, at the first primary satellite, a second routing path is obtained using a routing decision method. Once the data arrives at the first destination satellite, the data transmission of the first destination satellite is complete.

[0158] For the second destination satellite, the process first determines whether the source satellite is the first primary satellite. If so, the first primary satellite directly performs the first replication operation. If not, the data first reaches the first primary satellite via the first routing path, and then the first primary satellite performs the first replication operation. Then, the routing decision method is used to obtain the third routing path. Once the data reaches the second primary satellite, it is further determined whether the second destination satellite is the second primary satellite. If so, the data transmission is completed. If not, the second replication operation is performed at the second primary satellite. Then, the routing decision method is used to obtain the fourth routing path to reach the second destination satellite, completing the data transmission for the second destination satellite.

[0159] Once the data transmission to all endpoint satellites is completed, the multicast transmission task is finished.

[0160] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.

[0161] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0162] Based on the same inventive concept, corresponding to any of the above embodiments, this application also provides a satellite network control system implementation for realizing satellite network routing decisions, specifically including: a satellite control station and a ground control station;

[0163] The ground control station includes a traffic prediction server and a ground information receiver / transmitter connected to the traffic prediction server.

[0164] The satellite control station includes multiple link status detectors, multiple satellite information receivers and transmitters, and multiple routing decision servers; each node satellite is equipped with one of the link status detectors, one of the satellite information receivers and transmitters, and one of the routing decision servers, and the routing decision server of each node satellite is connected to the link status detector and the satellite information receiver and transmitter;

[0165] The traffic prediction server carries and runs a traffic prediction model. After obtaining the predicted traffic load of the node satellite, the traffic prediction server transmits the predicted traffic load of the node satellite to the ground information receiver and transmitter, and then the ground information receiver and transmitter transmits the information to the satellite information receiver and transmitter of the satellite control station of each node satellite.

[0166] The ground information receiver and transmitter is also used to receive network topology information of the satellite network and link status information of each node satellite sent by the satellite information receiver and transmitter, and send the information to the traffic prediction server. The traffic prediction server uses the information to perform traffic prediction and obtain the predicted traffic load of each node satellite.

[0167] After receiving the predicted traffic load, the satellite information receiver and transmitter of the satellite control station transmits the information to the corresponding routing decision server.

[0168] The link state detector is used to obtain real-time traffic load and send the real-time traffic load to the corresponding routing decision server;

[0169] When a user initiates a routing path decision task, the routing decision server obtains first information and second information, determines candidate routing node satellites, and runs the routing decision model based on the real-time traffic load, the predicted traffic load, the first information, and the second information to determine the next routing node satellite. Optionally, the multiple link state detectors, multiple satellite information receivers and transmitters, and multiple routing decision servers of the satellite control station can be set up on the satellite or on the ground. Users can configure them according to their actual application scenarios and needs, and there are no specific restrictions.

[0170] Based on this, the routing path decision method or multicast routing path decision method of this application can also be implemented using a satellite network control system.

[0171] In addition, other systems that can implement the routing decision method, routing path decision method, or multicast routing path decision method of this application can also be applied to this application, without any specific limitations.

[0172] Based on the same inventive concept, corresponding to any of the above embodiments, this application also provides a routing decision device.

[0173] refer to Figure 4 The routing decision-making device includes:

[0174] The first acquisition module 401 is used to acquire the first information of the source node satellite and the second information of the destination node satellite in the routing path decision task.

[0175] The first determining module 402 is used to determine the satellite to be decided and to take the neighboring node satellites of the satellite to be decided as candidate routing node satellites.

[0176] The second acquisition module 403 is used to acquire the real-time traffic load of the candidate routing node satellites, as well as the predicted traffic load within a preset time step in the future.

[0177] The routing decision module 404 is used to determine the next routing node satellite from the candidate routing node satellites based on the real-time traffic load, the predicted traffic load, the first information and the second information, using a routing decision model pre-built based on the Markov decision process.

[0178] In some embodiments, the second acquisition module 403 further includes:

[0179] The first acquisition unit is used to acquire network topology information of the satellite network and link status information of each node satellite;

[0180] The traffic prediction unit is used to predict the traffic load of each node satellite by using a pre-built traffic prediction model based on the link status information of each node satellite and the network topology information.

[0181] In some embodiments, the traffic prediction model includes a graph convolutional neural network, a gated recurrent neural network, and a fully connected layer; the traffic prediction unit further includes:

[0182] The first feature extraction element is used to extract spatial features based on the link status information of each node satellite and the network topology information using the graph convolutional neural network.

[0183] The second feature extraction element is used to extract temporal features based on the spatial features using the gated recurrent neural network, and to fuse the spatial features and the temporal features using the gated recurrent neural network to obtain spatiotemporal features.

[0184] The traffic prediction element is used to calculate the weights of the spatiotemporal features using an attention mechanism, and outputs the predicted traffic load of each node satellite through the fully connected layer based on the spatiotemporal features and their corresponding weights.

[0185] In some embodiments, the routing decision model is constructed through the following steps:

[0186] Define the agent, state space, action space, and reward policy of the Markov decision process to obtain the initial routing decision model;

[0187] A reinforcement learning network is constructed based on the dual-deep Q-learning algorithm;

[0188] The initial routing decision model is trained using the reinforcement learning network until a preset condition is met, thus obtaining the routing decision model.

[0189] The state space consists of the real-time traffic load, the predicted traffic load, the first information, and the second information.

[0190] The action space is the set of all possible actions obtained by the agent based on the state space;

[0191] The reward strategy is determined by the link transmission cost and link status of the satellite network.

[0192] In some embodiments, the reinforcement learning network includes a value function network and a target network;

[0193] The step of training the initial routing decision model using the reinforcement learning network includes:

[0194] Based on the current state of the agent, the value function Q value corresponding to all possible actions to be executed in the current state is calculated in the value function network;

[0195] Based on the value function Q value corresponding to all possible actions to be executed in the current state, a greedy strategy is used to determine the first target action that the agent plans to execute in the current state, and the value function Q value corresponding to the first target action is recorded.

[0196] Instruct the agent to perform the first target action to obtain the state at the next moment;

[0197] Based on the state at the next moment, the value function Q value corresponding to all possible actions in the state at the next moment is calculated in the value function network, and the action corresponding to the maximum value of the value function Q value among all possible actions in the state at the next moment is taken as the second target action.

[0198] Based on the state at the next moment and the second target action, the target Q value corresponding to the second target action is calculated in the target network;

[0199] The value function value is calculated based on the target Q value;

[0200] Using the value function value as the actual value and the value function Q value corresponding to the first target action as the predicted value, backpropagation of the error is performed to complete one update of the network parameters of the value function network.

[0201] Specifically, at predetermined intervals, the network parameters of the target network are adjusted to be the same as those of the value function network.

[0202] In some embodiments, the link state information includes link connectivity, bandwidth capacity, latency, and packet loss rate;

[0203] The neighboring node satellites of the satellite to be decided include adjacent satellites in the orbital plane of the satellite to be decided and satellites with the same number as the satellite to be decided in the adjacent orbital plane of the satellite to be decided.

[0204] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0205] The apparatus described above is used to implement a corresponding routing decision method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0206] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a routing path decision device, including:

[0207] The second determining module is used to determine the source node satellite and the destination node satellite;

[0208] The path decision module is used to take the source node satellite as the satellite to be decided, use the routing decision method to determine the next routing node satellite of the satellite to be decided, and determine whether the next routing node satellite of the satellite to be decided is the destination node satellite.

[0209] In response to determining that the next routing node satellite of the satellite to be decided is not the destination node satellite, the next routing node is taken as a new satellite to be decided, until the next routing node determined by the routing decision method is the destination node satellite, thus obtaining a routing path including the source node satellite, all routing node satellites and the destination node satellite.

[0210] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0211] The apparatus described above is used to implement a corresponding routing path decision method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0212] Based on the same inventive concept, corresponding to any of the above embodiments, this application also provides a multicast routing path decision-making device. The satellite network includes multiple satellite regions, each satellite region includes multiple satellites, and each satellite region includes a master satellite node. The multicast routing path includes a first multicast routing path and a second multicast routing path. The device includes:

[0213] The third determining module is used to determine the source node satellite and multiple destination node satellites of the routing path decision task. The multiple destination node satellites include a first destination node satellite belonging to the same satellite region as the source node satellite and / or a second destination node satellite not belonging to the same satellite region as the source node satellite. The primary satellite node of the satellite region to which the source node satellite belongs is the first primary satellite node, and the primary satellite node of the satellite region to which the second destination node satellite belongs is the second primary satellite node.

[0214] The first multicast routing path decision module is configured to, in response to determining that the destination node satellite is the first destination node satellite and the source node satellite is not the first master satellite node, take the source node satellite as the satellite to be decided, use the routing decision method to determine the next routing node satellite of the satellite to be decided, and determine whether the next routing node satellite of the satellite to be decided is the first master satellite node.

[0215] In response to determining that the next routing node satellite of the satellite to be decided is not the first primary satellite node, the next routing node is taken as a new satellite to be decided, until the next routing node determined by the routing decision method is the first primary satellite node, thus obtaining a first routing path including the source node satellite, all routing node satellites and the first primary satellite node.

[0216] The first primary satellite node is used as the satellite to be decided. The routing decision method is used to determine the next routing node satellite of the first primary satellite node, and to determine whether the next routing node satellite of the first primary satellite node is the first destination node satellite.

[0217] In response to determining that the next routing node satellite of the first primary satellite node is not the first destination satellite, the next routing node is taken as a new satellite to be decided, until the next routing node determined by the routing decision method is the first destination satellite, thus obtaining a second routing path including the first primary satellite node, all routing node satellites and the first destination satellite.

[0218] The first multicast routing path, consisting of the first routing path and the second routing path, is the first multicast routing path from the source node satellite to the first destination node satellite; and / or,

[0219] The second multicast routing path decision module is used to determine, in response to the determination that the destination node satellite is the second destination node satellite, the source node satellite is not the first master satellite node, and the second destination node satellite is not the second master satellite node, to use the source node satellite as the satellite to be decided, and to obtain the first routing path;

[0220] The first primary satellite node is used as the satellite to be decided. The routing decision method is used to determine the next routing node satellite of the first primary satellite node, and to determine whether the next routing node satellite of the first primary satellite node is the second primary satellite node.

[0221] In response to determining that the next routing node satellite of the first primary satellite node is not the second primary satellite node, the next routing node is taken as a new satellite to be decided, until the next routing node determined by the routing decision method is the second primary satellite node, thus obtaining a third routing path including the first primary satellite node, all routing node satellites and the second primary satellite node.

[0222] The second primary satellite node is used as the satellite to be decided. The routing decision method is used to determine the next routing node satellite of the second primary satellite node, and to determine whether the next routing node satellite of the second primary satellite node is the second destination node satellite.

[0223] In response to determining that the next routing node satellite of the second primary satellite node is not the second destination satellite, the next routing node is taken as a new satellite to be decided, until the next routing node determined by the routing decision method is the second destination satellite, thus obtaining a fourth routing path including the second primary satellite node, all routing node satellites and the second destination satellite.

[0224] The second multicast routing path, consisting of the first routing path, the third routing path, and the fourth routing path, runs from the source node satellite to the second destination node satellite.

[0225] In some embodiments, it also includes:

[0226] The fourth determining module is used to determine the data to be transmitted and transmit the data to be transmitted from the source node satellite to the first master satellite node according to the first routing path;

[0227] The first replication module is used to perform a first replication operation on the data to be transmitted through the first primary satellite node, and distribute the data to be transmitted after the first replication operation to each of the first destination satellite nodes and the second primary satellite node according to the second routing path and the third routing path, wherein the number of the data to be transmitted after the first replication operation is equal to the sum of the number of the first destination satellite nodes and the second primary satellite node.

[0228] The second replication module is used to perform a second replication operation on the data to be transmitted through the second primary satellite node, and distribute the data to be transmitted after the second replication operation to each of the second destination node satellites according to the fourth routing path, wherein the number of the data to be transmitted after the second replication operation is equal to the number of the second destination node satellites.

[0229] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0230] The apparatus described above is used to implement a corresponding multicast routing path decision method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0231] Based on the same inventive concept, corresponding to any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements a routing decision method and / or a routing path decision method and / or a multicast routing path decision method as described in any of the above embodiments.

[0232] Figure 5 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0233] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0234] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0235] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0236] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0237] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0238] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0239] The electronic devices described above are used to implement a routing decision method and / or a routing path decision method and / or a multicast routing path decision method in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0240] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute a routing decision method and / or a routing path decision method and / or a multicast routing path decision method as described in any of the above embodiments.

[0241] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0242] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute a routing decision method and / or a routing path decision method and / or a multicast routing path decision method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0243] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0244] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0245] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0246] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

Claims

1. A method of multicast routing path decision, characterized by, The satellite network includes multiple satellite regions, each satellite region includes multiple satellites, and each satellite region includes a master satellite node. The multicast routing path includes a first multicast routing path and a second multicast routing path. The method includes: The source node satellite and multiple destination node satellites for the route path decision task are determined. The multiple destination node satellites include a first destination node satellite belonging to the same satellite region as the source node satellite and / or a second destination node satellite not belonging to the same satellite region as the source node satellite. The primary satellite node of the satellite region to which the source node satellite belongs is the first primary satellite node, and the primary satellite node of the satellite region to which the second destination node satellite belongs is the second primary satellite node. In response to determining that the endpoint satellite is the first endpoint satellite and the source satellite is not the first master satellite, the source satellite is designated as the satellite to be decided, the next routing node satellite of the satellite to be decided is determined using the routing decision method, and it is determined whether the next routing node satellite of the satellite to be decided is the first master satellite. In response to determining that the next routing node satellite of the satellite to be decided is not the first primary satellite node, the next routing node is taken as a new satellite to be decided, until the next routing node determined by the routing decision method is the first primary satellite node, thus obtaining a first routing path including the source node satellite, all routing node satellites and the first primary satellite node. The first primary satellite node is used as the satellite to be decided. The routing decision method is used to determine the next routing node satellite of the first primary satellite node, and to determine whether the next routing node satellite of the first primary satellite node is the first destination node satellite. In response to determining that the next routing node satellite of the first primary satellite node is not the first destination satellite, the next routing node is taken as a new satellite to be decided, until the next routing node determined by the routing decision method is the first destination satellite, thus obtaining a second routing path including the first primary satellite node, all routing node satellites and the first destination satellite. The first multicast routing path, consisting of the first routing path and the second routing path, is the first multicast routing path from the source node satellite to the first destination node satellite; and / or, In response to determining that the endpoint satellite is the second endpoint satellite, the source satellite is not the first primary satellite, and the second endpoint satellite is not the second primary satellite, the source satellite is designated as the satellite to be decided, and the first routing path is obtained. The first primary satellite node is used as the satellite to be decided. The routing decision method is used to determine the next routing node satellite of the first primary satellite node, and to determine whether the next routing node satellite of the first primary satellite node is the second primary satellite node. In response to determining that the next routing node satellite of the first primary satellite node is not the second primary satellite node, the next routing node is taken as a new satellite to be decided, until the next routing node determined by the routing decision method is the second primary satellite node, thus obtaining a third routing path including the first primary satellite node, all routing node satellites and the second primary satellite node. The second primary satellite node is used as the satellite to be decided. The routing decision method is used to determine the next routing node satellite of the second primary satellite node, and to determine whether the next routing node satellite of the second primary satellite node is the second destination node satellite. In response to determining that the next routing node satellite of the second primary satellite node is not the second destination satellite, the next routing node is taken as a new satellite to be decided, until the next routing node determined by the routing decision method is the second destination satellite, thus obtaining a fourth routing path including the second primary satellite node, all routing node satellites and the second destination satellite. The second multicast routing path, consisting of the first routing path, the third routing path, and the fourth routing path, runs from the source node satellite to the second destination node satellite. The routing decision method includes: Acquire the first information of the source node satellite and the second information of the destination node satellite in the routing path decision task; Identify the satellite to be decided and use its neighboring node satellites as candidate routing node satellites; Obtain the real-time traffic load of the candidate routing node satellites, as well as the predicted traffic load within a preset time step in the future; Based on the real-time traffic load, the predicted traffic load, the first information, and the second information, the next routing node satellite is determined from the candidate routing node satellites using a routing decision model pre-built based on a Markov decision process.

2. The multicast routing path decision method of claim 1, wherein, The step of obtaining the predicted traffic load of the candidate routing node satellites within a future preset time step includes: Obtain network topology information of the satellite network and link status information of each node satellite; Based on the link status information of each node satellite and the network topology information, traffic prediction is performed using a pre-built traffic prediction model to obtain the predicted traffic load of each node satellite.

3. The multicast routing path decision method according to claim 2, characterized in that, The traffic prediction model includes a graph convolutional neural network, a gated recurrent neural network, and a fully connected layer; the traffic prediction based on the link state information of each node satellite and the network topology information, using the pre-built traffic prediction model, yields the predicted traffic load of each node satellite, including: Based on the link status information of each node satellite and the network topology information, spatial features are extracted using the graph convolutional neural network. Based on the spatial features, the gated recurrent neural network is used to extract the temporal features, and the gated recurrent neural network is used to fuse the spatial features and the temporal features to obtain the spatiotemporal features. The weights of the spatiotemporal features are calculated using an attention mechanism, and the predicted traffic load of each node satellite is output through the fully connected layer based on the spatiotemporal features and their corresponding weights.

4. The multicast routing path decision method according to claim 1, characterized in that, The routing decision model is constructed through the following steps: Define the agent, state space, action space, and reward policy of the Markov decision process to obtain the initial routing decision model; A reinforcement learning network is constructed based on the dual-deep Q-learning algorithm; The initial routing decision model is trained using the reinforcement learning network until a preset condition is met, thus obtaining the routing decision model. The state space consists of the real-time traffic load, the predicted traffic load, the first information, and the second information. The action space is the set of all possible actions obtained by the agent based on the state space; The reward strategy is determined by the link transmission cost and link status of the satellite network.

5. The multicast routing path decision method of claim 4, wherein, The reinforcement learning network includes a value function network and a target network; The step of training the initial routing decision model using the reinforcement learning network includes: Based on the current state of the agent, the value function Q value corresponding to all possible actions to be executed in the current state is calculated in the value function network; Based on the value function Q value corresponding to all possible actions to be executed in the current state, a greedy strategy is used to determine the first target action that the agent plans to execute in the current state, and the value function Q value corresponding to the first target action is recorded. Instruct the agent to perform the first target action to obtain the state at the next moment; Based on the state at the next moment, the value function Q value corresponding to all possible actions in the state at the next moment is calculated in the value function network, and the action corresponding to the maximum value of the value function Q value among all possible actions in the state at the next moment is taken as the second target action. Based on the state at the next moment and the second target action, the target Q value corresponding to the second target action is calculated in the target network; The value function value is calculated based on the target Q value; Using the value function value as the actual value and the value function Q value corresponding to the first target action as the predicted value, backpropagation of the error is performed to complete one update of the network parameters of the value function network. Specifically, at predetermined intervals, the network parameters of the target network are adjusted to be the same as those of the value function network.

6. The multicast routing path decision method of claim 2, wherein, The link status information includes link connectivity, bandwidth capacity, latency, and packet loss rate; The neighboring node satellites of the satellite to be decided include adjacent satellites in the orbital plane of the satellite to be decided and satellites with the same number as the satellite to be decided in the adjacent orbital plane of the satellite to be decided.

7. The method of claim 1, wherein, Also includes: Determine the data to be transmitted, and transmit the data to be transmitted from the source node satellite to the first master satellite node according to the first routing path; The first primary satellite node performs a first copy operation on the data to be transmitted, and distributes the data to be transmitted after the first copy operation to each of the first destination satellites and the second primary satellites according to the second routing path and the third routing path, wherein the number of data to be transmitted after the first copy operation is equal to the sum of the number of the first destination satellites and the second primary satellites; The second primary satellite node performs a second copy operation on the data to be transmitted, and distributes the data to be transmitted after the second copy operation to each of the second destination satellites according to the fourth routing path, wherein the number of data to be transmitted after the second copy operation is equal to the number of the second destination satellites.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements a multicast routing path decision method as described in any one of claims 1 to 7.

Images