Network path planning optimization method considering multiple transmission constraint models

Abstract

The present invention relates to a network path planning optimization method considering multiple types of transmission constraint models, comprising the following steps: step S1, a satellite acquires environmental information, and generates a semantic vector based on the environmental information; step S2, in any communication cycle, the generated semantic vector is transmitted to an adjacent satellite; step S3, when a satellite has a service requiring a path planning decision, the semantic vectors from the adjacent satellite enter a semantic aggregation module to generate an aggregated semantic vector; step S4, based on the aggregated semantic vector, a strategy vector is output to formulate a next-hop decision for the satellite's service. The present invention can efficiently utilize communication resources, avoid local optimality, and achieve more global and accurate path planning decisions.

Description

Technical Field

[0001] The present invention relates to the field of satellite Internet technology, and in particular to a network path planning optimization method considering multiple types of transmission constraint models. Background Art

[0002] Satellite Internet is a communication network that uses satellites for data exchange. It has the characteristics of diverse business types, complex transmission constraints, and limited frequency resources. Its service quality depends on efficient path planning methods. In satellite Internet, path planning is used to determine the optimal intersatellite link sequence from the source to the destination. The data volume, real-time performance, reliability and other transmission requirements of the services carried by satellite Internet are different. In addition, the communication satellites are large in scale, time-varying in topology, and heterogeneous in frequency bands, which leads to high dimensions of the state and action space of the path planning method. Traditional path planning methods are difficult to calculate the optimal solution in real time. For example, traditional path planning methods such as Dijkstra and A* are only applicable to simple network topologies and single business requirements. At present, with the development of artificial intelligence technology, there are path planning methods based on multi-agent deep reinforcement learning. The path planning function of each communication satellite is modeled as an agent, and distributed path planning is realized through multi-agent collaborative decision-making. However, the agents in the existing methods have limited knowledge of the global network topology, link load, and planning strategies of other agents. The independent decision-making of each agent is difficult to achieve efficient intersatellite collaboration in a highly dynamic environment.

[0003] Therefore, it is necessary to improve the existing technology and enhance the effectiveness of satellite Internet path planning methods. Summary of the invention

[0004] In order to solve the technical problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a network path planning optimization method considering multiple types of transmission constraint models, which can effectively utilize semantic information, reasonably allocate links, and improve service transmission efficiency.

[0005] To achieve the above-mentioned object of the invention, the present invention provides a network path planning optimization method considering multiple types of transmission constraint models, comprising the following steps:

[0006] Step S1: The satellite acquires environmental information and generates a semantic vector based on the environmental information;

[0007] Step S2: in any communication cycle, the generated semantic vector is transmitted to the adjacent satellite;

[0008] Step S3: When a satellite has a service that requires a path planning decision, the semantic vectors from the adjacent satellites enter the semantic aggregation module to generate an aggregated semantic vector;

[0009] Step S4: Based on the aggregated semantic vector, a strategy vector is output to make a next-hop decision for the satellite service.

[0010] According to a technical solution of the present invention, the step S1 specifically includes:

[0011] Step S11, pre-processing the original data of the satellite's environmental information, extracting the satellite's orbit status, link status, satellite-borne service information, and specific service requirements as characteristic information of the satellite node;

[0012] Step S12: Concatenate the extracted feature information of the satellite nodes to form a vector as the input of the semantic coding module;

[0013] Step S13: The semantic encoding module uses a neural network to encode the input vector to generate a semantic vector.

[0014] According to a technical solution of the present invention, in the step S2, it specifically includes:

[0015] Step S21, setting different communication cycle durations based on the business requirements of a specific scenario;

[0016] Step S22: At the beginning of each communication cycle, the satellite inputs the semantic vector into the semantic communication module;

[0017] Step S23: the semantic communication module searches for an available link and transmits the semantic communication vector to an adjacent satellite through the link;

[0018] Step S24: After the transmission is completed, the semantic communication module receives the semantic vectors from other satellites as input to the semantic aggregation module.

[0019] According to a technical solution of the present invention, step S3 specifically includes:

[0020] Step S31, the semantic aggregation module concatenates the semantic vectors received from the semantic communication module;

[0021] Step S32: The satellite gradually extracts and fuses semantic information of different adjacent satellites through the aggregation network, and outputs an aggregated semantic vector.

[0022] According to a technical solution of the present invention, the step S4 specifically includes:

[0023] The aggregated semantic vector is input into the neural network of the intelligent decision module, and a decision vector is output. The decision vector represents the probability of the service selecting the next-hop satellite, and the satellite makes the service next-hop decision based on the decision vector.

[0024] According to a technical solution of the present invention, in step S11, the orbital state of satellite i includes one or more of orbital inclination, altitude, subsatellite point, ascending node, period, intercept, eccentricity, perigee angle, and mean anomaly angle;

[0025] The link status of satellite i includes one or more of communication satellite, working frequency band, antenna aperture, modulation mode, and coding mode;

[0026] The service information of satellite i includes one or more of the starting satellite, the ending satellite, the data volume, and the deadline.

[0027] According to a technical solution of the present invention, step S13 is expressed as:

[0028]

[0029] in, are vectors representing the service requirements of satellite i, satellite orbit status, satellite link status, and other service transmission status carried by the satellite itself. Encoder represents the encoding network, LSTM represents the recurrent neural network long short-term memory network, represents the LSTM hidden state vector of satellite i at time t-1, Represents the LSTM hidden state vector of satellite i at time t, and is also used as the final output semantic vector.

[0030] According to a technical solution of the present invention, step S32 is expressed as:

[0031]

[0032] in, are the current satellite i and the adjacent satellite j respectively 1 ,…,j k The semantic vector of Agg represents the aggregation network, which is implemented as a single hidden layer MLP. Represents the aggregated semantic vector.

[0033] According to a technical solution of the present invention, step S4 is expressed as:

[0034]

[0035] Among them, π θ is the Actor network, θ is the parameter of the Actor network, is the input aggregate semantic vector, is the output strategy vector.

[0036] According to a technical solution of the present invention, the step S4 further includes: updating the Actor network and updating the Critic network;

[0037] The update method of the Actor network is as follows:

[0038]

[0039] Among them, Q w is the Critic network, which is used to evaluate the value of the strategy. θ is the learning rate;

[0040] The updating method of the Critic network is as follows:

[0041]

[0042] Among them, r i is the reward function, defined as the difference between the amount of data successfully transmitted and the amount of data failed to be transmitted, γ is the discount factor, Q w is the Critic network function, with parameter w, To adopt strategies The aggregated semantic vector generated by satellite i, For Actor networks is the output of the input, α w is the learning rate.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] The present invention proposes a network path planning optimization method that considers multiple types of transmission constraint models. It uses semantic communication to achieve efficient information transmission, save communication resources, and extract key information. It filters out useless information through the self-learning encoding process, making the transmitted information more refined. The algorithm can obtain more comprehensive and accurate environmental information through communication and information sharing between satellites, which is conducive to avoiding local optimality and making path planning decisions more global and accurate.

[0045] Furthermore, based on the network path planning optimization method of the present invention, the satellite has better collaborative learning and communication capabilities, and can adjust the path planning in real time to cope with changes in dynamic network conditions and mission requirements. At the same time, by enhancing the strategic coordination of path planning for multiple satellites to multiple services, planning conflicts are avoided, which is conducive to achieving optimal network path planning optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0047] Figure 1 A flowchart schematically shows a network path planning optimization method considering multiple types of transmission constraint models provided in accordance with an embodiment of the present invention;

[0048] Figure 2 A specific flow chart of step S1 according to an embodiment of the present invention is schematically shown;

[0049] Figure 3 A specific flow chart of step S2 according to an embodiment of the present invention is schematically shown;

[0050] Figure 4 A specific flow chart of step S3 according to an embodiment of the present invention is schematically shown;

[0051] Figure 5 A module diagram of a network path planning optimization method considering multiple types of transmission constraint models according to an embodiment of the present invention;

[0052] Figure 6 This is a schematic diagram of an encoder network in step S1 of a network path planning optimization method considering multiple types of transmission constraint models in an embodiment of the present invention.

[0053] Figure 7 This is a schematic diagram of a long short-term memory network in step S1 of a network path planning optimization method considering multiple types of transmission constraint models in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] like Figures 1 to 7 As shown, a network path planning optimization method considering multiple types of transmission constraint models of the present invention comprises the following steps:

[0056] Step S1: The satellite acquires environmental information and generates a semantic vector based on the environmental information, such as Figure 2As shown, specifically including:

[0057] Step S11, pre-processing the original data of the satellite's environmental information, extracting the satellite's orbit status, link status, satellite-borne service information, and specific service requirements as characteristic information of the satellite node;

[0058] In this step, the orbital state of satellite i includes one or more of orbital inclination, altitude, subsatellite point, ascending node, period, intercept, eccentricity, perigee angle, and mean perigee angle, which can be expressed as a vector:

[0059] [Orbital inclination, altitude, subsatellite point, ascending node, period, intercept, eccentricity, perigee angle, mean anomaly angle]

[0060] Among them, the orbital inclination is the angle between the equatorial plane and the satellite orbital plane. The specific calculation is the angle from the equatorial plane to the orbital plane when the satellite orbit is in the ascending phase. Altitude, the distance of the satellite from the earth's surface. Subsatellite point, the intersection of the line connecting the satellite and the center of the earth on the earth's surface. Ascending node, the intersection of the satellite's flight trajectory from south to north on the equator. Period, the time required for the satellite to orbit the earth. Intercept, the degree of rotation of the earth when the satellite orbits the earth. Eccentricity, the ratio of the focal length to the semi-major axis of the orbit. Perigee angle, the angle between the ascending node and the perigee and the center of the earth in the orbital plane. Average anomaly angle, if the time when the satellite passes the perigee is t p , the average angular velocity of the satellite is n, then the average anomaly angle at any time is M = n(tt p );

[0061] The link status of satellite i includes one or more of the communication satellite, working frequency band, antenna aperture, modulation mode, and coding mode, which can be expressed as a vector:

[0062] [communication satellite, operating frequency band, antenna aperture, modulation method, coding method];

[0063] The service information of satellite i includes one or more of the starting satellite, the ending satellite, the data volume, and the deadline, which can be expressed as a vector:

[0064] [starting point satellite, ending point satellite, data volume, deadline];

[0065] Step S12: Concatenate the extracted feature information of the satellite nodes to form a vector as the input of the semantic coding module;

[0066] Step S13: The semantic encoding module uses a neural network to encode the input vector to generate a semantic vector, which is expressed as:

[0067]

[0068] in, are vectors representing the service requirements of satellite i, satellite orbit status, satellite link status, and other service transmission status carried by the satellite itself. Encoder represents the encoding network, which is implemented using a single hidden layer perceptron (Multilayer Perceptron, MLP). The number of neurons in the hidden layer is 128, and the number of neurons in the output layer is 64. LSTM represents a recurrent neural network long short-term memory network, which uses a gating mechanism and hidden states to solve long-term dependency problems. In one embodiment of the present invention, the length of the hidden state is 128. represents the LSTM hidden state vector of satellite i at time t-1, Represents the LSTM hidden state vector of satellite i at time t, and is also used as the final output semantic vector.

[0069] Step S2: In any communication cycle, the generated semantic vector is transmitted to the adjacent satellites, such as Figure 3 As shown, specifically including:

[0070] Step S21: different communication cycle durations are set based on the business requirements of a specific scenario. For example, a communication cycle may be set to 60 seconds.

[0071] Step S22: At the beginning of each communication cycle, the satellite inputs the semantic vector into the semantic communication module;

[0072] Step S23: the semantic communication module searches for an available link and transmits the semantic communication vector to an adjacent satellite through the link;

[0073] Step S24: After the transmission is completed, the semantic communication module receives the semantic vectors from other satellites as input to the semantic aggregation module.

[0074] In step S2, the satellite searches for currently visible satellites through known satellite orbits and establishes a link with each satellite for transmission.

[0075] Step S3: When a satellite has a service that requires a path planning decision, the semantic vectors from the adjacent satellites enter the semantic aggregation module to generate an aggregated semantic vector, such as Figure 4 As shown, specifically including:

[0076] Step S31, the semantic aggregation module concatenates the semantic vectors received from the semantic communication module;

[0077] Step S32: The satellite gradually extracts and fuses the semantic information of different adjacent satellites through the aggregation network, and outputs an aggregated semantic vector, which is calculated as follows:

[0078]

[0079] in, are the current satellite i and the adjacent satellite j respectively 1 ,…,j k The semantic vector of Agg represents the aggregation network, which is implemented as a single hidden layer MLP. Specifically, the number of neurons in the hidden layer is 128, and the number of neurons in the output layer is 64. represents the aggregated semantic vector, whose length is defined as 64;

[0080] In step S3, the link for satellite transmission of semantic vectors is the same as the link for transmission of services. However, since the semantic vectors are very short, the occupancy of the link is almost negligible, and therefore their impact on service transmission is not considered.

[0081] Step S4: Based on the aggregated semantic vector, a strategy vector is output to make a next-hop decision for the satellite service, specifically including:

[0082] The aggregated semantic vector is input into the neural network of the intelligent decision module, and a decision vector is output. The decision vector represents the probability of the service selecting the next-hop satellite, and the satellite makes the service next-hop decision based on the decision vector.

[0083] Among them, the aggregated semantic vector is input into the neural network of the intelligent decision-making module, and the decision vector is output. The calculation formula is as follows:

[0084]

[0085] Among them, π θ is the Actor network, θ is the parameter of the Actor network, is the input aggregate semantic vector, is the output strategy vector. Specifically, π θ It is implemented with a single hidden layer MLP and softmax. The number of neurons in the hidden layer of MLP is 128, and the number of neurons in the output layer is 400. Softmax maps the outputs of multiple neurons to the interval (0,1) and ensures that the sum of all numbers is 1.

[0086] The decision vector represents the probability of the service selecting the next-hop satellite, based on which the satellite makes the next-hop decision for the service. The output strategy vector is defined as 400 in length. The satellite selects the next hop satellite according to the number in the strategy vector. Specifically, taking the vector [0.3, 0.4, 0.2, 0.1] as an example, the satellite selects the adjacent satellite 1 with a probability of 0.3, the adjacent satellite 2 with a probability of 0.4, the adjacent satellite 3 with a probability of 0.2, and the adjacent satellite 4 with a probability of 0.1. The intelligent decision module selects the adjacent satellite 1 with a probability of 0.3, the adjacent satellite 2 with a probability of 0.4, the adjacent satellite 3 with a probability of 0.2, and the adjacent satellite 4 with a probability of 0.1. The next-hop satellite selection is generated, representing the path planning decision for the satellite service.

[0087] In one embodiment of the present invention, preferably, step S4 also includes: updating of the Actor network and updating of the Critic network, wherein the updating of the Actor network aims to maximize the cumulative reward, and the updating of the Critic network can improve the accuracy of the value evaluation of the strategy.

[0088] The update method of the Actor network is as follows:

[0089]

[0090] Among them, Q w is the Critic network, which is used to evaluate the value of the strategy. θ is the learning rate;

[0091] The updating method of the Critic network is as follows:

[0092]

[0093] Among them, r i is the reward function, defined as the difference between the amount of data successfully transmitted and the amount of data failed to be transmitted, γ is the discount factor, Q w is the Critic network function, with parameter w, To adopt strategies The aggregated semantic vector generated by the satellite i is For Actor networks is the output of the input, α w is the learning rate.

[0094] According to one aspect of the present invention, an electronic device is provided, comprising: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory, and when the electronic device is running, the processor executes the one or more computer programs stored in the memory, so that the electronic device executes a network path planning optimization method considering multiple types of transmission constraint models as described in any of the above technical solutions.

[0095] According to one aspect of the present invention, a computer-readable storage medium is provided for storing computer instructions. When the computer instructions are executed by a processor, a network path planning optimization method considering multiple types of transmission constraint models as described in any of the above technical solutions is implemented.

[0096] Computer readable storage media may include any medium capable of storing or transmitting information. Examples of computer readable storage media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segments may be downloaded via a computer network such as the Internet, an intranet, etc.

[0097] The present invention provides a network path planning optimization method that considers multiple types of transmission constraint models. First, the satellite acquires environmental information and generates a semantic vector based on the environmental information; second, in any communication cycle, the generated semantic vector is transmitted to adjacent satellites; third, when the satellite has a service that requires a path planning decision, the semantic vector from the adjacent satellite enters the semantic aggregation module to generate an aggregated semantic vector; finally, based on the aggregated semantic vector, a strategy vector is output to make a next-hop decision for the satellite's service. The use of semantic communication can achieve efficient information transmission, save communication resources, and extract key information. Useless information is filtered out through an autonomous learning encoding process, making the transmitted information more refined. The algorithm can obtain more comprehensive and accurate environmental information through communication and information sharing between satellites, which is conducive to avoiding local optimality and making path planning decisions more global and accurate.

[0098] Furthermore, based on the network path planning optimization method of the present invention, the satellite has better collaborative learning and communication capabilities, and can adjust the path planning in real time to cope with changes in dynamic network conditions and mission requirements. At the same time, by enhancing the strategic coordination of path planning for multiple satellites to multiple services, planning conflicts are avoided, which is conducive to achieving optimal network path planning optimization.

[0099] In addition, it should be noted that the present invention can be provided as a method, an apparatus or a computer program product. Therefore, the embodiments of the present invention can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0100] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0101] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0102] It should also be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0103] Finally, it should be noted that the above is a preferred embodiment of the present invention. It should be pointed out that although the preferred embodiment of the present invention has been described, for those skilled in the art, once the basic creative concept of the present invention is known, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention. Therefore, the attached claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A network path planning optimization method considering multiple types of transmission constraint models, characterized in that: The following steps are involved: Step S1: The satellite acquires environmental information and generates a semantic vector based on the environmental information; Step S2: in any communication cycle, the generated semantic vector is transmitted to the adjacent satellite; Step S3: When a satellite has a service that requires a path planning decision, the semantic vectors from the adjacent satellites enter the semantic aggregation module to generate an aggregated semantic vector; Step S4: Based on the aggregated semantic vector, a strategy vector is output to make a next-hop decision for the satellite service; The step S1 specifically includes: Step S11, pre-processing the original data of the satellite's environmental information, extracting the satellite's orbit status, link status, satellite-borne service information, and specific service requirements as characteristic information of the satellite node; Step S12: Concatenate the extracted feature information of the satellite nodes to form a vector as the input of the semantic coding module; Step S13: The semantic encoding module uses a neural network to encode the input vector to generate a semantic vector; In the step S2, it specifically includes: Step S21, setting different communication cycle durations based on the business requirements of a specific scenario; Step S22: At the beginning of each communication cycle, the satellite inputs the semantic vector into the semantic communication module; Step S23: the semantic communication module searches for an available link and transmits the semantic communication vector to an adjacent satellite through the link; Step S24: After the transmission is completed, the semantic communication module receives the semantic vector from the adjacent satellite as the input of the semantic aggregation module; The step S3 specifically includes: Step S31, the semantic aggregation module concatenates the semantic vectors received from the semantic communication module; Step S32: The satellite gradually extracts and fuses semantic information of different adjacent satellites through an aggregation network, and outputs an aggregated semantic vector; The step S4 specifically includes: The aggregated semantic vector is input into the neural network of the intelligent decision module, and a decision vector is output, wherein the decision vector represents the probability of the service selecting the next-hop satellite, and the satellite makes a service next-hop decision based on the decision vector; In step S11, the orbital state of satellite i includes one or more of orbital inclination, altitude, subsatellite point, ascending node, period, intercept, eccentricity, perigee angle, and mean anomaly angle; The link status of satellite i includes one or more of communication satellite, working frequency band, antenna aperture, modulation mode, and coding mode; The service information of satellite i includes one or more of the starting satellite, the ending satellite, the data volume, and the deadline; The step S13 is expressed as: in, are vectors representing the service requirements of satellite i, satellite orbit status, satellite link status, and other service transmission status carried by the satellite itself. Encoder represents the encoding network, LSTM represents the recurrent neural network long short-term memory network, represents the LSTM hidden state vector of satellite i at time t-1, Represents the LSTM hidden state vector of satellite i at time t, and is also used as the final output semantic vector; The step S32 is expressed as: in, are the current satellite i and the adjacent satellites j1,…,j k The semantic vector of Agg represents the aggregation network, which is implemented as a single hidden layer MLP. represents the aggregated semantic vector; The step S4 is expressed as: Among them, π θ is the Actor network, θ is the parameter of the Actor network, is the input aggregate semantic vector, is the output strategy vector.

2. The network path planning optimization method considering multiple types of transmission constraint models according to claim 1 is characterized in that: The step S4 also includes: updating the Actor network and updating the Critic network; The update method of the Actor network is as follows: Among them, the role of the Critic network is to evaluate the value of the strategy, α θ is the learning rate; The updating method of the Critic network is as follows: Among them, r i is the reward function, defined as the difference between the amount of data successfully transmitted and the amount of data failed to be transmitted, γ is the discount factor, Q w is the Critic network function, with parameter w, is the strategy vector for taking output The aggregated semantic vector generated by satellite i, For Actor networks is the output of the input, α w is the learning rate.

Images