Low earth orbit satellite network cross-beam switching method and system based on diffusion model

Abstract

The invention discloses a low-orbit satellite network cross-beam switching method based on a diffusion model, which relates to the technical field of low-orbit satellite beam switching and comprises the following steps: constructing a satellite network model, a user equipment model, a diffusion model and a deep reinforcement learning network model; when the user equipment model is in different beam coverage ranges, beam switching is considered; the diffusion model predicts future position distribution of the user equipment model, and beams needing to be switched are determined in advance; and the intelligent agent decides an optimal beam switching strategy according to future position distribution, enables the user equipment model to execute beam switching and feeds back a result to the intelligent agent to update model parameters. According to the method, the diffusion model is used for predicting the future position distribution of the user equipment, and the optimal beam switching strategy is selected by using the deep reinforcement learning network model, so that a plurality of attributes influencing switching judgment are comprehensively considered, the method is suitable for various complex scenes, the success rate and the efficiency of beam switching are improved, and the user experience is improved. And the user service quality is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of low-orbit satellite mobile communication networks, and more specifically, to a low-orbit satellite network cross-beam switching method and system based on a diffusion model. Background Art

[0002] Low Earth Orbit (LEO) satellite mobile communication networks are widely expected to have great development potential in the future construction of space-ground integration due to their high efficiency, low latency, low transmission loss and powerful multiple access capabilities. They have become an indispensable and important part of modern mobile communication systems. In order to cover a larger area, multi-beam low-orbit satellites replace a single large-angle beam with multiple small beams, and each beam forms a cell. However, when users cross multiple beams during movement, frequent beam switching will cause signal interference and large latency, thereby reducing communication quality. In addition, the additional processing and calculations required during the switching process will also increase the energy consumption of the satellite and shorten its service life. Therefore, in order to improve the service quality of multi-beam low-orbit satellite mobile communication networks, it is crucial to design efficient and reliable access and switching solutions.

[0003] The existing handover decision method based on single attribute decision takes a single attribute factor as the decision criterion and makes handover decisions by comparing the measured value with the threshold. This type of method is relatively simple and can reduce the number of handovers to a certain extent, ensuring the communication quality of user terminals. It is more suitable for scenarios with simple satellite network structures and relatively few user nodes, but it has poor adaptability to complex and changeable network environments and lacks comprehensive consideration of handover decision factors, which easily leads to the optimization of individual attributes while other attributes are poor, making it difficult to meet the needs of user terminals and the success rate and efficiency of beam switching are not high enough.

[0004] The core idea of ​​the switching decision method based on multi-attribute decision-making is to comprehensively consider dynamic factors such as signal strength, delay, traffic load and terminal speed that affect the switching decision, as well as static factors such as network type, user attributes and user preferences. Different weight values ​​are assigned to relevant attribute factors according to the actual situation and different needs of low-orbit satellite network nodes and user terminals, so as to select the most suitable switching target. Although this type of method comprehensively considers multiple attributes that affect the switching decision, with the goal of assisting user terminals to select the best switching, the success rate and efficiency of beam switching are relatively high, but most of the multi-attribute switching decision methods are aimed at the exact optimization target problem of a single scenario. The complexity increases with the increase of decision attributes, and it is difficult to apply to multiple types of complex scenarios. Summary of the invention

[0005] The purpose of the present invention is to overcome the problem in the prior art that it is difficult to comprehensively consider multiple attributes that affect switching decisions while being applicable to various complex scenarios and improving the success rate and efficiency of beam switching, and to provide a cross-beam switching method for a low-orbit satellite network based on a diffusion model.

[0006] To achieve the above object, the technical solution of the present invention is as follows:

[0007] The present invention provides a low-orbit satellite network cross-beam switching method based on a diffusion model, comprising the following steps:

[0008] According to the application scenarios of low-orbit satellite networks, satellite network models, user equipment models, diffusion models and deep reinforcement learning network models are constructed;

[0009] In the application scenario, if the user equipment model is in different beam coverages in the satellite network model at different time points, the user equipment model performs beam switching according to real-time channel conditions and satellite positions;

[0010] The diffusion model predicts the future position distribution of the user equipment model, thereby determining the beam that needs to be switched in advance;

[0011] The agent of the deep reinforcement learning network model dynamically decides the optimal beam switching strategy of the user equipment model according to the future position distribution, the user equipment model performs beam switching according to the optimal beam switching strategy and feeds back the result to the agent, and the agent updates the parameters of the deep reinforcement learning network model according to the feedback result.

[0012] Furthermore, when constructing the diffusion model, data needs to be collected, and the data includes: historical location data, speed data and direction data of the user equipment in the user equipment model, and the data is used to train the diffusion model.

[0013] Furthermore, the diffusion model is a Gaussian process GP model.

[0014] Furthermore, the training of the diffusion model is specifically as follows:

[0015] Using the historical location data p i (i=1,2,3...N), and the speed data v i (i=1,2,3,...,N) to train the Gaussian process GP model:

[0016] GP~N(μ(x),k(x,x′))

[0017] Where x = [p, v], the mean function μ(x) and the kernel function k(x, x′) are used to describe the distribution of the data.

[0018] Furthermore, the diffusion model predicts the future location distribution of the user equipment model, including: according to the current state of the given user equipment model:

[0019] x t =[p t ,v t ]

[0020] where p t Indicates the current position, v t Represents the current speed and predicts the position of the next T steps:

[0021] p t+i (i=1,2,3,...,T): p t+1 ~GP(x t+1 )

[0022] x t+1 =[p t+1 ,v t+1 ]

[0023] And so on, we get the position sequence of the next T steps.

[0024] Furthermore, the construction of the deep reinforcement learning network model is implemented using a deep Q network DQN.

[0025] Furthermore, the deep Q network DQN includes: state s t 、Action a t and reward r t ;

[0026] The status t Including: current beam ID, channel quality indicator CQI, current position of user equipment, current speed and future position predicted by the diffusion model;

[0027] The action a t Indicates which beam to switch to at the current moment;

[0028] The reward r t It is determined by factors such as beam switching success rate, channel utilization, signal strength, and beam switching delay. The reward function is designed as follows:

[0029] R = α × CQI - β × δ - χ × η

[0030] Among them, α, β, χ are weight coefficients, δ is the switching delay, and η represents the switching success rate. The above parameters are adjusted according to actual needs;

[0031] The deep Q network DQN selects the optimal action by updating the following Q value function, the expression of which is:

[0032] Q(s t ,a t )=r t+γmaxa′Q(s t+1 ,a′)

[0033] Here, γ is the discount factor.

[0034] Furthermore, the experience replay and target network method is used to stabilize the training process of the deep Q network DQN. The specific steps are as follows:

[0035] Given a sample in the experience replay pool (s t ,a t ,r t ,s t+1 ,d t ), calculate the target value y j :

[0036] y j =r j +γ(1-d j )max a′ Q target (s j+1 ,a′)

[0037] Among them, d t Indicates whether to terminate, Q target is the target network;

[0038] Calculate the loss function L:

[0039]

[0040] Update the parameters of the deep Q network DQN through back propagation and update the current state to the next state s t+1 .

[0041] Furthermore, dynamically determining the optimal beam switching strategy of the user equipment model according to the future position distribution includes the following steps:

[0042] Update current status t , including the current beam ID, the channel quality indicator CQI, the current position of the user equipment, the current speed and the future position predicted by the diffusion model;

[0043] Use the ò-greedy strategy to select action a t :

[0044]

[0045] Perform the selected action a t , and get the next state s t+1 , and the corresponding reward r t and whether to terminate the flag d t ;

[0046] Update the experience replay pool to convert newly gained experience (s t ,a t ,r t ,s t+1 ,d t ) is added to the experience replay pool

[0047] Randomly sample a batch of samples from the experience replay pool for training and update the parameters of the Q network. The specific steps include: calculating the target value y t And the loss function L, and then update the parameters of the Q network through back propagation.

[0048] On the other hand, the present invention also provides a low-orbit satellite network cross-beam switching system based on a diffusion model, and the low-orbit satellite network cross-beam switching system based on a diffusion model includes: a memory and a processor, wherein the memory includes a program of a low-orbit satellite network cross-beam switching method based on a diffusion model, and when the program of the low-orbit satellite network cross-beam switching method based on a diffusion model is executed by the processor, the following steps are implemented:

[0049] According to the application scenarios of low-orbit satellite networks, satellite network models, user equipment models, diffusion models and deep reinforcement learning network models are constructed;

[0050] In the application scenario, when the user equipment model is in different beam coverages in the satellite network model at different time points, the user equipment model performs beam switching according to real-time channel conditions and satellite positions;

[0051] The diffusion model predicts the future position distribution of the user equipment model, thereby determining the beam that needs to be switched in advance;

[0052] The agent of the deep reinforcement learning network model dynamically decides the optimal beam switching strategy of the user equipment model according to the future position distribution, the user equipment model performs beam switching according to the optimal beam switching strategy and feeds back the result to the agent, and the agent updates the parameters of the deep reinforcement learning network model according to the feedback result.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] The present invention uses a diffusion model to predict the future location distribution of user devices and a deep reinforcement learning network model to select the optimal beam switching strategy. While comprehensively considering multiple attributes that affect the switching decision, it is applicable to various complex scenarios and improves the success rate and efficiency of beam switching, thereby improving the user service quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1It is a flowchart of a cross-beam switching method of a low-orbit satellite network based on a diffusion model of the present application;

[0056] Figure 2 It is a principle block diagram of the beam switching algorithm of Example 2 of the present application. DETAILED DESCRIPTION

[0057] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0058] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0059] Example 1

[0060] See also Figure 1 The embodiment of the present invention provides a low-orbit satellite network cross-beam switching method based on a diffusion model, comprising the following steps:

[0061] S1: According to the application scenarios of low-orbit satellite networks, satellite network models, user equipment models, diffusion models and deep reinforcement learning network models are constructed.

[0062] Satellite network model: A network model consisting of multiple low-orbit satellites, each of which covers a certain ground area, which is divided into multiple beams.

[0063] User equipment model: user equipment or network controller, etc.

[0064] Diffusion model: The diffusion model is used to predict the movement trajectory and future location of user equipment, so as to determine the beam that needs to be switched in advance. The diffusion model simulates the movement path of user equipment on the ground, combines historical data and geographic information, and predicts the future location distribution.

[0065] In this embodiment, the diffusion model is modeled using the Gaussian process method. The historical location information, speed, direction and other data of the user device are collected, and then model training is carried out. The diffusion model is trained using these data to enable it to predict the future location distribution of the user device. The specific training of the diffusion model is:

[0066] Use historical location datap i (i=1,2,3...N), and speed data v i (i=1,2,3,...,N) to train the diffusion model, namely the Gaussian process GP model:

[0067] GP~N(μ(x),k(x,x′))

[0068] Where x = [p, v], the mean function μ(x) and the kernel function k(x, x′) are used to describe the distribution of the data.

[0069] Deep reinforcement learning network model: used to dynamically decide on beam switching strategies. By continuously interacting with the environment (satellite network), the agent learns how to select the optimal beam switching action under different states.

[0070] In this embodiment, the construction of the deep reinforcement learning network model is implemented by using a deep Q network DQN, which includes: state s t 、Action a t and reward r t ;

[0071] Status t Including: current beam ID, channel quality indicator CQI, current position of user equipment, current speed and future position predicted by the diffusion model;

[0072] Action a t Indicates which beam to switch to at the current moment;

[0073] Reward t It is determined by factors such as beam switching success rate, channel utilization, signal strength, and beam switching delay. The reward function is designed as follows:

[0074] R = α × CQI - β × δ - χ × η

[0075] Among them, α, β, χ are weight coefficients, δ is the switching delay, and η represents the switching success rate. The above parameters are adjusted according to actual needs.

[0076] The deep Q network DQN selects the optimal action by updating the following Q value function, which is expressed as:

[0077] Q(s t ,a t )=r t +γmaxa′Q(s t+1 ,a′)

[0078] Here, γ is the discount factor.

[0079] Use experience replay and target network methods to stabilize the training process of the deep Q network DQN. The specific steps are as follows:

[0080] Given a sample in the experience replay pool (s t ,a t ,rt ,s t+1 ,d t ), calculate the target value y j :

[0081] y j =r j +γ(1-d j )max a′ Q target (s j+1 ,a′)

[0082] Among them, d t Indicates whether to terminate, Q target is the target network;

[0083] Calculate the loss function L:

[0084]

[0085] Update the parameters of the deep Q network DQN through back propagation and update the current state to the next state s t+1 .

[0086] The goal of the diffusion model and deep reinforcement learning network model is to minimize the interruption and delay of the beam switching process while maximizing channel utilization and user experience.

[0087] S2: In the application scenario, when the user equipment model is in different beam coverages in the satellite network model at different time points, the user equipment model performs beam switching according to the real-time channel conditions and satellite positions.

[0088] S3: The diffusion model predicts the future location distribution of the user equipment model, thereby determining the beam that needs to be switched in advance.

[0089] In this example, the future location distribution of the user equipment model predicted by the diffusion model is specifically:

[0090] Given the current state of the user device model:

[0091] x t =[p t ,v t ]

[0092] where p t Indicates the current position, v t Represents the current speed and predicts the position of the next T steps:

[0093] p t+i (i=1,2,3,...,T): p t+1 ~GP(x t+1 )

[0094] x t+1 =[p t+1 ,v t+1 ]

[0095] And so on, we get the position sequence of the next T steps.

[0096] S4: The intelligent agent of the deep reinforcement learning network model dynamically decides the optimal beam switching strategy of the user device model according to the future position distribution. The user device model performs beam switching according to the optimal beam switching strategy and feeds back the results to the intelligent agent. The intelligent agent updates the parameters of the deep reinforcement learning network model according to the feedback results.

[0097] In this example, the optimal beam switching strategy of the user equipment model is dynamically determined according to the future position distribution, including the following steps:

[0098] Status update: Update the current status t , including the current beam ID, the channel quality indicator CQI, the current position of the user equipment, the current speed and the future position predicted by the diffusion model.

[0099] Action selection: Use the ò-greedy strategy to select action a t :

[0100]

[0101] Execute action: Execute the selected action a t , and get the next state s t+1 , and the corresponding reward r t and whether to terminate the flag d t .

[0103] Update the experience replay pool: newly gained experience (s t ,a t ,r t ,s t+1 ,d t ) is added to the experience replay pool.

[0104] Deep Q network DQN training: randomly sample a batch of samples from the experience replay pool for training and update the parameters of the Q network. The specific steps include: calculating the target value y in the above steps t And the loss function L, and then update the parameters of the Q network through back propagation.

[0105] Example 2

[0106] This embodiment proposes a beam switching algorithm based on embodiment 1. Figure 2 : is a principle block diagram of the beam switching algorithm of embodiment 2 of the present application. The beam switching algorithm is specifically as follows:

[0107] The input of the algorithm is user status data, that is, historical or current location data, speed data, direction data, etc. of user equipment or network controllers.

[0108] The first step of the algorithm is initialization:

[0109] Train the diffusion model GP, initialize the reinforcement learning model Q(s,a) and the target network Q target (s,a), initialize the experience replay pool, set the discount factor γ, learning rate α and ε-greedy strategy parameter ò.

[0110] The main loop of the algorithm is:

[0111] For each time step (t):

[0112] Status acquisition: Get the current status t , including the current beam ID, CQI, device location and speed.

[0113] Future position prediction: Use the diffusion model GP to predict the position p of the next (T) steps t+i (i=1,2,...,N), update state s t To include future location distribution information.

[0114] Action selection:

[0115] Use the ò-greedy strategy to select action a t :

[0116]

[0117] Execute action: Execute action a t Get the next state s t+1 , Reward t and the termination mark d t .

[0118] Update the experience replay pool: (s t ,a t ,r t ,s t+1 ,d t ) is added to the experience replay pool.

[0119] Q network training: Randomly sample a batch of samples (s j ,a j ,r j ,s j+1 ,d j ).

[0120] Calculate the target value y j :

[0121] y j =r j +γ(1-d j )max a′ Q target (s j+1 ,a′)

[0122] Calculate the loss function L:

[0123]

[0124] Update the parameters of the Q network via back-propagation.

[0125] Update the current state to the next state s t+1 .

[0126] Check whether it is finished. If the current state is the terminal state, end the loop.

[0127] The algorithm also includes periodic updates to the target network outside the main loop:

[0128] Every fixed time step, the parameters of the current deep Q network are copied to the target network.

[0129] Example 3

[0130] Based on Example 1, this embodiment 3 further provides a low-orbit satellite network cross-beam switching system based on a diffusion model, characterized in that the low-orbit satellite network cross-beam switching system based on a diffusion model includes: a memory and a processor, the memory includes a program of a low-orbit satellite network cross-beam switching method based on a diffusion model, and the program of the low-orbit satellite network cross-beam switching method based on a diffusion model is executed by the processor to implement the following steps:

[0131] According to the application scenarios of low-orbit satellite networks, satellite network models, user equipment models, diffusion models and deep reinforcement learning network models are constructed;

[0132] In the application scenario, when the user equipment model is in different beam coverages in the satellite network model at different time points, the user equipment model performs beam switching according to real-time channel conditions and satellite positions;

[0133] The diffusion model predicts the future position distribution of the user equipment model, thereby determining the beam that needs to be switched in advance;

[0134] The agent of the deep reinforcement learning network model dynamically decides the optimal beam switching strategy of the user equipment model according to the future position distribution, the user equipment model performs beam switching according to the optimal beam switching strategy and feeds back the result to the agent, and the agent updates the parameters of the deep reinforcement learning network model according to the feedback result.

[0135] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store used or received data, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices.

[0136] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A cross-beam switching method for a low-orbit satellite network based on a diffusion model, characterized in that: The following steps are involved: According to the application scenarios of low-orbit satellite networks, satellite network models, user equipment models, diffusion models and deep reinforcement learning network models are constructed; In the application scenario, when the user equipment model is in different beam coverages in the satellite network model at different time points, the user equipment model performs beam switching according to real-time channel conditions and satellite positions; The diffusion model predicts the future position distribution of the user equipment model, thereby determining the beam that needs to be switched in advance; The agent of the deep reinforcement learning network model dynamically decides the optimal beam switching strategy of the user equipment model according to the future position distribution, the user equipment model performs beam switching according to the optimal beam switching strategy and feeds back the result to the agent, and the agent updates the parameters of the deep reinforcement learning network model according to the feedback result.

2. According to the method of claim 1, a low-orbit satellite network cross-beam switching method based on a diffusion model is characterized in that: When constructing the diffusion model, data needs to be collected, and the data includes: historical location data, speed data and direction data of the user equipment in the user equipment model, and the data is used to train the diffusion model.

3. According to the method of claim 2, a low-orbit satellite network cross-beam switching method based on a diffusion model is characterized in that: The diffusion model is a Gaussian process GP model.

4. According to claim 3, a low-orbit satellite network cross-beam switching method based on a diffusion model is characterized in that: The training of the diffusion model is specifically as follows: Using the historical location data p i (i=1,2,3...N), and the speed data v i (i=1,2,3,...,N) to train the Gaussian process GP model: GP~N(μ(x),k(x,x′)) Where x = [p, v], the mean function μ(x) and the kernel function k(x, x′) are used to describe the distribution of the data.

5. According to the method of claim 3, the cross-beam switching method of a low-orbit satellite network based on a diffusion model is characterized in that: The diffusion model predicts the future position distribution of the user equipment model, including: predicting the position of the future T steps according to the current state of the given user equipment model: x t =[p t ,v t ] p t+i (i=1,2,3,...,T):p t+1 ~GP(x t+1 ) x t+1 =[p t+1 ,v t+1 ] Among them, x t Indicates the current state of the user device model, p t Indicates the current position, v t Indicates the current speed, p t+i represents the position of the next T steps, x t+1 Represents the future state of the user device model; And so on, we get the position sequence of the next T steps.

6. According to the method of claim 1, wherein the method comprises: The construction of the deep reinforcement learning network model is implemented using a deep Q network DQN.

7. The method for switching between low-orbit satellite networks across beams based on a diffusion model according to claim 6, characterized in that: The deep Q network DQN includes: state s t 、Action a t and reward r t ; The status t Including: current beam ID, channel quality indicator CQI, current position of user equipment, current speed and future position predicted by the diffusion model; The action a t Indicates which beam to switch to at the current moment; The reward r t It is determined by factors such as beam switching success rate, channel utilization, signal strength, and beam switching delay. The reward function is designed as follows: R = α × CQI - β × δ - χ × η Among them, α, β, χ are weight coefficients, δ is the switching delay, and η represents the switching success rate. The above parameters are adjusted according to actual needs; The deep Q network DQN selects the optimal action by updating the following Q value function, the expression of which is: Q(s t ,a t )=r t +γmaxa′Q(s t+1 ,a′) Here, γ is the discount factor.

8. The method for switching between low-orbit satellite networks across beams based on a diffusion model according to claim 7, characterized in that: The experience replay and target network method is used to stabilize the training process of the deep Q network DQN. The specific steps are as follows: Given a sample in the experience replay pool (s t ,a t ,r t ,s t+1 ,d t ), calculate the target value y j : y j =r j +γ(1-d j )max a′ Q target (s j+1 ,a′) Among them, d t Indicates whether to terminate, Q target is the target network; Calculate the loss function L: Update the parameters of the deep Q network DQN through back propagation and update the current state to the next state s t+1 .

9. The method for switching between low-orbit satellite networks across beams based on a diffusion model according to claim 8, characterized in that: The step of dynamically deciding the optimal beam switching strategy for the user equipment model according to the future position distribution comprises the following steps: Update current status t , including the current beam ID, the channel quality indicator CQI, the current position of the user equipment, the current speed and the future position predicted by the diffusion model; Use the ò-greedy strategy to select action a t : Perform the selected action a t , and get the next state s t+1 , and the corresponding reward r t and whether to terminate the flag d t ; Update the experience replay pool to convert newly gained experience (s t ,a t ,r t ,s t+1 ,d t ) is added to the experience replay pool; Randomly sample a batch of samples from the experience replay pool for training and update the parameters of the Q network. The specific steps include: calculating the target value y t And the loss function L, and then update the parameters of the Q network through back propagation.

10. A low-orbit satellite network cross-beam switching system based on a diffusion model, characterized in that: The low-orbit satellite network cross-beam switching system based on a diffusion model comprises: a memory and a processor, wherein the memory comprises a program of a low-orbit satellite network cross-beam switching method based on a diffusion model, and when the program of the low-orbit satellite network cross-beam switching method based on a diffusion model is executed by the processor, the following steps are implemented: According to the application scenarios of low-orbit satellite networks, satellite network models, user equipment models, diffusion models and deep reinforcement learning network models are constructed; In the application scenario, when the user equipment model is in different beam coverages in the satellite network model at different time points, the user equipment model performs beam switching according to real-time channel conditions and satellite positions; The diffusion model predicts the future position distribution of the user equipment model, thereby determining the beam that needs to be switched in advance; The agent of the deep reinforcement learning network model dynamically decides the optimal beam switching strategy of the user equipment model according to the future position distribution, the user equipment model performs beam switching according to the optimal beam switching strategy and feeds back the result to the agent, and the agent updates the parameters of the deep reinforcement learning network model according to the feedback result.

Images