Method, apparatus and device for low earth orbit satellite beam hopping scheduling and medium

Abstract

The application discloses a low-orbit satellite skip beam scheduling method, device, equipment and medium, and the method comprises the steps of constructing satellite images, geographical positions and transmission channel information of each ground cell into multiple environment subsets, each environment subset containing multiple ground cells; for each environment subset, generating a training sample according to the satellite images, geographical positions and transmission channel information of the multiple ground cells contained in the environment subset, and storing the training sample into a generalization memory pool; the training sample contains a state space, an action space, a reward value and a next state space; updating a target network based on the generalization memory pool, and deploying the updated target network to a satellite system for predicting a beam service cell in each time slot. By combining the satellite images with deep reinforcement learning, adaptive beam scheduling under the condition that the distribution of cell service demand is unknown is realized.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and specifically to a method, apparatus, equipment, and medium for low-orbit satellite beam hopping scheduling. Background Technology

[0002] Low-Earth orbit (LEO) satellite communication, with its wide-area coverage, high spectral efficiency, and flexible networking capabilities, has become a key component in building a global communication system. In resource-constrained satellite-to-ground transmission scenarios, beam hopping technology provides communication services by dynamically adjusting beam direction, significantly improving system resource utilization and service quality, and is one of the core means to achieve efficient satellite communication. However, due to uneven distribution of terrestrial services and the fixed allocation of existing beam hopping schemes, problems such as resource waste, low system throughput, and large transmission delays arise. Summary of the Invention

[0003] The purpose of this invention is to address the shortcomings of the prior art by providing a method, apparatus, device, and medium for low-Earth orbit satellite beam hopping scheduling. This purpose is achieved through the following technical solutions.

[0004] A first aspect of the present invention provides a method for low-Earth orbit satellite beam hopping scheduling, the method comprising: The satellite imagery, geographic location, and transmission channel information of each ground cell are used to construct multiple environmental subsets, and each environmental subset contains multiple ground cells; For each environment subset, training samples are generated based on satellite images, geographical locations, and transmission channel information of multiple ground cells contained in the environment subset, and stored in the generalization memory pool; the training samples include a state space, an action space, a reward value, and a next state space; The target network is updated based on the generalized memory pool, and the updated target network is deployed to the satellite system for predicting beam serving cells in each time slot.

[0005] A second aspect of the present invention provides an apparatus for low-Earth orbit satellite beam hopping scheduling, the apparatus comprising: The subset construction module is used to construct multiple environment subsets from satellite imagery, geographic location, and transmission channel information of various ground cells, with each environment subset containing multiple ground cells; The sample generation module is used to generate training samples for each environment subset based on satellite images, geographical locations, and transmission channel information of multiple ground cells contained in the environment subset, and store them in the generalization memory pool; the training samples include a state space, an action space, a reward value, and a next state space; The network update module is used to update the target network based on the generalized memory pool, and deploy the updated target network to the satellite system for predicting beam service cells in each time slot.

[0006] A third aspect of the invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the method as described in the first aspect.

[0007] A fourth aspect of the invention provides a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the method as described in the first aspect.

[0008] Based on the above-described method, apparatus, equipment, and medium for low-Earth orbit satellite beam hopping scheduling, the present invention has the following beneficial effects or advantages: This application aims to improve satellite system throughput and reduce satellite transmission latency by combining satellite imagery with deep reinforcement learning to achieve adaptive beam scheduling under conditions of unknown cell service demand distribution. By dividing satellite imagery, geographical location, and transmission channel information of various surface cells covered by the satellite system into multiple environmental subsets to characterize various heterogeneous service distribution scenarios, and by constructing a generalized memory pool that aggregates experience samples from different service distribution scenarios, the deep learning network learns general decision-making rules during training, rather than being limited to the distribution characteristics of specific scenarios. This improves the network's decision-making stability and effectiveness under various real-world service demand distribution scenarios. Attached Figure Description

[0009] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a flowchart illustrating a method for low-Earth orbit satellite beam hopping scheduling according to an exemplary embodiment of the present invention; Figure 2 This is a schematic diagram of a system architecture according to an exemplary embodiment of the present invention; Figure 3 This is a hyperparameter-selective thermal distribution diagram according to an exemplary embodiment of the present invention; Figure 4 This is a comparison chart of training rewards for different algorithms according to an exemplary embodiment of the present invention; Figure 5 This is a performance comparison chart of different algorithms according to an exemplary embodiment of the present invention; Figure 6 This is a schematic diagram of a device structure for low-Earth orbit satellite beam hopping scheduling according to an exemplary embodiment of the present invention; Figure 7 This is a schematic diagram of the hardware structure of an electronic device according to an exemplary embodiment of the present invention. Detailed Implementation

[0010] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of systems and methods consistent with some aspects of the invention as detailed in the appended claims.

[0011] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0012] It should be understood that although the terms first, second, third, etc., may be used in this invention to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of this invention, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0013] Low Earth orbit (LEO) satellite communication, as a core technology for global wireless communication coverage, relies on efficient beam scheduling for its performance. Beam hopping, as one of the core methods for achieving efficient satellite communication, provides communication services to designated cells by dynamically adjusting beam direction. Therefore, an algorithm capable of making accurate beam hopping allocation decisions in resource-constrained satellite-to-ground transmission scenarios is fundamental to improving the performance of LEO satellite communication systems.

[0014] Currently, considering services in beam-hopping schemes is a crucial step in improving the performance of satellite communication systems. However, existing methods still have significant limitations. Although heuristic search strategies can improve system performance to some extent, they are computationally expensive and unsuitable for dynamic satellite scenarios. Furthermore, most existing research is based on the real-time sensing of ground service distribution, resulting in insufficient generalization ability of the models in real-world systems.

[0015] Based on this, the present invention proposes a low-orbit satellite beam hopping scheduling scheme to improve satellite system throughput and reduce satellite transmission latency. By combining satellite imagery with deep reinforcement learning, adaptive beam scheduling is achieved under conditions where the distribution of cell service demand is unknown.

[0016] The following detailed description of the low-orbit satellite hopping beam scheduling scheme proposed in this invention is illustrated with specific embodiments.

[0017] Figure 1 The present invention provides a flowchart of a method for low-Earth orbit satellite beam hopping scheduling according to an exemplary embodiment, comprising the following steps: Step 101: Construct multiple environment subsets from satellite images, geographical locations, and transmission channel information of each ground cell, with each environment subset containing multiple ground cells.

[0018] Step 102: For each environmental subset, generate training samples based on satellite images, geographical locations, and transmission channel information of multiple ground cells included in the environmental subset, and store them in the generalization memory pool; the training samples include state space, action space, reward value, and next state space.

[0019] Step 103: Update the target network based on the generalized memory pool, and deploy the updated target network to the satellite system for predicting beam serving cells in each time slot.

[0020] In step 101, a ground cell can be understood as a portion of the ground area covered by a satellite system. Typically, the coverage area is divided into multiple grid regions, each of which is a ground cell. Satellite imagery usually contains visual patterns such as building distribution, vegetation cover, and road networks, which can indirectly reflect the characteristics of the cell's service requirements. The latitude and longitude of the four corners of each ground cell can be used as its geographical location, and the transmission channel information reflects the characteristics of the wireless signal transmission between the satellite system and the ground cell. For example, assuming there are 1000 ground cells, these 1000 ground cells are divided into 10 environmental subsets, each containing 100 ground cells.

[0021] In step 102, samples for training the deep learning network are generated using satellite imagery, geographic location, and transmission channel information. In each training sample, the state space defines the environmental information perceived by the satellite agent (i.e., the deep learning network) in each time slot, including satellite imagery features, geographic location, and transmission channel information of the ground cells. The action space defines the beam scheduling actions that the satellite agent can take in each time slot, i.e., which ground cells to select for service. The reward value is used to evaluate the effectiveness of the beam scheduling actions taken by the satellite agent in each time slot. The design of the reward value needs to comprehensively consider multiple objectives such as system throughput, transmission latency, and service priority of cells with high service demand. The next state space represents the next state the environment will enter after the satellite agent takes the current action. The information in the next state space is crucial for the agent's continuous decision-making because it reflects the impact of the current action on the environment and possible future state changes.

[0022] A generalized memory pool is a data structure used to store experience samples from various heterogeneous business distribution scenarios. These samples include information such as state space, action space, reward value, and next state space.

[0023] In step 103, the target network is a deep Q-learning network, such as DDQN (Double Deep Q-Network) or DDPG (Deep Deterministic Policy Gradient). By periodically sampling samples from the generalization memory pool for training, the parameters of the target network can be continuously updated. The updated target network has stronger generalization ability and decision-making performance, and can more accurately predict the optimal beam serving cell for each time slot.

[0024] Therefore, deploying the updated target network into the satellite system enables real-time and efficient beam scheduling decisions, thereby improving the overall performance of the satellite communication system.

[0025] The embodiments of this application effectively solve the problems of service awareness and adaptive beam hopping scheduling in low-Earth orbit satellite communication by using satellite image features and a generalized memory pool design, significantly improving system performance and generalization capabilities.

[0026] In an optional embodiment, the process of generating training samples in step 102 above may include: High-dimensional image features of satellite images for each ground cell are extracted using a trained feature extraction network. The high-dimensional image features of satellite images of each ground cell are mapped to a low-dimensional latent space using a variational autoencoder to obtain the image features of each ground cell. Training samples are generated based on the image features, geographical location, and transmission channel information of each ground cell.

[0027] Among them, the feature extraction network, as the backbone network for feature extraction, can adopt deep convolutional neural networks, such as ResNet, Transformer, CNN and other models.

[0028] Considering that directly inputting high-dimensional image features into subsequent deep Q-learning networks would introduce significant computational complexity and the curse of dimensionality, a variational autoencoder (VAE) is introduced to map high-dimensional features to a compact and more generalizable low-dimensional latent space.

[0029] In this embodiment, image features are extracted from cell satellite images to capture high-level semantic information, such as the shape, texture, and spatial distribution of buildings. These features indirectly reflect the service requirements of the ground cell. By combining image features, geographic location, and transmission channel information, more comprehensive and accurate training samples can be generated. These samples not only include the visual features of the ground cell but also consider its spatial location and communication environment, thus providing richer decision-making basis for the deep reinforcement learning model. During training, the model can learn how to dynamically adjust beam pointing based on this information to optimize system throughput, reduce transmission latency, and meet the service priority of cells with high service demands.

[0030] In one optional embodiment, the process of generating training samples based on image features, geographic location, and transmission channel information of each ground cell may include: The current state space is generated based on the number of satellite beams N; the state space includes image features, geographical location, and transmission channel information of N ground cells. An action space is generated based on a preset decision network and the current state space; the decision network has the same structure as the target network, and the action space contains N cell numbers that require beam service. The reward value is calculated based on the throughput and latency of the action space performed by the satellite system. The next state space is generated based on the action space; Generate a training sample from the current state space, the action space, the reward value, and the next state space; The next state space is used as the current state space, and the steps of generating an action space based on a preset decision network and the current state space are continued to generate multiple training samples.

[0031] The state space contains the image features, geographical locations, and transmission channel information of N ground cells. This information together describes the service demand potential and communication environment of each ground cell, and together constitutes the environmental context that the satellite needs to process in the current time slot.

[0032] The decision network is a deep learning model (such as a deep Q-network) with a structure consistent with the target network. It is used to generate the action space based on the current state space. The action space contains N cell numbers that require beam service, representing the cells that the satellite should serve in the current time slot. While the decision and target networks are structurally consistent, the target network is typically used to generate the target Q-value during training to guide parameter updates in the decision network. This design helps stabilize the training process and improves model convergence.

[0033] After the satellite system executes its action space (i.e., selects a specific cell for service), it calculates a reward value based on the actual communication performance (such as throughput and latency). The reward value is used to evaluate the effectiveness of the current action and guide the decision-making network to make better decisions in the future.

[0034] After the satellite system performs its current action, the environment changes, leading to the next state. This next state space contains the image features, geographical location, and transmission channel information of the corresponding ground cells. This next state space provides the basis for continuous decision-making by the decision network, enabling it to dynamically adjust its beam scheduling strategy according to environmental changes.

[0035] Training samples are the basic units for training deep reinforcement learning models. They contain information about the environment state, the actions performed, the rewards obtained, and the next state. By continuously generating and storing training samples, a rich experience replay pool (i.e., a generalization memory pool) can be built to train the decision network, thereby improving the model's generalization ability and stability.

[0036] In an optional embodiment, the process of generating an action space based on a preset decision network and the current state space may include: Generate random numbers; If the random number is greater than a preset value, the current state space is input into the decision network to obtain the action space output by the decision network; If the random number is less than or equal to a preset value, N ground cells are randomly selected from the plurality of ground cells to generate an action space.

[0037] Random numbers can introduce randomness, which can control the switching ratio between the decision network and random exploration, and prevent the model from getting trapped in local optima.

[0038] In this embodiment, by comparing random numbers with preset values, the model dynamically switches between using known optimal strategies and exploring unknown strategies, preventing the model from getting stuck in suboptimal solutions.

[0039] In an alternative embodiment, the process of calculating the reward value for the throughput and latency of performing the action space using a satellite system may include: Calculate the weight of the service demand of the cell indicated by the action space relative to the total service demand; the total service demand is the sum of the service demands of all cells covered by the satellite system; The reward value is calculated based on the throughput, the latency, and the weight.

[0040] The action space contains the N cell numbers that the satellite needs to serve in the current time slot. The service demand of each cell typically refers to its data transmission requirements (such as user-requested traffic, bandwidth requirements, etc.), reflecting the communication urgency of the cell. Cells with higher demands should be prioritized for service. This can be dynamically estimated through historical data statistics, real-time monitoring, or predictive models (such as LSTM).

[0041] Throughput is the amount of data successfully transmitted by a satellite per unit of time (e.g., Mbps). It is typically calculated individually for each cell in the operational space and then aggregated according to weights into a system-level metric. High throughput means that satellite resources are used efficiently, meeting the needs of more users.

[0042] Latency is the transmission time (in milliseconds) of data from the sender to the receiver, including propagation latency, processing latency, and queuing latency. Latency is typically calculated individually for each cell in the action space and then aggregated according to weights. Low latency is crucial for real-time applications (such as voice and video) and is a key metric for user experience.

[0043] The formula for calculating the reward value is as follows:

[0044] In the above formula, t represents time slot t. This indicates the number of cells in the action space, which is also the number of satellite beams. Indicates the cell in time slot t The weight of business demand in the total business demand ensures the service priority of cells with high business demand. Indicates the cell in time slot t throughput size, express The normalized value, Indicates the cell in time slot t The time delay, , Indicates the cell in time slot t The business demand, Indicates the cell in time slot t The transmission rate.

[0045] In this embodiment, by using service demand weights in the reward calculation, the needs of different cells can be balanced. The reward value takes into account both throughput and latency, which can guide the model to learn multi-objective optimization strategies.

[0046] In an optional embodiment, the process of updating the target network based on the generalized memory pool in step 103 above may include: If the number of samples in the generalized memory pool is greater than or equal to the batch sampling number, then select several target samples from the generalized memory pool for the batch sampling. For each target sample, the target value is calculated using the next state space and reward value of the target sample; The predicted value is obtained by using the state space of the target sample and the decision network; The network parameters of the decision network are updated based on the mean squared error of the target value and the predicted value of each target sample. The network parameters of the target network are updated using the network parameters of the decision network.

[0047] The generalized memory pool is a data structure used to store samples of state-action-reward-next state quadruples collected during satellite beam hopping scheduling.

[0048] Batch size is the number of samples drawn from the memory pool during each training session. Once the memory pool has accumulated enough data, the network is updated to avoid ineffective training due to insufficient data in the early stages.

[0049] The target value is the ideal output that the model should achieve in the next state space, and this output is generated by the target network. The predicted value is the output of the decision network after executing the action space in the current state space, representing the model's estimate of the value of the current action.

[0050] Mean squared error (MSE) is the average of the squared differences between the target value and the predicted value, used to quantify the gap between the model's prediction and the ideal output.

[0051] The target network is a stable copy of the decision network, and its parameters are synchronized from the decision network through soft or hard updates.

[0052] In this embodiment, experience samples are stored in a generalization memory pool through satellite-environment interaction. By randomly sampling from the memory pool, target and predicted values ​​are calculated. The decision network parameters are updated using MSE loss. The decision network parameters are periodically synchronized to the target network to stabilize the training process.

[0053] In an alternative embodiment, the process of calculating the target value using the next state space and reward value of the target sample may include: Input the next state space into the target network; The target value is calculated using the action space and reward value output by the target network.

[0054] The next state space serves as the input to the target network and is used to predict the action space in the next state. This action space represents the optimal action in the next state.

[0055] The formula for calculating the target value is as follows: ) In the above formula, Indicates the reward value. Indicates the discount factor. ) represents the next state space The value estimate of the optimal action output by the target network after input. This represents the parameters of the target network.

[0056] Based on the description of the above embodiments, the low-Earth orbit satellite hopping beam scheduling scheme proposed in this application will be described in detail below, such as... Figure 2 The system architecture diagram shown mainly consists of two parts: satellite image feature extraction and a deep Q-learning network based on a generalized memory pool.

[0057] (1) Satellite image feature extraction First, a dataset of real satellite service cells in a specific region is collected. This dataset includes satellite images, latitude and longitude coordinates, and transmission channel information for each surface cell within the region. Multiple environment subsets are defined within the collected dataset: Subset 1…Subset I. To improve the accuracy of satellite image feature extraction, all satellite images undergo uniform preprocessing operations, including image size normalization and pixel value standardization. To effectively obtain high-level semantic information from satellite images, a ResNet-18 model pre-trained on a large image dataset is used as the backbone network for feature extraction. Its top classification layer is removed, and the output of the last pooling layer is used as the initial deep feature representation of the image. This feature contains high-level semantic information, such as visual patterns like building distribution, vegetation cover, and road networks, thus indirectly reflecting the characteristics of cell service requirements. Considering that directly inputting high-dimensional image features into subsequent deep Q-learning networks would introduce significant computational complexity and the curse of dimensionality, a variational autoencoder (VAE) is introduced to map high-dimensional features to a compact and more generalizable low-dimensional latent space.

[0058] (2) Deep Q-learning network based on generalized memory pool Beam hopping scheduling in low-Earth orbit satellite systems can be viewed as a multi-objective optimization problem, aiming to maximize system throughput and minimize transmission latency while prioritizing cells with high service demand. Its beam scheduling decision process can be described as a discrete-time Markov decision process, represented by a triplet of state space S, action space A, and reward function R, defined as follows: State space S: Because the satellite system cannot obtain real-time information on the distribution of ground service demands when performing beam scheduling, but it can globally perceive satellite imagery and latitude / longitude information of ground cells, the state space S can be defined as follows in each time slot t:

[0059] in, The image feature matrix of the N ground cells served by the satellite in time slot t, where N is the number of satellite beams. Given a geographic location matrix of N ground cells, This is the channel transmission information matrix for N ground cells.

[0060] Action Space A: In each time slot t, the satellite system needs to select N ground cells from the ground cells within its coverage area for service. To simplify the dimensions of the action space, cell numbers are used to construct the action space, representing the set of cells selected for service in time slot t. Considering that there is no specific correspondence between beams and cells, the cell numbers in the action space are not sequential, and are specifically described as follows: A=

[0061] Reward function R: To prioritize serving cells with high service demand while ensuring throughput and latency performance, the reward function for time slot t is designed as follows:

[0062] in, This indicates the number of cells in the action space, which is also the number of satellite beams. Indicates the cell in time slot t The weight of business demand in the total business demand ensures the service priority of cells with high business demand. Indicates the cell in time slot t throughput size, express The normalized value, Indicates the cell in time slot t The time delay, , Indicates the cell in time slot t The business demand, Indicates the cell in time slot t The transmission rate.

[0063] Based on this, the training process of a deep Q-learning network based on a generalized memory pool is as follows: 1. Initialization Phase: Initialize the parameters of the policy network. Parameters of the target network ,make Create a generalized memory pool and set its maximum capacity; initialize the deep reinforcement learning exploration rate ε, batch sampling size B, target network update interval H, and total number of training epochs. Each round has T steps; construct I subsets of environments. Each environmental subset contains C ground cells and N satellite beams.

[0064] 2. Iterative training phase Round loop (episode=1 to ) Traverse each environment subset (i=1 to I): Generate the state space (Initially from subset) (N ground cells are randomly selected for generation, and subsequent generation is based on the action space of the previous time slot t-1.) Step-by-step loop (t=1 to T) Action selection: Generate a random number rand. If rand > ε, use the policy network based on the input state space. Predict the Q value and select the top N cells to form the action space. If rand≤ε, randomly select N cells as the action space. The specific expression formed is as follows:

[0065] Environmental interaction: Satellite system execution Obtain throughput, latency, and weights, and calculate the reward. From the action space Generate the next state ; Experience storage: will Store the sample in the generalized memory pool; if the pool exceeds the maximum capacity, delete the oldest sample. Network update: If the number of samples in the generalization memory pool is ≥ B, randomly sample B samples and calculate the target Q value: ) and predicted Q value: Calculate the mean square error between the target Q value and the predicted Q value, and update the value based on the minimum mean square error. ; Target network update: Every H steps, let .

[0066] In this embodiment, cell service requirements are indirectly represented by easily obtainable satellite image features, without requiring real-time service data. This allows for adaptation to unknown service distribution scenarios and provides real-time beam scheduling decisions for satellites. By aggregating heterogeneous service distribution experience through a generalized memory pool, the overfitting problem of traditional reinforcement learning models is solved, enabling the satellite agent to learn a general scheduling strategy that transcends specific training distributions and possesses strong generalization capabilities.

[0067] To verify the performance of the aforementioned low-Earth orbit satellite beam scheduling algorithm, an exemplary simulation scenario was used, comprising a satellite with an orbital altitude of 550 km, equipped with six beams, each occupying a bandwidth of 100 MHz, a center frequency of 2 GHz, a transmit antenna gain of 35 dBi, and a receive antenna gain of 0 dBi. The satellite continuously covered all ground cells throughout the simulation, with each environmental subset consisting of 100 cells.

[0068] First, an analysis was conducted on the generalized memory pool capacity and the DQN target network update interval. The results are as follows: Figure 3 As shown in the heat map, the distribution of rewards obtained by the DQN target network under different parameter combinations is intuitively displayed. By comparing the maximum reward value corresponding to network convergence, it can be seen that the algorithm performance is optimal when the generalization memory pool capacity is 100,000 and the DQN target network update interval is 50.

[0069] Secondly, to quantify the contribution of satellite image features to the implicit representation of business requirements, the training reward curves of the algorithm of this invention were compared with those of the DQN baseline without image features. The comparison results are as follows: Figure 4 As shown, throughout the training process, the cumulative reward obtained by the algorithm of this invention is consistently significantly higher than that of the baseline model, while also achieving convergence close to the effect of a brute-force exploration approach. This indicates that image features provide more effective state information for the DQN target network, thereby improving decision quality. It is worth noting that the convergence process of the algorithm of this invention is relatively smooth, reflecting that, driven by the generalized memory pool, the algorithm continuously explores the complex mapping relationship between image features and business requirements, thereby learning a universal scheduling strategy, rather than relying on mechanical memorization of specific scenarios. In contrast, the DQN baseline without image features, lacking the assistance of image features, rapidly converges to a local optimum, with the obtained reward remaining at a low level, exhibiting obvious overfitting characteristics.

[0070] Then, to systematically evaluate the overall performance of the algorithm of this invention, it was compared with several baseline algorithms from two dimensions: reward and computational efficiency. The algorithms involved in the comparison were divided into two categories: first, particle swarm optimization and brute-force search algorithms under an idealized scenario with omniscient business distribution, serving as theoretical upper bound references; second, traditional DQN, DQN without image features, and random beam skipping algorithms under an unknown business distribution scenario, with the latter serving as a performance lower bound. The performance comparison results are as follows: Figure 5As shown, regarding rewards, the results of 30 rounds of testing in untrained regions demonstrate that the algorithm of this invention improves the average reward by approximately 30% compared to traditional DQN and DQN without image features, with a smaller fluctuation in the reward curve, exhibiting superior stability. The poor performance of traditional DQN is mainly due to the lack of a generalization memory pool mechanism during training, leading to overfitting of the model to limited scenarios and failing to effectively learn the implicit mapping between image features and business requirements, thus resulting in insufficient generalization ability in unknown environments. The random beam skipping algorithm, due to its unpredictable decision-making, can serve as a lower bound for the system. The algorithm of this invention shows significant advantages over it, further validating the rationality of the algorithm design. It is worth emphasizing that even under conditions of unknown business distribution, the algorithm of this invention still achieves approximately 90% of the performance of the particle swarm optimization algorithm under conditions of complete knowledge of business distribution. In terms of computational efficiency, the inference time of the algorithm of this invention during the inference phase is far lower than that of the particle swarm optimization algorithm, enabling rapid decision-making. The simulation results above, from both scheduling performance and time efficiency perspectives, confirm that the generalized memory pool and image feature representation play a key role in improving the algorithm's decision-making ability, enabling the algorithm to balance decision quality and speed even when business distribution information is unknown.

[0071] Corresponding to the aforementioned embodiments of the method for low-Earth orbit satellite beam hopping scheduling, this application also provides embodiments of an apparatus for low-Earth orbit satellite beam hopping scheduling.

[0072] Figure 6 This is a schematic diagram of a device for low-Earth orbit (LEO) satellite beam hopping scheduling according to an exemplary embodiment. This device is used to perform the LEO satellite beam hopping scheduling method provided in any of the above embodiments, such as... Figure 6 As shown, the device for low-Earth orbit satellite beam hopping scheduling includes: The subset construction module 310 is used to construct multiple environmental subsets from satellite images, geographical locations and transmission channel information of various ground cells, and each environmental subset contains multiple ground cells; The sample generation module 320 is used to generate training samples for each environmental subset based on satellite images, geographical locations and transmission channel information of multiple ground cells included in the environmental subset, and store them in the generalization memory pool; the training samples include a state space, an action space, a reward value, and a next state space; The network update module 330 is used to update the target network based on the generalized memory pool and deploy the updated target network to the satellite system for predicting beam service cells in each time slot.

[0073] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0074] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0075] Figure 7 This is a hardware structure diagram of an electronic device according to an exemplary embodiment of the present invention. The electronic device includes: a communication interface 401, a processor 402, a machine-readable storage medium 403, and a bus 404; wherein the communication interface 401, the processor 402, and the machine-readable storage medium 403 communicate with each other through the bus 404. The processor 402 can execute the low-Earth orbit satellite hopping beam scheduling method described above by reading and executing machine-executable instructions corresponding to the control logic of the low-Earth orbit satellite hopping beam scheduling method in the machine-readable storage medium 403. The specific content of the method is described in the above embodiment and will not be repeated here.

[0076] The machine-readable storage medium 403 mentioned in this invention can be any electronic, magnetic, optical, or other physical storage system that can contain or store information, such as executable instructions, data, etc. For example, the machine-readable storage medium can be volatile memory, non-volatile memory, or similar storage media. Specifically, the machine-readable storage medium 403 can be RAM (Random Access Memory), flash memory, storage drives (such as hard disk drives), any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0077] The present invention also provides a computer-readable storage medium corresponding to the low-Earth orbit satellite beam hopping scheduling method provided in the foregoing embodiments. The computer-readable storage medium is an optical disc, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it executes the low-Earth orbit satellite beam hopping scheduling method provided in any of the foregoing embodiments.

[0078] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0079] The computer-readable storage medium provided in the above embodiments of the present invention and the low-orbit satellite beam hopping scheduling method provided in the embodiments of the present invention are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0080] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.

[0081] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0082] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for low-Earth orbit satellite beam hopping scheduling, characterized in that, The method includes: The satellite imagery, geographic location, and transmission channel information of each ground cell are used to construct multiple environmental subsets, and each environmental subset contains multiple ground cells; For each environment subset, training samples are generated based on satellite images, geographical locations, and transmission channel information of multiple ground cells contained in the environment subset, and stored in the generalization memory pool; the training samples include a state space, an action space, a reward value, and a next state space; The target network is updated based on the generalized memory pool, and the updated target network is deployed to the satellite system for predicting beam serving cells in each time slot.

2. The method according to claim 1, characterized in that, The step of generating training samples based on satellite imagery, geographic location, and transmission channel information of multiple ground cells included in the environmental subset includes: High-dimensional image features of satellite images for each ground cell are extracted using a trained feature extraction network. The high-dimensional image features of satellite images of each ground cell are mapped to a low-dimensional latent space using a variational autoencoder to obtain the image features of each ground cell. Training samples are generated based on the image features, geographical location, and transmission channel information of each ground cell.

3. The method according to claim 2, characterized in that, The generation of training samples based on the image features, geographical location, and transmission channel information of each ground cell includes: The current state space is generated based on the number of satellite beams N; the state space includes image features, geographical location, and transmission channel information of N ground cells. An action space is generated based on a preset decision network and the current state space; the decision network has the same structure as the target network, and the action space contains N cell numbers that require beam service. The reward value is calculated based on the throughput and latency of the action space performed by the satellite system. The next state space is generated based on the action space; Generate a training sample from the current state space, the action space, the reward value, and the next state space; The next state space is used as the current state space, and the steps of generating an action space based on a preset decision network and the current state space are continued to generate multiple training samples.

4. The method according to claim 3, characterized in that, The generation of the action space based on the preset decision network and the current state space includes: Generate random numbers; If the random number is greater than a preset value, the current state space is input into the decision network to obtain the action space output by the decision network; If the random number is less than or equal to a preset value, N ground cells are randomly selected from the plurality of ground cells to generate an action space.

5. The method according to claim 3, characterized in that, The calculation of reward values ​​for throughput and latency in the action space using the satellite system includes: Calculate the weight of the service demand of the cell indicated by the action space relative to the total service demand; the total service demand is the sum of the service demands of all cells covered by the satellite system; The reward value is calculated based on the throughput, the latency, and the weight.

6. The method according to claim 3, characterized in that, The step of updating the target network based on the generalized memory pool includes: If the number of samples in the generalized memory pool is greater than or equal to the batch sampling number, then select several target samples from the generalized memory pool for the batch sampling. For each target sample, the target value is calculated using the next state space and reward value of the target sample; The predicted value is obtained by using the state space of the target sample and the decision network; The network parameters of the decision network are updated based on the mean squared error of the target value and the predicted value of each target sample. The network parameters of the target network are updated using the network parameters of the decision network.

7. The method according to claim 6, characterized in that, The step of calculating the target value using the next state space and reward value of the target sample includes: Input the next state space into the target network; The target value is calculated using the action space and reward value output by the target network.

8. A device for low-Earth orbit satellite beam hopping scheduling, characterized in that, The device includes: The subset construction module is used to construct multiple environment subsets from satellite imagery, geographic location, and transmission channel information of various ground cells, with each environment subset containing multiple ground cells; The sample generation module is used to generate training samples for each environment subset based on satellite images, geographical locations, and transmission channel information of multiple ground cells contained in the environment subset, and store them in the generalization memory pool; the training samples include a state space, an action space, a reward value, and a next state space; The network update module is used to update the target network based on the generalized memory pool, and deploy the updated target network to the satellite system for predicting beam service cells in each time slot.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is executed by a processor to implement the method as described in any one of claims 1-7.

Images