Satellite access method and method for training an agent

Abstract

Embodiments of the present application provide a satellite access method and a method for training an intelligent agent. It can be applied to the field of satellite communication technology. The method comprises: obtaining satellite access information, wherein the satellite access information comprises the number of terminals in the satellite coverage range, the distance information between the satellite and each terminal in the satellite coverage range, and the satellite-ground channel state information; dividing the terminals in the satellite coverage range into a plurality of device clusters according to the satellite access information, each device cluster comprising a plurality of terminals; sending the information of each device cluster to the terminals in each device cluster, the information of the device cluster being used by the terminals in the device cluster to determine a satellite access strategy of the terminals through an intelligent agent, wherein the parameters of the intelligent agent are obtained through centralized training in advance, and the satellite access strategy comprises the access power and access channel of the terminals. The method can cooperatively control the satellite access strategies of multiple terminals, thereby improving the success rate of multiple-terminal devices accessing a satellite.

Description

Technical Field

[0001] This application relates to the field of satellite communication technology, and in particular to a satellite access method and a training method for intelligent agents. Background Technology

[0002] With the development of communication technology, satellite communication networks, due to their low latency and wide coverage, have become an important infrastructure for future Internet of Things (IoT) communication. In areas where terrestrial base stations cannot provide continuous network coverage, satellites can enable large-scale, multi-device terminal device connections and communication network support.

[0003] With the surge in the number of terminals, existing satellite access methods lack coordinated control over access strategies among multiple terminal devices, leading to a decrease in satellite access success rate in high-density access scenarios.

[0004] Therefore, in scenarios involving multiple terminal devices accessing satellites, improving the success rate of satellite access for these devices has become an urgent problem to be solved. Summary of the Invention

[0005] This application provides a satellite access method and an agent training method to improve the success rate of multi-terminal devices accessing satellites.

[0006] In a first aspect, embodiments of this application provide a satellite access method applied to a satellite, the method comprising:

[0007] Obtain satellite access information, wherein the satellite access information includes the number of terminals within the satellite's coverage area, the distance information between the satellite and each terminal within the satellite's coverage area, and the satellite-to-ground channel status information;

[0008] Based on the satellite access information, the terminals within the satellite coverage area are divided into multiple device clusters, and each device cluster includes multiple terminals;

[0009] Information from each of the device clusters is sent to the terminals within each device cluster. This information is used by the terminals within the device cluster to determine their satellite access strategies through an intelligent agent. The parameters of the intelligent agent are obtained in advance through centralized training. The satellite access strategy includes the terminal's access power and access channel.

[0010] In one possible implementation, dividing the terminals within the satellite coverage area into multiple device clusters based on the satellite access information includes:

[0011] The channel gain of each terminal within the satellite coverage area is determined based on the satellite access information;

[0012] The terminals within the satellite coverage area are grouped according to the channel gain to obtain multiple device groups, and each device group includes multiple terminals;

[0013] One terminal is determined from each of the multiple device groups to obtain a device cluster. This step is repeated to obtain multiple device clusters.

[0014] In one possible implementation, grouping the terminals within the satellite coverage area according to the channel gain to obtain multiple device groups includes:

[0015] Terminals within the satellite coverage area are sorted according to the channel gain. Multiple consecutive terminals are grouped into a device group according to the sorting, resulting in multiple device groups. The number of device groups is the same as the number of terminals in the device cluster, and the number of terminals in each device group is the same.

[0016] In one possible implementation, determining one terminal from each of the plurality of device groups to obtain a device cluster includes:

[0017] Determine the channel gain difference matching coefficient between the two terminals;

[0018] The device cluster is obtained by selecting one terminal from each of the multiple device groups with the goal of maximizing the minimum channel gain difference matching coefficient among the terminals in the device cluster.

[0019] Secondly, embodiments of this application provide a satellite access method applied to a terminal, wherein the terminal is equipped with an intelligent agent that has been pre-trained in a centralized manner, and the method includes:

[0020] The terminal receives information about the device cluster to which it belongs, wherein the information about the device cluster is determined based on satellite access information, and the satellite access information includes the number of terminals within the satellite coverage area, the distance information between the satellite and each terminal within the satellite coverage area, and the satellite-to-ground channel status information.

[0021] The intelligent agent determines the satellite access strategy of the terminal based on the terminal's state and the information of the device cluster. The satellite access strategy includes the terminal's access power and access channel.

[0022] In one possible implementation, the state of the terminal includes the action performed by the terminal at the previous moment and the observed variables at the previous moment. The action performed includes the terminal not transmitting or the terminal connecting to the satellite. The observed variables include the channel gain of the terminal, the distance between the terminal and the satellite, and the acknowledgment signal of the satellite.

[0023] Thirdly, embodiments of this application provide a method for training an intelligent agent, applied to a ground gateway station, the method comprising:

[0024] Receive information from multiple device clusters, wherein each device cluster includes multiple terminals, and the information of the device clusters is determined based on satellite access information, which includes the number of terminals within the satellite coverage area, the distance information between the satellite and each terminal within the satellite coverage area, and satellite-to-ground channel status information;

[0025] Based on the information of the multiple device clusters and the state of the terminals in each device cluster, the intelligent agent is trained, and the parameters of the trained intelligent agent are sent to each terminal.

[0026] Fourthly, embodiments of this application provide a satellite access device, the device comprising:

[0027] The information acquisition module is used to acquire satellite access information, wherein the satellite access information includes the number of terminals within the satellite coverage area, the distance information between the satellite and each terminal within the satellite coverage area, and the satellite-to-ground channel status information;

[0028] The terminal segmentation module is used to segment the terminals within the satellite coverage area into multiple device clusters based on the satellite access information, wherein each device cluster includes multiple terminals;

[0029] A strategy determination module is used to send information of each device cluster to the terminal in each device cluster. The information of the device cluster is used by the terminal in the device cluster to determine the satellite access strategy of the terminal through an intelligent agent. The parameters of the intelligent agent are obtained in advance through centralized training. The satellite access strategy includes the access power and access channel of the terminal.

[0030] In one possible implementation, the terminal segmentation module is further configured to:

[0031] The channel gain of each terminal within the satellite coverage area is determined based on the satellite access information;

[0032] The terminals within the satellite coverage area are grouped according to the channel gain to obtain multiple device groups, and each device group includes multiple terminals;

[0033] One terminal is determined from each of the multiple device groups to obtain a device cluster. This step is repeated to obtain multiple device clusters.

[0034] In one possible implementation, the terminal segmentation module is further configured to:

[0035] Terminals within the satellite coverage area are sorted according to the channel gain. Multiple consecutive terminals are grouped into a device group according to the sorting, resulting in multiple device groups. The number of device groups is the same as the number of terminals in the device cluster, and the number of terminals in each device group is the same.

[0036] In one possible implementation, the terminal segmentation module is further configured to:

[0037] Determine the channel gain difference matching coefficient between the two terminals;

[0038] The device cluster is obtained by selecting one terminal from each of the multiple device groups with the goal of maximizing the minimum channel gain difference matching coefficient among the terminals in the device cluster.

[0039] Fifthly, embodiments of this application provide a satellite access device, the device comprising:

[0040] The information receiving module is used to receive information about the device cluster to which the terminal belongs. The information about the device cluster is determined based on satellite access information, which includes the number of terminals within the satellite coverage area, the distance between the satellite and each terminal within the satellite coverage area, and the satellite-to-ground channel status information.

[0041] An access strategy determination module is used by the intelligent agent to determine the satellite access strategy of the terminal based on the state of the terminal and the information of the device cluster. The satellite access strategy includes the access power and access channel of the terminal.

[0042] Sixthly, embodiments of this application provide a satellite access device, including: a memory and a processor;

[0043] The memory stores computer-executed instructions;

[0044] The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of the first or second aspects above.

[0045] In a seventh aspect, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of the first or second aspects above.

[0046] Eighthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method as described in any one of the first or second aspects above.

[0047] The satellite access method and agent training method provided in this application, on the satellite side, divide the terminals within the satellite coverage area into multiple device clusters based on the number of terminals within the satellite coverage area, the distance information between the satellite and each terminal within the satellite coverage area, and the satellite-to-ground channel status information, resulting in multiple independent small-scale GF-NOMA subsystems. The device cluster information is sent to each terminal via satellite. The ground gateway station forwards the parameters of the agent, which has been pre-trained in a centralized manner, to the terminal via satellite. Within each terminal, the agent determines its satellite access strategy based on the device cluster information, enabling autonomous terminal access decisions within the subsystem. This eliminates the need for centralized satellite scheduling and authorization processes, reduces uplink signaling interaction, and in scenarios with large-scale terminal access, eliminates the need to wait for resource allocation, reduces the probability of access collisions, and improves the satellite access success rate. Attached Figure Description

[0048] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0049] Figure 1 This is a schematic diagram illustrating an application scenario of a satellite access method provided in this application;

[0050] Figure 2 A flowchart illustrating a satellite access method provided in this application. Figure 1 ;

[0051] Figure 3 A flowchart illustrating a satellite access method provided in this application. Figure 2 ;

[0052] Figure 4 A flowchart illustrating a satellite access method provided in this application. Figure 3 ;

[0053] Figure 5 A flowchart illustrating a training method for an intelligent agent provided in this application;

[0054] Figure 6 A comparison chart of training curves between a training method for an intelligent agent provided in this application and related technologies;

[0055] Figure 7 A comparison chart of the success probability of satellite access in a satellite access method and related technologies provided in this application;

[0056] Figure 8 A comparison chart of average throughput per device cluster in a satellite access method and related technologies provided in this application;

[0057] Figure 9 A schematic diagram of the structure of a satellite access device provided in this application Figure 1 ;

[0058] Figure 10 A schematic diagram of the structure of a satellite access device provided in this application Figure 2 ;

[0059] Figure 11 A schematic diagram of the structure of a training device for an intelligent agent provided in this application;

[0060] Figure 12 A schematic diagram of the satellite access equipment provided in this application.

[0061] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0062] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0063] Satellite spectrum resources are limited. To achieve large-scale access for IoT terminal devices, the Grant-Free Non-Orthogonal Multiple Access (GF-NOMA) method can be used. This technology differentiates terminals by uplink power, allowing multiple ground terminals to share the same time-frequency radio resources to complete satellite access. While this method reduces signaling overhead, it does not consider interference coordination between terminals, leading to a significant increase in the probability of collisions in high-density access scenarios, and a gradual decrease in access success rate as the number of terminals increases.

[0064] To address the aforementioned technical problems, this application proposes a satellite access method. On the satellite side, multiple terminals within the satellite's coverage area are divided into multiple device clusters, and device cluster information is sent to the terminals. On the terminal side, each terminal deploys a pre-trained, centrally trained agent. The agent deployed on each device determines the terminal's satellite access strategy based on the device cluster information and the terminal's status. The agents deployed on each terminal are centrally trained at a ground gateway station. The ground gateway station forwards the trained agent parameters to each terminal via satellite.

[0065] Figure 1This is a schematic diagram illustrating an application scenario of a satellite access method provided in this application, such as... Figure 1 As shown, the specific application scenario of this application is a satellite network where a large number of terminals access the satellite via an uplink unlicensed random access method. The satellite is equipped with a phased array antenna, which can change its direction to cover a specific area within a certain time period, covering multiple terminals within that area. The ground gateway station trains the agent based on device cluster information and satellite-terminal interaction information, and transmits the trained agent's parameters to each terminal via the satellite. The satellite divides the multiple terminals within its coverage area into multiple device clusters and sends the device cluster information to each terminal. The agent in each terminal determines its satellite access strategy based on the device cluster information it belongs to.

[0066] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0067] Figure 2 A flowchart illustrating a satellite access method provided in this application. Figure 1 This method is applied to satellites, such as Figure 2 As shown, the method includes:

[0068] S201. Obtain satellite access information.

[0069] The satellite access information includes the number of terminals within the satellite's coverage area, the distance between the satellite and each terminal within the satellite's coverage area, and the satellite-to-ground channel status information.

[0070] The satellite is equipped with a phased array antenna. By controlling the beam direction, the satellite beam can stably cover a fixed ground area for a certain period of time. The satellite moves around the Earth at high speed, and the beam coverage area changes dynamically with the orbit. This step only collects data from all terminals within the current beam coverage area. The number of terminals within the satellite's coverage area refers to the total number of all ground IoT terminals within the satellite beam coverage area that are waiting to access the uplink.

[0071] For example, the distance information between a satellite and each terminal within its coverage area can be calculated using the distance formula between two points by using the satellite's own three-dimensional spatial coordinates and the ground terminal's latitude, longitude, and elevation.

[0072] Satellite-to-ground channel state information is used to describe the complete quality of the wireless transmission link between the satellite and a single terminal. It can include large-scale fading, small-scale fading, and the bandwidth of the satellite-to-ground communication system.

[0073] In this embodiment, acquiring satellite access information provides a data foundation for subsequent device cluster segmentation. Timely updates to satellite access information can reduce link fluctuations caused by satellite orbital motion, providing data support for dynamically adjusting satellite access strategies.

[0074] S202. Based on satellite access information, the terminals within the satellite coverage area are divided into multiple device clusters. Each device cluster includes multiple terminals.

[0075] In this step, the distribution status of terminals or satellite-to-ground link conditions within the satellite coverage area are determined based on satellite access information. The terminals within the satellite coverage area are then grouped according to the relevant information to obtain multiple device clusters.

[0076] A device cluster is a collection of terminals formed by grouping them together. By dividing devices into clusters, terminals within the satellite coverage area can be organized and managed, providing constraints for determining the satellite access strategy for each terminal.

[0077] In this embodiment, each device cluster is an independent GF-NOMA subsystem. Multiple terminals within a device cluster share the same set of time-frequency resources, distinguished by power domain. Terminals in each GF-NOMA subsystem can directly transmit data to the satellite without requesting resources or waiting for scheduling authorization signals. Multiple terminals within a device cluster are distinguished by differences in transmit power when transmitting data to the satellite. The satellite employs a Successive Interference Cancellation (SIC) scheme to decode and remove interference sequentially from high to low received power, enabling concurrent transmission of multiple users using the same resources.

[0078] By dividing the equipment into clusters, each cluster contains multiple terminals with distinct channel characteristics. This allows the satellite to successfully distinguish information from multiple terminals during SiC decoding, reducing the decoding failure rate. Since multiple terminals within a cluster function as an independent GF-NOMA subsystem, power coordination is only required within a single cluster, eliminating the need for cross-cluster joint optimization and reducing computational overhead.

[0079] S203. Send the information of each device cluster to the terminal in each device cluster.

[0080] The information of the device cluster is used by the terminals in the device cluster to determine the satellite access strategy of the terminals through the intelligent agent. The parameters of the intelligent agent are obtained in advance through centralized training. The satellite access strategy includes the terminal's access power and access channel.

[0081] In this step, the information of the device cluster may include the device cluster identifier, terminal identifier, and the order of each terminal. This device cluster information can be distributed by the satellite to multiple terminals within the coverage area. The parameters of the agents in the terminals are pre-trained centrally by the ground gateway station, then transmitted from the ground gateway station to the satellite, which in turn forwards the parameters of each agent to each terminal.

[0082] Optionally, the information of the device clusters can be periodically updated via satellite to adapt to changes in satellite high-speed movement, satellite-to-ground distance information, and satellite-to-ground channel status information. For example, the time threshold for updating device clusters can be set to 100 time slots, with device clusters re-divided every 100 time slots, and the device cluster information sent to the terminals in each device cluster.

[0083] The information of the device cluster is sent to the terminals in each device cluster, so that each device determines the satellite access strategy according to the device cluster. When the intelligent agent autonomously selects the access power or access channel, it will refer to the channel differences within the cluster and actively increase the distinguishability of the receiving power to reduce the interference of satellite reception.

[0084] The satellite access method provided in this application divides the terminals within the satellite coverage area into multiple device clusters based on the number of terminals within the satellite coverage area, the distance information between the satellite and each terminal within the satellite coverage area, and the satellite-to-ground channel status information, resulting in multiple independent small-scale GF-NOMA subsystems. The device cluster information is transmitted to each terminal via satellite. The ground gateway station forwards the parameters of the pre-centrally trained agent to the terminal via satellite. Within each terminal, the agent determines its satellite access strategy based on the device cluster information. This enables autonomous decision-making for terminal access within the subsystem, eliminating the need for centralized satellite scheduling and authorization processes, reducing uplink signaling interactions, and, in scenarios with large-scale terminal access, eliminating the need to wait for resource allocation, reducing the probability of access collisions, and improving the satellite access success rate.

[0085] Figure 3 A flowchart illustrating a satellite access method provided in this application. Figure 2 ,like Figure 3 As shown, in this embodiment... Figure 2 Based on the embodiments, the method of dividing device clusters is described in detail, including:

[0086] S301. Determine the channel gain of each terminal within the satellite coverage area based on the satellite access information.

[0087] Channel gain is a quantitative indicator that characterizes the transmission capability of the wireless link between satellite and ground terminal. The larger the value, the smaller the link loss and the better the communication quality.

[0088] In this embodiment, the channel between the satellite and each terminal can be regarded as a line-of-sight channel. When the terminal uses the GF-NOMA scheme to access the satellite, the system bandwidth W can be divided into K sub-channels at equal intervals. Therefore, the channel gain of each terminal when accessing the satellite can be expressed by the following formula:

[0089] (1)

[0090] in, The channel gain is the value of the nth terminal when it accesses the satellite through the kth sub-channel. This is a small-scale fading phenomenon that follows a Rayleigh distribution with a Rayleigh factor of 1. ; For large-scale fading, it can be obtained using the free space loss formula.

[0091] Small-scale fading refers to the phenomenon where electromagnetic waves emitted by a terminal reach a satellite via multiple paths after being reflected or scattered by mountains, sea surfaces, or vegetation. These multiple paths of signals overlap and interfere with each other, causing rapid and drastic fluctuations in the received signal power over a short period of time.

[0092] Large-scale fading is the power attenuation caused by the long-distance propagation of electromagnetic waves in unobstructed line-of-sight scenarios. It is dominated by the satellite-to-ground distance and free-space propagation loss. The farther the satellite is from the terminal, the greater the large-scale loss and the lower the channel gain.

[0093] S302. Group the terminals within the satellite coverage area according to the channel gain to obtain multiple device groups. Each device group includes multiple terminals.

[0094] Grouping terminals based on channel gain can reduce the computational cost of dividing device clusters. If the terminals are not pre-grouped and the satellite access information of multiple terminals is directly solved to divide the device clusters, the computational load is too large and it is not suitable for application scenarios where satellite computing power is limited.

[0095] As one implementation method, terminals within the satellite coverage area can be grouped according to channel gain. This includes: sorting the terminals within the satellite coverage area according to the magnitude of the channel gain, and dividing multiple consecutive terminals into a device group according to the sorting to obtain multiple device groups. The number of device groups is the same as the number of terminals in the device cluster, and the number of terminals in each device group is the same.

[0096] For example, each device group is configured to include Terminals, comprising a total of There are several equipment groups, among which... The number of terminals within the satellite coverage area is less than or equal to the number of terminals within the satellite coverage area. Given a fixed channel gain for each terminal, the terminals within the satellite coverage area are sorted in descending order of channel gain, and the top terminals are selected. The terminals are divided into several categories in sequence. A group of equipment, denoted as Each device group contains For each terminal, the channel gain of the devices within these groups decreases as the index increases. For example, This indicates a group of devices with strong channel gain. This indicates a group of devices with weak channel gain.

[0097] S303. Determine one terminal from each of the multiple device groups to obtain a device cluster. Repeat this step to obtain multiple device clusters.

[0098] In this step, a terminal is selected from each device group to form a device cluster. The selected terminal is then removed from the candidate list of each device group. This cross-group selection operation is repeated until no terminals remain in any device group, at which point the iteration terminates.

[0099] One implementation method is to select one terminal from each of multiple device groups to obtain a device cluster, with the goal of calculating the minimum channel gain difference matching coefficient between terminals. Thus, determining one terminal from each of multiple device groups to obtain a device cluster includes: determining the channel gain difference matching coefficient between two terminals; and selecting one terminal from each of multiple device groups to obtain a device cluster with the goal of maximizing the minimum channel gain difference matching coefficient between terminals in the device cluster.

[0100] definition Let be a binary matching variable indicating whether terminals are assigned to the same device cluster. The minimum channel gain difference matching coefficient between two terminals can be calculated using the following formula:

[0101] (2)

[0102] in, This refers to the minimum channel gain difference matching coefficient among terminals in multiple device groups. For the first Terminal in the equipment group Channel gain, For the first Terminal in the equipment group Channel gain.

[0103] The matching coefficient that maximizes the minimum channel gain difference between terminals in a device cluster can be expressed by the following constraint:

[0104] (3)

[0105] The objective of solving this constraint is to maximize the minimum channel gain difference matching coefficient. In this step, it is necessary to determine that each terminal in the device cluster comes from a different device group, and there are multiple terminals. It was assigned to a cluster of devices.

[0106] For example, solving this constraint can be viewed as an integer programming problem, which can be solved using an open-source linear and mixed integer programming solver (Coin-or Branch and Cut Solver, CBC Solver) to obtain the allocation result of the device cluster.

[0107] First, the terminals are grouped according to the channel gain. Then, based on the grouping results, a problem is constructed to maximize the minimum channel gain difference matching coefficient. Terminals are selected from the device groups to form device clusters, ensuring that there are both high-gain strong link terminals and low-gain weak link terminals within the device cluster. The channel gain difference within the cluster remains stable at a high level, thereby improving the satellite access success rate.

[0108] Figure 4 A flowchart illustrating a satellite access method provided in this application. Figure 3 This method is applied to terminals, which are equipped with pre-trained, centralized agents, such as... Figure 4 As shown, the method includes:

[0109] S401, Receive information about the device cluster to which the terminal belongs.

[0110] The information about the device cluster is determined based on satellite access information, which includes the number of terminals within the satellite coverage area, the distance between the satellite and each terminal within the satellite coverage area, and the satellite-to-ground channel status information.

[0111] In this step, device cluster information is sent from the satellite to the terminal. The terminal can determine its device cluster and its order within the cluster based on this information.

[0112] S402. The intelligent agent determines the satellite access strategy of the terminal based on the terminal's status and the information of the device cluster.

[0113] Satellite access strategies include the terminal's access power and access channel.

[0114] In this step, the terminal's access power can be used It means that, among them, Let be the set of terminal access power within the device cluster at time t, and let n represent the nth terminal within the device cluster. This refers to the number of terminals within a single device cluster. The terminal's access channel can be... It means that, among them, This represents the number of sub-channels to which the current device cluster belongs. This means that terminal n is assigned to sub-channel k, and vice versa. .

[0115] In one implementation, the terminal's state includes the terminal's actions performed in the previous moment and the observed variables in the previous moment. The actions performed include the terminal not transmitting or the terminal connecting to the satellite. The observed variables include the terminal's channel gain, the distance between the terminal and the satellite, and the satellite's acknowledgment signal.

[0116] For example, define the action space It is used to represent the access power and access channel of terminal n. Specifically, the access power set is defined as follows: The access channel set is In the action space, 0 represents that the terminal's access strategy is an access power of The access channel is ;1 indicates that the terminal's access strategy is an access power of The access channel is And so on, The access strategy representing the terminal is that the access power is The access channel is .

[0117] definition Let be the action performed by terminal n at time t, where This means that terminal n does not transmit at time t. Let terminal n select the m-th power level and allocate the k-th sub-channel to access the satellite. Then the observed variables of terminal n are: ,in, Indicates that terminal n passes through the first Channel gain of sub-channel access satellite, Let n be the distance between terminal n and the satellite. This is a confirmation response signal from the satellite. This means that terminal n successfully accessed the satellite at time t; otherwise, .

[0118] The terminal state is determined based on the actions performed by the terminal in the previous moment and the observed variables from the previous moment. The current state of the terminal is input into the agent, which then determines the terminal's access power. and access channel .

[0119] The satellite decodes access signals transmitted by multiple terminals within the device cluster using the SIC decoding algorithm, prioritizing those with the highest received power to those with the lowest. In this embodiment, the success of uplink access signals transmitted by a terminal according to its access strategy can be determined by the received signal-to-interference-plus-noise ratio (SINR) and the terminal's access power. Specifically, when the satellite receives uplink access signals from multiple terminals, if the uplink access signal from the terminal with the highest received power has been decoded and the SINR is high, the satellite can determine whether the uplink access signal transmitted by the terminal according to its access strategy can be successfully transmitted. Greater than the threshold Then the signal from terminal n can be successfully decoded. Meanwhile, due to the limitation on the transmission power of the terminal, the maximum transmission power of the IoT terminal is defined as... Under the GF-NOMA mechanism, the method used to determine if the terminal's uplink access signal has been successfully transmitted can be represented as:

[0120] (4)

[0121] in, This is used to indicate whether the access strategy of terminal n can successfully access the network. Let n be the access power of terminal n. Let n be the received signal-to-interference-plus-noise ratio (SIR) of terminal n on sub-channel k. This is the preset received signal-to-interference-plus-noise ratio (SINNR) threshold.

[0122] The received signal-to-interference-plus-noise ratio (SINNR) of terminal n accessing the satellite via sub-channel k can be expressed as:

[0123] (5)

[0124] in, Let n be the received signal-to-interference-plus-noise ratio (SIR) of terminal n accessing the satellite via sub-channel k. Let r be the satellite access power of terminal n via sub-channel k, and r be the current ranking of the terminal in the device cluster. The summation is the total of the interference terms, with the starting point being r+1. This means that the terminals corresponding to the first r-1 terms have stronger access power and will not interfere with the r-th terminal. The power spectral density represents additive white Gaussian noise. This refers to the sub-channel bandwidth.

[0125] In this embodiment, the satellite access strategy is generated by the terminal's local intelligent agent, eliminating the need for real-time scheduling signaling from the satellite. This saves on resource authorization interaction processes, significantly reduces latency for massive terminal access, and minimizes uplink and downlink control signaling overhead. The satellite receiver only needs to handle interference from multiple terminals within a single cluster, thereby greatly reducing decoding computational power consumption.

[0126] Figure 5 This is a flowchart illustrating a training method for an intelligent agent provided in this application. This method is applied to a ground gateway station, such as... Figure 5 As shown, the method includes:

[0127] S501: Receive information from multiple device clusters.

[0128] Each device cluster includes multiple terminals. The information of the device cluster is determined based on satellite access information, which includes the number of terminals within the satellite coverage area, the distance between the satellite and each terminal within the satellite coverage area, and the satellite-to-ground channel status information.

[0129] S502. Based on information from multiple device clusters and the status of terminals in each device cluster, train the intelligent agent and send the parameters of the trained intelligent agent to each terminal.

[0130] In this step, information about multiple device clusters and the status of terminals within each cluster are transmitted from the satellite to the ground gateway station. The status of terminals within a device cluster includes historical interaction information between the satellite and the terminals, and this interaction information includes state variables. Action variables and rewards .

[0131] At the ground gateway station, historical interaction data between the satellite and the terminal is used as training samples. A centralized training node trains the agents corresponding to each terminal using the Multi-Agent Proximal Policy Optimization (MAPPO) algorithm. During training, importance sampling is used to improve sample utilization and accelerate convergence. In the agent training phase, the policy network parameters corresponding to the agent that records past data are... Introducing the clipping function Subsequently, the policy gradient update during the training phase can be expressed as:

[0132] (6)

[0133] in, To maximize the objective function; Represents the current policy network All trainable parameters are used, and the training process relies on gradient ascent to update the parameters. This represents the mathematical expectation of state s. Indicates action According to the old strategy Sampling and calculating the expectation; The importance sampling ratio is used to measure the difference between the old and new strategies. Under the new strategy, the state Execute action The probability of, under the old strategy, This represents the probability of performing the same action in the same state under the old strategy.

[0134] Clipping function Used to force the probability ratio to be limited to a range. This limits the magnitude of a single policy update and prevents the policy from oscillating drastically.

[0135] As the dominant function, in this embodiment, This indicates that the current action can increase the number of successful long-term accesses; This indicates that the current action is more likely to cause decoding failure and no ACK feedback.

[0136] In this embodiment, the network structure corresponding to the multi-agent near-end policy optimization method during the training phase includes multiple independent policy networks and value networks. Each independent policy network corresponds to one agent. For the policy network, a long short-term memory layer is used to set the time step D to extract time-related features from the satellite trajectory. Considering the dynamic time-varying characteristics of the satellite, historical terminal states can be used to assist in the selection of access power and access channel. For this purpose, two fully connected layers are designed to process hidden layer information and extract time-related features to support subsequent action decisions. For the policy network output layer, a branch output structure is designed, with the output layer dimension consistent with the action space dimension. The output layer contains two parallel Softmax layers, outputting the probability distribution of the action space, which corresponds to the access power selection variable. With channel selection variables The branch output structure decouples the joint action space into two independent access power selection subspace and access channel selection subspace, which helps to accelerate the algorithm training and convergence speed.

[0137] In a value network, the effectiveness of the current satellite access strategy is evaluated by defining a utility function and a reward function. The utility function can be defined as:

[0138] (7)

[0139] in, This is a utility function, which takes the value 1 or -1. When the access strategy can successfully access the satellite, the utility function is mapped to 1; when the access strategy cannot access the satellite, its value is mapped to -1.

[0140] The reward function can be defined as:

[0141] (8)

[0142] in, This is the value of the reward function, used to statistically analyze the success rate of satellite access for terminals within the device cluster under the current access policy.

[0143] After the agent's training is complete, the agent's parameters are sent from the ground gateway station to the satellite, which then forwards the parameters to the corresponding terminal. Training the agent at the ground gateway station separates the training and inference phases. All training computations are completed at the ground gateway station, while the satellite is only responsible for forwarding interactive data and distributing agent parameters. The terminal only performs lightweight forward inference and does not need to carry training computing power, significantly reducing the cost and battery life of both satellite and terminal hardware.

[0144] The agent training method provided in this application adopts a centralized training and decentralized execution framework. The ground gateway station serves as the training node, with a fixed location and communication with the satellite via a feed link. The training process employs an offline pre-training combined with periodic online updates: First, the ground gateway station completes offline pre-training of the policy network and value network based on historical interaction data between the satellite and the terminal. Then, the agents deployed in the terminals within each device cluster interact with the satellite. The interaction data is collected by the satellite and forwarded to the ground gateway station via the feed link, where the agent's periodic iterative training is completed. After training, the ground gateway station sends the updated agent network parameters to the satellite, which then sends the network parameters to each terminal. The terminal runs the agent based on the received network parameters, outputs a satellite access policy, and accesses the satellite according to the policy.

[0145] Figure 6 A comparison chart of training curves for the intelligent agent training method provided in this application and related technologies.

[0146] like Figure 6 As shown, the agent training method based on the MAPPO algorithm provided in this application can fully learn the access power and access channel selection strategy from historical interaction data, thereby maintaining a high access success rate. In contrast, related technologies using the Double Deep Q-Network (DDQN) algorithm for agent training suffer from slow convergence speed and large variance in rewards during convergence. Figure 6 As can be seen from the curves, the reward curve of the MAPPO algorithm during the convergence process is more stable, and it has a stronger generalization ability in access channel and access power selection tasks, which is better than the DDQN algorithm.

[0147] Figure 7 A comparison chart showing the success probability of satellite access in a satellite access method and related technologies provided in this application.

[0148] like Figure 7 As shown in the figure, one (…) is configured within each single device cluster. ) and 2 ( The success rate of each algorithm in the sub-channel is shown. It can be seen that as the number of terminals in a single device cluster increases, the success rate of all algorithms decreases. Moreover, the satellite access method provided in this application, based on the MAPPO algorithm, achieves a higher success rate than the DDQN algorithm and the ALOHA-Non-Orthogonal Multiple Access (ALOHA-NOMA) algorithm.

[0149] Figure 8 A comparison chart of the average throughput of a single device cluster in a satellite access method and related technologies provided in this application.

[0150] like Figure 8 As shown in the figure, the impact of the SINR threshold on the average throughput of a single device cluster is compared. The higher the SINR threshold, the greater the transmit power required by the terminals within the device cluster. Due to the constraint of the maximum transmit power of the terminals, some high-power levels cannot be reached, and the available operational space is reduced accordingly. Therefore, as the signal-to-interference-plus-noise ratio (SINR) threshold increases, both the average throughput of a single device cluster and the probability of successful access will decrease slightly. However, the satellite access method provided in this application, based on the MAPPO algorithm, achieves better satellite access performance than the DDQN and ALOHA-NOMA algorithms.

[0151] Figure 9 A schematic diagram of the structure of a satellite access device provided in this application Figure 1 ,like Figure 9 As shown, the satellite access device 90 provided in this embodiment includes:

[0152] The information acquisition module 901 is used to acquire satellite access information, which includes the number of terminals within the satellite coverage area, the distance information between the satellite and each terminal within the satellite coverage area, and the satellite-to-ground channel status information.

[0153] The terminal segmentation module 902 is used to divide the terminals within the satellite coverage area into multiple device clusters based on satellite access information, with each device cluster including multiple terminals;

[0154] The strategy determination module 903 is used to send information from each device cluster to the terminals in each device cluster. The information from the device cluster is used by the terminals in the device cluster to determine the satellite access strategy of the terminal through an intelligent agent. The parameters of the intelligent agent are obtained in advance through centralized training. The satellite access strategy includes the terminal's access power and access channel.

[0155] In one possible implementation, the terminal partitioning module 902 is further configured to:

[0156] The channel gain of each terminal within the satellite coverage area is determined based on satellite access information;

[0157] Terminals within the satellite coverage area are grouped according to channel gain to obtain multiple device groups, each containing multiple terminals;

[0158] Identify one terminal from each of the multiple device groups to obtain a device cluster. Repeat this step to obtain multiple device clusters.

[0159] In one possible implementation, the terminal partitioning module 902 is further configured to:

[0160] Terminals within the satellite coverage area are sorted according to their channel gain. Multiple consecutive terminals are then grouped into a single device group, resulting in multiple device groups. The number of device groups is the same as the number of terminals in a device cluster, and the number of terminals in each device group is the same.

[0161] In one possible implementation, the terminal partitioning module 902 is further configured to:

[0162] Determine the channel gain difference matching coefficient between the two terminals;

[0163] With the goal of maximizing the minimum channel gain difference matching coefficient among terminals in a device cluster, a device cluster is obtained by selecting one terminal from each of multiple device groups.

[0164] The satellite access device provided in this embodiment can execute the method provided in the above-described method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0165] Figure 10 A schematic diagram of the structure of a satellite access device provided in this application Figure 2 ,like Figure 10 As shown, the satellite access device 100 provided in this embodiment includes:

[0166] The information receiving module 1001 is used to receive information about the device cluster to which the terminal belongs. The information about the device cluster is determined based on satellite access information, which includes the number of terminals within the satellite coverage area, the distance information between the satellite and each terminal within the satellite coverage area, and the satellite-to-ground channel status information.

[0167] The access strategy determination module 1002 is used by the intelligent agent to determine the satellite access strategy of the terminal based on the terminal's status and the information of the device cluster. The satellite access strategy includes the terminal's access power and access channel.

[0168] The satellite access device provided in this embodiment can execute the method provided in the above-described method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0169] Figure 11This is a schematic diagram of the structure of a training device for an intelligent agent provided in this application, as shown below. Figure 11 As shown, the training device 110 for the intelligent agent provided in this embodiment includes:

[0170] The device cluster information receiving module 1101 is used to receive information from multiple device clusters, wherein each device cluster includes multiple terminals. The information of the device cluster is determined based on satellite access information, which includes the number of terminals within the satellite coverage area, the distance information between the satellite and each terminal within the satellite coverage area, and the satellite-to-ground channel status information.

[0171] The agent training module 1102 is used to train the agent based on information from multiple device clusters and the status of terminals in each device cluster, and to send the parameters of the trained agent to each terminal.

[0172] The training device for an intelligent agent provided in this embodiment can execute the method provided in the above-described method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0173] Figure 12 A schematic diagram of the satellite access equipment provided in this application. Figure 12 As shown, the electronic device 120 provided in this embodiment includes at least one processor 1201 and a memory 1202. Optionally, the device 120 further includes a communication component 1203. The processor 1201, the memory 1202, and the communication component 1203 are connected via a bus 1204.

[0174] In a specific implementation, at least one processor 1201 executes computer execution instructions stored in memory 1202, causing at least one processor 1201 to perform the above-described method.

[0175] The specific implementation process of processor 1201 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0176] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0177] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0178] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0179] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0180] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0181] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0182] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0183] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0184] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0185] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0186] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0187] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0188] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A satellite access method, characterized in that, Applied to satellites, the method includes: Obtain satellite access information, wherein the satellite access information includes the number of terminals within the satellite's coverage area, the distance information between the satellite and each terminal within the satellite's coverage area, and the satellite-to-ground channel status information; Based on the satellite access information, the terminals within the satellite coverage area are divided into multiple device clusters, and each device cluster includes multiple terminals; Information from each of the device clusters is sent to the terminals within each device cluster. This information is used by the terminals within the device cluster to determine their satellite access strategies through an intelligent agent. The parameters of the intelligent agent are obtained in advance through centralized training. The satellite access strategy includes the terminal's access power and access channel.

2. The method according to claim 1, characterized in that, The step of dividing the terminals within the satellite coverage area into multiple device clusters based on the satellite access information includes: The channel gain of each terminal within the satellite coverage area is determined based on the satellite access information; The terminals within the satellite coverage area are grouped according to the channel gain to obtain multiple device groups, and each device group includes multiple terminals; One terminal is determined from each of the multiple device groups to obtain a device cluster. This step is repeated to obtain multiple device clusters.

3. The method according to claim 2, characterized in that, The process of grouping terminals within the satellite coverage area according to the channel gain to obtain multiple device groups includes: Terminals within the satellite coverage area are sorted according to the channel gain. Multiple consecutive terminals are grouped into a device group according to the sorting, resulting in multiple device groups. The number of device groups is the same as the number of terminals in the device cluster, and the number of terminals in each device group is the same.

4. The method according to claim 2, characterized in that, The step of determining one terminal from each of the plurality of device groups to obtain a device cluster includes: Determine the channel gain difference matching coefficient between the two terminals; The device cluster is obtained by selecting one terminal from each of the multiple device groups with the goal of maximizing the minimum channel gain difference matching coefficient among the terminals in the device cluster.

5. A satellite access method, characterized in that, Applied to a terminal, on which an agent pre-trained in a centralized manner is deployed, the method includes: The terminal receives information about the device cluster to which it belongs, wherein the information about the device cluster is determined based on satellite access information, and the satellite access information includes the number of terminals within the satellite coverage area, the distance information between the satellite and each terminal within the satellite coverage area, and the satellite-to-ground channel status information. The intelligent agent determines the satellite access strategy of the terminal based on the terminal's state and the information of the device cluster. The satellite access strategy includes the terminal's access power and access channel.

6. The method according to claim 5, characterized in that, The state of the terminal includes the actions performed by the terminal at the previous moment and the observed variables at the previous moment. The actions performed include the terminal not transmitting or the terminal connecting to the satellite. The observed variables include the channel gain of the terminal, the distance between the terminal and the satellite, and the acknowledgment signal of the satellite.

7. A method for training an intelligent agent, characterized in that, Applied to ground gateway stations, the method includes: Receive information from multiple device clusters, wherein each device cluster includes multiple terminals, and the information of the device clusters is determined based on satellite access information, which includes the number of terminals within the satellite coverage area, the distance information between the satellite and each terminal within the satellite coverage area, and satellite-to-ground channel status information; Based on the information of the multiple device clusters and the state of the terminals in each device cluster, the intelligent agent is trained, and the parameters of the trained intelligent agent are sent to each terminal.

8. A satellite access device, characterized in that, The device includes: The information acquisition module is used to acquire satellite access information, wherein the satellite access information includes the number of terminals within the satellite coverage area, the distance information between the satellite and each terminal within the satellite coverage area, and the satellite-to-ground channel status information; The terminal segmentation module is used to segment the terminals within the satellite coverage area into multiple device clusters based on the satellite access information, wherein each device cluster includes multiple terminals; A strategy determination module is used to send information of each device cluster to the terminal in each device cluster. The information of the device cluster is used by the terminal in the device cluster to determine the satellite access strategy of the terminal through an intelligent agent. The parameters of the intelligent agent are obtained in advance through centralized training. The satellite access strategy includes the access power and access channel of the terminal.

9. A satellite access device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 7.

Images