Satellite internet of things resource allocation method and system

Abstract

The application provides a satellite Internet of Things resource allocation method and system, and relates to the technical field of satellite resource scheduling. The method comprises the following steps: receiving dynamic strategy parameters generated by a ground station based on intention description data, and constructing a joint state vector based on network environment state information within a satellite coverage range and a security performance target threshold in the dynamic strategy parameters; inputting the joint state vector into a pre-trained deep reinforcement learning decision network to obtain a joint resource allocation action, and executing the joint resource allocation action. The satellite Internet of Things resource allocation method and system provided by the application establish an end-to-end learnable mapping system from a semantic operation intention to a joint allocation action of underlying physical layer communication resources and security resources, realize real-time and accurate response of a satellite Internet of Things system to a high-level dynamic operation intention, and realize intelligent endogenous collaborative optimization of communication performance and security strength.

Description

Technical Field

[0001] This application relates to the field of satellite resource scheduling technology, and in particular to a satellite Internet of Things resource allocation method and system. Background Technology

[0002] Satellite IoT, through low-Earth orbit satellite constellations, achieves wide-area seamless coverage of terrestrial IoT terminals, becoming a key infrastructure for building integrated air-space-ground networks and supporting communications in remote areas and emergency situations. In this network, the sudden and dynamic nature of massive terminal service demands, along with the limited spectrum and power resources of satellite nodes, makes efficient and intelligent resource allocation methods a core challenge for ensuring overall network performance. However, methods based on fixed rules or static optimization in related technologies are difficult to adapt to dynamically changing network environments.

[0003] Therefore, there is an urgent need for a satellite resource scheduling method that can directly and dynamically transform the semantic operational intentions of the higher layers into a joint optimization allocation strategy for the underlying physical layer communication and security resources. Summary of the Invention

[0004] The purpose of this application is to provide a satellite Internet of Things (IoT) resource allocation method and system. By establishing an end-to-end learnable mapping system from semantic operational intent to the joint allocation of underlying physical layer communication and security resources, the satellite IoT system achieves real-time and accurate response to high-level dynamic operational intent, as well as intelligent endogenous collaborative optimization of communication performance and security strength.

[0005] This application provides a satellite Internet of Things (IoT) resource allocation method, including: The system receives dynamic policy parameters generated by the ground station based on intent description data, and constructs a joint state vector based on network environment status information within the satellite coverage area and the security performance target threshold in the dynamic policy parameters. The joint state vector is then input into a pre-trained deep reinforcement learning decision network to obtain joint resource allocation actions, which are then executed. The deep reinforcement learning decision network includes a feature extraction and fusion network and a decision network. The feature extraction and fusion network performs feature fusion and dimensionality reduction on the joint state vector to generate an environmental feature representation vector. The decision network takes the environmental feature representation vector as input and outputs the joint resource allocation actions. The joint resource allocation actions include communication resource allocation actions and security resource allocation actions. The security resource allocation actions include interference channel selection actions and noise power allocation actions to combat potential eavesdropping.

[0006] Optionally, the dynamic strategy parameters further include: a reward function weight adjustment vector; the dynamic strategy parameters are generated based on the following steps: based on a preset structured description template, operational instructions are converted into structured intent description data, and expected effect keywords and intensity indicator values ​​are extracted from the structured intent description data; by querying the intent knowledge base, the expected effect keywords are mapped to the reward function weight adjustment vector, and the value of the reward function weight adjustment vector is determined according to the intensity indicator value; when the expected effect keywords contain security requirements, the intensity indicator value is quantified by combining historical security event records and the current network situation, and the quantified intensity indicator value is used as the security performance target threshold.

[0007] Optionally, the feature extraction and fusion network includes: a parallel graph convolutional network and a convolutional neural network; the step of inputting the joint state vector into a pre-trained deep reinforcement learning decision network to obtain a joint resource allocation action includes: using the graph convolutional network to model the topological connection relationship between the terminal and the channel and the terminal attribute features, outputting node feature vectors, and using the convolutional neural network to extract local correlation features from the channel state information matrix, outputting a channel feature map; concatenating the node feature vectors with the channel feature map, and then fusing and reducing the dimensionality through a fully connected layer to generate the environmental feature representation vector.

[0008] Optionally, the method further includes: calculating an intent alignment index to evaluate the degree of matching between the execution effect of the joint resource allocation action and the operational intent expressed by the intent description data; and storing or outputting the intent alignment index as a basis for system performance monitoring.

[0009] Optionally, after performing the joint resource allocation action, the method further includes: calculating the immediate reward obtained after performing the joint resource allocation action using a multi-objective reward function; wherein, the weight of each sub-reward item in the multi-objective reward function is dynamically adjusted by the reward function weight adjustment vector; and the security gain sub-reward in the multi-objective reward function is calculated based on the security performance target threshold.

[0010] Optionally, the calculation steps of the security gain sub-reward include: determining the security action efficiency factor, and calculating the security gain sub-reward based on the obtained legitimate channel capacity, eavesdropping channel capacity, and the security performance target threshold; wherein, the security action efficiency factor is used to characterize the degree of eavesdropping channel capacity reduction caused by unit noise power.

[0011] Optionally, the deep reinforcement learning decision network is trained and optimized based on the following steps: using the state information, action information, reward information and next state information generated after executing the joint resource allocation action to construct experience sample data; storing the experience sample data in an experience replay buffer, and using the experience replay mechanism to update the parameters of the deep reinforcement learning decision network based on the sample data in the experience replay buffer.

[0012] Optionally, the training and optimization steps further include: after completing local training, uploading the local model parameter update to the federated aggregation server, so that the federated aggregation server assigns differentiated aggregation weights to the model parameter update of each satellite agent according to the importance of the operational intent currently carried by each satellite agent, and weighted aggregation to generate global model parameters; receiving the global model parameters issued by the federated aggregation server, and using the global model parameters to update the local model.

[0013] This application also provides a satellite Internet of Things (IoT) resource allocation system, including: The system comprises a ground station and multiple satellites; the ground station transmits dynamic policy parameters to the satellites via a satellite-to-ground link; each satellite is equipped with an intelligent agent; and the system is used to execute the satellite Internet of Things resource allocation method as described in any of the preceding claims.

[0014] This application also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of any of the satellite Internet of Things resource allocation methods described above.

[0015] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the satellite Internet of Things resource allocation methods described above.

[0016] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the satellite Internet of Things resource allocation methods described above.

[0017] The satellite IoT resource allocation method and system provided in this application first receive dynamic policy parameters generated by a ground station based on intent description data, and construct a joint state vector based on network environment status information within the satellite coverage area and the security performance target threshold in the dynamic policy parameters. Then, the joint state vector is input into a pre-trained deep reinforcement learning decision network to obtain a joint resource allocation action, which is then executed. The deep reinforcement learning decision network includes a feature extraction and fusion network and a decision network. The feature extraction and fusion network performs feature fusion and dimensionality reduction on the joint state vector to generate an environmental feature representation vector. The decision network takes the environmental feature representation vector as input and outputs the joint resource allocation action. The joint resource allocation action includes a communication resource allocation action and a security resource allocation action. The security resource allocation action includes interference channel selection actions and noise power allocation actions to combat potential eavesdropping. Thus, by establishing an end-to-end learnable mapping system from semantic operational intent to the joint allocation actions of communication and security resources at the underlying physical layer, the satellite IoT system achieves real-time and accurate response to high-level dynamic operational intents, as well as intelligent endogenous collaborative optimization of communication performance and security strength. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the three-layer collaborative architecture of the satellite Internet of Things resource allocation system provided in this application; Figure 2 This is a flowchart illustrating the satellite Internet of Things resource allocation method provided in this application; Figure 3 This is a schematic diagram of the structured composition of the intent description template provided in this application; Figure 4 This is a schematic diagram of the network structure provided in this application; Figure 5 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship. All actions involving the acquisition of signal information or data in this application are performed in accordance with the relevant data protection laws and policies of the country where the application is located and with authorization from the owner of the relevant device.

[0022] To address the aforementioned technical problems in related technologies, this application provides a satellite Internet of Things (IoT) resource allocation method. This method constructs a three-layer collaborative architecture comprising an intent understanding layer, an intelligent decision-making layer, and a resource execution layer. Through an end-to-end learnable mapping system, it dynamically transforms high-level operational intents into a joint optimization allocation strategy for communication and security resources at the underlying physical layer. Specifically, as follows... Figure 1 The diagram shown is a schematic of the three-layer collaborative architecture of the satellite Internet of Things resource allocation system provided in this application embodiment. The architecture includes an intent understanding layer, an intelligent decision-making layer, and a resource execution layer. Each layer realizes data interaction and command transmission through the satellite-to-ground link, and together completes the transformation from high-level operational intent to low-level resource allocation action.

[0023] The satellite Internet of Things (IoT) resource allocation method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0024] like Figure 2 As shown in the embodiment of this application, a satellite Internet of Things (IoT) resource allocation method is provided. This method is applied to a satellite intelligent agent and may include the following steps 201 and 202: Step 201: Receive dynamic policy parameters generated by the ground station based on intent description data, and construct a joint state vector based on network environment status information within the satellite coverage area and the security performance target threshold in the dynamic policy parameters.

[0025] For example, the satellite agent periodically collects network environment state information within its coverage area to construct an enhanced joint state vector. This joint state vector contains state information in both the communication and security dimensions.

[0026] Specifically, the status information of the aforementioned communication dimensions includes: terminal traffic queue length, channel status information, co-channel interference intensity, historical throughput and latency statistics, and terminal service type priority label. Here, the terminal traffic queue length represents the number of data packets waiting for transmission from each terminal; the channel status information represents the estimated uplink channel gain from each terminal to the satellite; the co-channel interference intensity represents the existing interference power spectral density on each channel; the historical throughput and latency statistics represent the average data transmission rate and packet latency of each terminal within a past time window; and the terminal service type priority label is set by the policy constraint identifier issued by the intent parsing module.

[0027] Specifically, the status information of the aforementioned security dimensions includes: channel feature difference degree, real-time security threat index, and the security performance target threshold. The channel feature difference degree represents the spatial separation between the legitimate terminal channel direction and the potential eavesdropping direction, estimated using multi-antenna technology. This difference degree is a dimensionless scalar; a larger value indicates better spatial distinguishability. The real-time security threat index represents the threat assessment value output after analyzing the characteristics of the received signal based on a lightweight anomaly detection model. This index is between zero and one; a higher value indicates a greater likelihood of eavesdropping or interference attacks. The security performance target threshold is distributed by the intent parsing module and input into the decision model as part of the status information.

[0028] For example, the aforementioned dynamic policy parameters are generated and distributed by the ground station's intent parsing module. This generation process specifically includes the following steps S1 to S3: Step S1: Based on the preset structured description template, the operation instructions are converted into structured intent description data, and the expected effect keywords and intensity indication values ​​are extracted from the structured intent description data.

[0029] Step S2: By querying the intent knowledge base, the expected effect keywords are mapped to the reward function weight adjustment vector, and the value of the reward function weight adjustment vector is determined according to the intensity indication value.

[0030] Step S3: When the desired effect keyword includes security requirements, the intensity indicator value is quantified by combining historical security event records and the current network situation, and the quantified intensity indicator value is used as the security performance target threshold.

[0031] For example, such as Figure 1 As shown, after receiving structured intent description data, the system parses it using an intent parsing module deployed on the ground station to generate dynamic policy parameters. This intent parsing module has a pre-defined intent knowledge base, which stores a mapping table between desired effect keywords, constraint keywords, and adjustable parameters of the deep reinforcement learning model.

[0032] Combination Figure 1 ,like Figure 3 The diagram illustrates the structured composition of the intent description template provided in this embodiment. The system pre-defines a digital description template for operational intent, used to convert natural language or structured high-level operational instructions into machine-processable structured data. This template contains four interrelated core elements: Target Subject: Specifies the object to which the intent is applied, ranging from at least one satellite, one beam, or one terminal group, such as satellite, beam, terminal group, etc.; Desired Effect: Describes the performance or status requirements for the target subject, selected from at least one of throughput improvement, latency reduction, energy saving, and security level improvement, such as throughput improvement, latency reduction, energy saving, and security level improvement; Constraints: Define the boundary conditions for achieving the desired effect, selected from at least one of time range, geographical area, resource budget, and business priority, such as time range, geographical area, resource budget, and business priority, etc.; Intensity Indicator: Quantifies the degree of the desired effect or constraint, as a percentage value or level label. After receiving the structured intent description data frame, the intent parsing module extracts the elements. The extracted elements are then transformed into dynamic policy parameters that can be executed by the satellite agent and distributed to the satellite local agent identified by the target subject.

[0033] For example, the parsing process performs the following operations: The first operation is element extraction, where the parsing module identifies and separates the target subject identifier, the list of expected effect keywords, the list of constraint keywords, and the intensity indicator value from the intent description data frame; the second operation is parameter mapping, where the parsing module queries the intent knowledge base and maps each keyword in the expected effect keyword list to a reward function weight adjustment vector. The dimension of the reward function weight adjustment vector is the same as the number of sub-items of the reward function, and each element corresponds to the adjustment amount of a sub-reward weight. The specific value of the reward function weight adjustment vector is determined based on the intensity indicator value; the third operation is security target quantification, where the parsing module combines historical security event records and current network situation reports to quantify the intensity indicator value into a specific security performance target threshold, which represents the minimum security performance level required by the system; the fourth operation is parameter encapsulation and distribution, where the parsing module encapsulates the generated reward function weight adjustment vector and security performance target threshold into a dynamic policy parameter package and distributes it to the corresponding satellite local agent via the satellite-to-ground link. Finally, the satellite agent receives the dynamic policy parameters distributed by the intent parsing module.

[0034] Step 202: Input the joint state vector into a pre-trained deep reinforcement learning decision network to obtain a joint resource allocation action, and execute the joint resource allocation action.

[0035] The deep reinforcement learning decision network includes a feature extraction and fusion network and a decision network. The feature extraction and fusion network is used to perform feature fusion and dimensionality reduction on the joint state vector to generate an environmental feature representation vector. The decision network takes the environmental feature representation vector as input and outputs a joint resource allocation action. The joint resource allocation action includes a communication resource allocation action and a security resource allocation action. The security resource allocation action includes an interference channel selection action and a noise power allocation action to combat potential eavesdropping.

[0036] For example, a deep reinforcement learning decision network includes a feature extraction and fusion network and a decision network. As... Figure 1 As shown, the intelligent decision-making layer network is used to perform feature fusion and dimensionality reduction on the joint state vector to generate an environmental feature representation vector. The decision network takes the environmental feature representation vector as input and outputs a joint resource allocation action.

[0037] Specifically, such as Figure 4As shown, the feature extraction and fusion network comprises two parallel sub-networks: the first sub-network is a Graph Convolutional Network (GCN), whose input is the topological connection matrix between the terminal and the channel, as well as the attribute features of the terminal nodes. This sub-network is used to model the complex interference and security relationships between terminals due to geographical proximity or channel sharing, and outputs a high-dimensional feature vector at the node level (node ​​feature vector). The second sub-network is a Convolutional Neural Network (CNN), whose input is the channel state information matrix. This sub-network is used to extract local correlation features of the channel in the frequency, time, or spatial dimensions, and outputs a channel feature map (channel feature vector). The node feature vector output by the Graph Convolutional Network is concatenated with the channel feature map output by the CNN, and then fused and dimensionality reduced through a fully connected layer to generate an environmental feature representation vector.

[0038] Specifically, step 202 above may also include the following steps 202a1 and 202a2: Step 202a1: Model the topological connection relationship between the terminal and the channel and the terminal attribute features using the graph convolutional network, output node feature vectors, and extract local correlation features from the channel state information matrix using the convolutional neural network to output channel feature map.

[0039] Step 202a2: Concatenate the node feature vector with the channel feature map, and then fuse and reduce the dimension through a fully connected layer to generate the environment feature representation vector.

[0040] For example, such as Figure 4 As shown, the decision network is constructed based on the Actor-Critic reinforcement learning framework. The joint resource allocation action includes two parts: communication resource allocation and security resource allocation. The communication resource allocation action specifies that each terminal currently requesting service is assigned a specific uplink channel number and a transmit power level on that channel. The transmit power level is selected from a predefined set of discrete power levels.

[0041] For example, the security resource allocation action, used to counter potential eavesdropping, includes interference channel selection and noise power allocation actions. The interference channel is selected from a set of idle channels not currently occupied by legitimate communication, and the noise power level is selected from a predefined set of discrete levels. The Actor network in the decision network maps the environmental feature representation vector to a continuous action probability distribution. For discrete actions such as channel selection, it outputs the selection probability of each available channel; for continuous actions such as power values, it outputs the mean and variance of a Gaussian distribution, from which the specific power level is sampled.

[0042] Combination Figure 1 ,like Figure 4As shown, the decision network is built based on the Actor-Critic reinforcement learning framework. The Actor network maps environmental feature representation vectors to joint resource allocation actions; the Critic network is used to evaluate the value of these actions. The joint resource allocation actions include communication resource allocation actions and security resource allocation actions. The communication resource allocation action specifies that each terminal currently requesting service is assigned a specific uplink channel number and a transmit power level on that channel, selected from a predefined set of discrete power levels. The security resource allocation action specifies that, to counter potential eavesdropping, artificial noise is injected into a specific jamming channel. This action specifically includes two sub-actions: jamming channel selection and noise power allocation. The jamming channel is selected from a set of idle channels not currently occupied by legitimate communication, and the noise power level is selected from a predefined set of discrete power levels.

[0043] like Figure 1 As shown, when the resource control unit of the resource execution layer performs the joint resource allocation action, it performs two synchronous operations: the first operation is communication resource execution, that is, according to the action output, the satellite sends an instruction to the designated terminal through the control channel, instructing it to use the allocated channel and power level for uplink data transmission; the second operation is security resource execution, that is, the satellite controls its onboard controllable jamming transmitter to inject artificial noise signals at the allocated noise power level on the jamming channel specified by the action.

[0044] For example, after resource execution, if the network environment changes, the satellite begins state awareness for the next cycle, thus forming a complete closed loop from intent resolution, state awareness, joint decision-making, reward calculation, model update to resource execution. This system can continuously adapt to the dynamically changing network environment and constantly updated high-level operational intents, achieving intelligent joint optimization of communication performance and security levels.

[0045] Optionally, the satellite IoT resource allocation method provided in this application embodiment can also perform multi-target reward calculation and intent alignment feedback.

[0046] For example, after step 202 above, the satellite Internet of Things resource allocation method provided in this application embodiment may further include the following step 203: Step 203: Calculate the immediate reward obtained after performing the joint resource allocation action using a multi-objective reward function.

[0047] The weights of each sub-reward item in the multi-objective reward function are dynamically adjusted by the reward function weight adjustment vector; the security gain sub-reward in the multi-objective reward function is calculated based on the security performance target threshold.

[0048] For example, a multi-objective reward function is used to calculate the immediate reward obtained after performing the joint resource allocation action. The reward signal is generated by a multi-objective reward function and is a weighted sum of multiple sub-reward items. For example... Figure 1 As shown, after the agent performs a joint resource allocation action, the environment transitions to a new state, generating an immediate reward signal. This reward signal is calculated and generated by a multi-objective reward function designed to dynamically reflect the operational intentions of higher levels.

[0049] The reward function is a weighted sum of multiple sub-reward items, and its mathematical expression can be represented by the following formula (1): (1); The symbols in formula (1) above are defined as follows: This represents the instantaneous reward scalar obtained at time step t; This represents the total throughput sub-reward scalar of the system at time step t, which is the sum of the throughputs of all successfully transmitting terminals; This represents the average packet delay penalty scalar of the system at time step t, and its value is the average delay of all transmission terminals; This represents the total energy consumption penalty scalar of the system at time step t, which is the sum of the transmit power of all terminals and the noise injection power of the satellite. This represents the security gain sub-reward scalar of the system at time step t; This represents the dynamic weighting coefficient of the throughput sub-reward at time step t; This represents the dynamic weighting coefficient of the delay penalty at time step t; This represents the dynamic weighting coefficient of the energy consumption penalty at time step t; This represents the dynamic weighting coefficient of the security gain at time step t.

[0050] For example, dynamic weight coefficient vector The weight adjustment vector is issued by the intent parsing module. With the basic weight vector Adding them together, we get: Basic weight vector These are the preset default trade-off parameters.

[0051] Specifically, in step 203 above, the calculation of the security gain sub-reward may further include the following step 203a: Step 203a: Determine the security action efficiency factor, and calculate the security gain sub-reward based on the obtained legitimate channel capacity, eavesdropping channel capacity, and the security performance target threshold.

[0052] The security action efficiency factor is used to characterize the degree of reduction in eavesdropping channel capacity caused by unit noise power.

[0053] For example, the security gain sub-reward in the above multi-objective reward function The calculation can be performed using the following formula (2): (2); The symbols in formula (2) above are defined as follows: This represents the instantaneous channel capacity scalar of a legal channel at time step t, calculated using Shannon's formula based on the allocated channel gain and transmit power. This represents the instantaneous channel capacity scalar of the eavesdropping channel at time step t, calculated based on the estimated worst-case eavesdropping channel gain. This represents a scalar value indicating the target threshold for security performance, issued by the intent parsing module. The safety action efficiency factor scalar can be calculated using the following formula (3): (3); in, This represents the decrease in eavesdropping channel capacity due to injected noise. This represents the injected noise power value. This factor is used to assess the efficiency of safety resource utilization.

[0054] In one possible implementation, the system calculates an intent alignment metric. This metric is not used directly for model updates, but rather serves as the basis for system-level performance monitoring and long-term optimization of the intent parsing model.

[0055] For example, the satellite Internet of Things resource allocation method provided in this application embodiment may further include the following steps 204 and 205: Step 204: Calculate the intent alignment index, which is used to evaluate the degree of matching between the execution effect of the joint resource allocation action and the operational intent expressed by the intent description data.

[0056] Step 205: Store or output the intent alignment index as the basis for system performance monitoring.

[0057] For example, an intent alignment metric is calculated to assess the degree of match between the execution effect of the joint resource allocation action and the operational intent expressed by the intent description data. This metric is not directly used for model updates, but rather serves as the basis for long-term optimization of the system-level performance monitoring and intent parsing model. For instance, if the intent is to improve the security level of a certain type of terminal, then... Calculated as the security gain for this type of terminal The average increase rate.

[0058] Optionally, the deep reinforcement learning decision network provided in this application embodiment is iteratively optimized through the following training steps. Specifically, after step 202 above, the satellite IoT resource allocation method provided in this application embodiment may further include the following steps 206 and 207: Step 206: Construct experience sample data using the status information, action information, reward information, and next status information generated after executing the joint resource allocation action.

[0059] Step 207: Store the experience sample data into the experience replay buffer, and use the experience replay mechanism to update the parameters of the deep reinforcement learning decision network based on the sample data in the experience replay buffer.

[0060] First, experience sample data is constructed using the state information, action information, reward information, and next state information generated after executing the joint resource allocation action. This experience sample data is stored in an experience replay buffer, and the parameters of the Actor and Critic networks of the deep reinforcement learning decision network are updated based on the sample data in the experience replay buffer using the experience replay mechanism to ensure training stability.

[0061] For example, to facilitate knowledge sharing among multiple satellite agents, the above training and optimization steps are also coordinated based on a federated learning framework, specifically including the following steps 208 and 209: Step 208: After completing local training, upload the local model parameter update to the federated aggregation server so that the federated aggregation server can assign differentiated aggregation weights to the model parameter update of each satellite agent according to the importance of the operational intent currently carried by each satellite agent, and generate global model parameters by weighted aggregation.

[0062] Step 209: Receive the global model parameters issued by the federated aggregation server, and update the local model using the global model parameters.

[0063] For example, after local training is completed, the model parameter updates or gradient information of the local Actor network and Critic network are encrypted and uploaded to the federated aggregation server. After collecting the uploaded information from all participating satellites, the federated aggregation server executes a secure model aggregation algorithm. Specifically, this aggregation algorithm introduces intent-based weights; that is, the federated aggregation server assigns differentiated aggregation weights to the model parameter updates of each satellite agent based on the importance of the operational intent currently carried by each satellite agent. Satellites carrying high-security-level, high-priority intents receive higher weights for their model updates during aggregation. The federated aggregation server then distributes the weighted aggregation-generated global model parameters. The satellite agent receives the global model parameters distributed by the federated aggregation server and uses them to initialize or partially replace its local model, thereby forming a continuous optimization loop of local training and global collaboration.

[0064] For example, each satellite's local agent uses collected state, action, reward, and next state sequence data to train its own Actor and Critic networks. Training employs experience replay and target network techniques to ensure stability.

[0065] Based on training, multiple satellite agents collaborate and share knowledge through a federated learning framework. The specific process is as follows: After each satellite's local agent performs several rounds of gradient descent training locally, it encrypts the model parameter updates or gradient information of its local Actor and Critic networks and uploads them to a ground station or a federated aggregation server deployed on a designated high-performance satellite. The federated aggregation server collects the uploaded information from all participating satellites and executes a secure model aggregation algorithm. This algorithm not only considers simple parameter averaging but also introduces intent-based weights. Specifically, the aggregation server assigns different aggregation weights to each satellite's model updates based on the importance of the intent currently carried by each satellite. Satellites carrying high-security, high-priority intents receive higher weights in their model updates during aggregation. The federated aggregation server then distributes the aggregated global model parameters to all participating satellites. Each satellite initializes or partially replaces its local model with the global model parameters and then continues training and adaptation in its local environment, thus forming a continuous optimization loop of local training and global collaboration.

[0066] The satellite IoT resource allocation method provided in this application establishes an end-to-end learnable mapping from semantic operational intent to the underlying physical layer resource joint allocation strategy, enabling real-time and precise alignment between network resource scheduling and high-level dynamic operational goals. Specifically, after receiving a structured intent instruction, the intent parsing module automatically interprets it into dynamically adjustable reinforcement learning reward function weights and security target thresholds, and then sends them to the satellite agent. This allows the underlying deep reinforcement learning decision model to directly adjust its optimization direction based on the high-level intent, so that the generated resource allocation strategy is not only a passive adaptation to environmental information such as channel state, but also an active fulfillment of abstract operational requirements such as improving security levels and prioritizing certain types of services. Thus, the system achieves a fundamental transformation from rigid, preset-goal automation to flexible, intent-driven intelligence.

[0067] The satellite IoT resource allocation method provided in this application significantly improves the overall utilization efficiency of scarce onboard resources while meeting dynamic security needs by explicitly incorporating security resources into the optimization framework and designing an efficiency-oriented reward mechanism. This method defines the power of artificial noise and channel occupancy as allocable security resources and optimizes them jointly with communication resources. The security gain sub-item introduced in its reward function includes a security action efficiency factor, which evaluates the security performance improvement generated by a unit investment of security resources. This drives the agent not only to pursue the achievement of security goals but also to autonomously learn efficient and economical security resource usage strategies, avoiding waste caused by blind allocation. Simultaneously, through intent-aware federated collaboration, different satellites can share efficient resource allocation experience under specific intents, promoting the accumulation and optimization of global knowledge and enhancing the system's adaptability and resource utilization economy in long-term operation.

[0068] To facilitate a more detailed understanding of the technical solution of this application, an embodiment combining an application scenario is also provided in this application, as follows: This embodiment is applicable to IoT communication scenarios in remote mountainous areas covered by low-orbit satellite constellations. The core objective is to achieve synergistic optimization of terminal communication latency reduction and energy saving through intent-driven resource scheduling.

[0069] In this embodiment, the specific implementation of system initialization and intent template definition is as follows: Network entity definition: Specifically, the satellite IoT system comprises eight low-Earth orbit satellites, numbered S1 to S8. Each satellite carries 10 phased array beams with a coverage angle of 8 degrees and a single beam ground coverage radius of approximately 40 kilometers. These eight satellites form a near-Earth orbit constellation, achieving coverage in remote mountainous areas ranging from 15°N to 35°N and 95°E to 115°E. This region has complex terrain, dispersed terminal distribution, and weak signal conditions in some areas.

[0070] 150 IoT terminals are deployed on the ground, divided into two terminal groups: Emergency Terminal Group T1 contains 30 terminals for data transmission of geological disaster monitoring in mountainous areas, and is centrally deployed in the area from 25°N to 26°N and from 100°E to 101°E; Ordinary Terminal Group T2 contains 120 terminals for daily data transmission of ecological environment monitoring in mountainous areas, and is dispersedly deployed in the above-mentioned coverage area.

[0071] Each satellite carries a local intelligent agent. The deep reinforcement learning decision-making model built into the agent is implemented by a lightweight on-board processing unit with a computing power of 8×10^5 floating-point operations per second, which meets the real-time decision-making needs in remote mountainous areas and is adapted to the limited computing resources on the satellite.

[0072] The template for digitally describing operational intent is defined as follows: It adopts the four-element template in the technical solution, and the specific values ​​of each element are adapted to the scenario of this embodiment: the target subject is the S3 to S6 satellites and their beams, and the T1 and T2 terminal groups; the expected effect includes the type of low power consumption required in mountainous areas, such as reduced latency and energy saving; the constraint conditions focus on the resource budget of the geographical area; the intensity indication sets specific numerical standards for latency and energy consumption.

[0073] For example, the structured intent description data frame issued by the ground control center is as follows: the target subject is T1 and T2 covered by the B1 to B6 beams of S3 to S6; the desired effect is reduced latency and energy saving; the constraints are: the time range starts with a timestamp of 1730100000 and ends with a timestamp of 1730100900, corresponding to the next 15 minutes; the geographical region is from 25°N to 26°N and from 100°E to 101°E; the resource budget is the terminal transmit power limit of 20dBm; the intensity indication is a latency reduction to within 35 milliseconds and an energy saving of 12%.

[0074] In this embodiment, the specific implementation of intent parsing and dynamic policy parameter generation is as follows: The intent parsing module is deployed at a regional ground station with a computation latency of ≤15 milliseconds. The module's pre-set intent knowledge base stores the mapping relationship between latency and energy consumption optimization. Specifically, the weight adjustment vector corresponding to the desired effect keyword and a 15-minute latency reduction is 0.05 0.3 0.25 0.05, and the weight adjustment vector corresponding to a 12% energy saving is 0.05 0.1 0.3 0.05. The strategy constraint identifier for the keyword's geographical region is C3, and the strategy constraint identifier for the resource budget is C4.

[0075] After receiving the structured intent data frame, the intent parsing module extracts the target subject identifier S3-S6-B1-B6-T1-T2, the expected effect keyword list latency reduction and energy saving, the constraint conditions keyword list time range 15 minutes, the geographic area resource budget 20dBm, and the intensity indication value 35 milliseconds 12% through the semantic recognition algorithm.

[0076] During the parameter mapping phase, the weight adjustment vectors corresponding to the keywords with the desired effect are summed to obtain the total weight adjustment vector. This total weight adjustment vector is then added to the preset base weight vector to obtain the dynamic weight coefficient vector. The policy constraint identifier set is C3C4, which corresponds to geographical area restrictions and power upper limit restrictions, respectively.

[0077] Since the desired effect does not involve security-related keywords, there is no need for security target quantification. The intent parsing module directly encapsulates the dynamic weight coefficient vector policy constraint identifier set into a dynamic policy parameter package, which is then sent to the local intelligent agents of S3 to S6 via the satellite-to-ground link at a transmission rate of 800kbps, adapting to the bandwidth conditions of satellite-to-ground links in mountainous areas.

[0078] In this embodiment, the specific implementation of joint state awareness for communication and security is as follows: Local agents S3 to S6 collect network environment status information within their coverage area in 8-second sensing cycles. In the communication dimension, the queue length of 30 terminals in T1 ranges from 8 to 12 data packets, averaging 10; the queue length of 120 terminals in T2 ranges from 4 to 7 data packets, averaging 5.5. Regarding channel gain, the average channel gain from beams B1 to B6 of S3 to T1 terminals is 0.75, and the average channel gain to T2 terminals is 0.68. The average co-channel interference intensity is -92 dBm. In the past sensing cycle, the average throughput of T1 is 8 Mbps, and the average throughput of T2 is 18 Mbps. The average system packet latency is 48 milliseconds. The service priority of T1 is level 4, and the service priority of T2 is level 2.

[0079] In the security dimension state information, the average channel feature difference is 0.65, the real-time security threat index is 0.2, and there are no special security performance target threshold requirements; only the default basic security threshold is used as the state information input. All collected state information is converted into standardized values ​​in the 0-1 range and stored in the agent's local cache unit with a cache capacity of 800MB, meeting the requirements for lightweight storage.

[0080] In this embodiment, the specific implementation of feature fusion and representation is as follows: The feature extraction neural network for the local agent adopts a dual-parallel sub-network structure consistent with the technical solution. The input to the graph convolutional network is a 150×10 terminal-channel topology connection matrix and a 150×3 terminal node attribute feature matrix. After processing through two graph convolutional layers and ReLU activation layers, the output is a 150×64 node-level high-dimensional feature vector.

[0081] The input to the convolutional neural network is a 10×150 channel state information matrix. After processing through three convolutional layers and one pooling layer, it is flattened and compressed into a 32-dimensional channel feature vector. The node aggregated feature vector is concatenated with the channel feature vector and then input into a fully connected layer for fusion and dimensionality reduction. Finally, a fixed-dimensional unified environmental feature representation vector is output, providing semantically rich feature support for subsequent decision-making.

[0082] In this embodiment, the joint decision-making process for communication and security is implemented as follows: The decision-making network of the local intelligent agent is built based on the Actor-Critic reinforcement learning framework. The Actor network consists of three fully connected layers, with a 64-dimensional environmental feature representation vector as input and a joint resource allocation action as output. Specifically, in the communication resource allocation action, the 30 terminals of T1 are allocated to channels 1 to 6 corresponding to beams B1 to B3 of S3 to S6, with 5 terminals allocated to each channel and a transmit power selection level 2 corresponding to 18dBm; the 120 terminals of T2 are allocated to channels 7 to 10 corresponding to beams B4 to B6, with 12 terminals allocated to each channel and a transmit power selection level 1 corresponding to 15dBm.

[0083] In the security resource allocation process, given the current low security threat index, only two idle channels are selected to inject low-power artificial noise. The noise power level is set to 1, corresponding to 12 dBm, to ensure basic security while avoiding energy waste. The Critic network output value estimate is 7.8, indicating that the current action is adapted to the desired reduction in latency and energy conservation.

[0084] In this embodiment, the specific implementation of multi-objective reward calculation and intent alignment feedback is as follows: After the agent executes the joint resource allocation action, it calculates the immediate reward using a multi-objective reward function. The total throughput sub-reward is the sum of the throughputs of T1 and T2, the average packet latency penalty is the weighted average latency of the two types of terminals, the total energy consumption penalty is the sum of the terminal's transmit power and the satellite noise injection power, and the security gain sub-reward is calculated based on a basic security threshold. The dynamic weight coefficient vector is determined by parameters issued by the intent parsing module, and the final immediate reward accurately reflects the optimization effects of latency reduction and energy saving.

[0085] The intent alignment metric is a weighted average of the latency reduction ratio and energy saving ratio between T1 and T2. The calculation results are used for system-level performance monitoring and provide data support for the long-term optimization of the intent parsing model.

[0086] In this embodiment, the specific implementation of model training and federated collaborative optimization is as follows: The local agent stores the state-action-reward sequence data into an experience replay buffer with a capacity of 8000 samples. Local training begins when the number of samples reaches 800. Training parameters are set to batch size 64, learning rate 0.0015, discount factor 0.9, and target network update cycle of 80 steps. Local training is performed for a total of 800 rounds. Training stops when the loss value is below 0.015 for 8 consecutive rounds to ensure model convergence.

[0087] During the federated collaborative optimization process, satellite agents S3 to S6 and S1, S2, S7, and S8 participate together. After each satellite completes local training, it encrypts and uploads the updated model parameters to the ground-based federated aggregation server. The aggregation phase employs an intent-aware weighted strategy. S3 to S6 carry the core intent for latency and energy consumption optimization, with an aggregation weight of 0.2 each. The remaining satellites each have an aggregation weight of 0.1. The aggregated global model parameters are then distributed to all participating satellites, achieving a cyclical optimization process of local training and global collaboration.

[0088] In this embodiment, the specific implementation of resource execution and closed-loop feedback is as follows: The satellite resource control unit simultaneously performs communication and security resource allocation actions. Regarding communication resource allocation, resource allocation commands are sent to the terminal via a control channel with a center frequency of 2.3 GHz. The command format is a fixed length of 64 bytes, ensuring accurate reception and adjustment of radio frequency parameters by the terminal. Regarding security resource allocation, a controllable interference transmitter is controlled to inject Gaussian white noise into a designated idle channel. The noise transmitter operates at a frequency range of 2.3 GHz to 2.4 GHz, with an adjustable output power range of 10 dBm to 20 dBm.

[0089] After the resource is executed, the network environment enters a new state. The local agent waits for the next perception cycle to start state acquisition, repeats the above complete process, and forms a closed-loop optimization system.

[0090] This embodiment adapts to the network deployment and intent requirements of remote mountainous areas, achieving a precise conversion of high-level latency reduction and energy saving intentions into low-level resource allocation. In actual operation, the system can effectively reduce terminal communication latency and energy consumption while ensuring basic communication performance. Resource utilization efficiency is improved compared to existing technologies, adapting to the application needs of low-power, long-lasting IoT terminals in remote mountainous areas, and possessing good practical feasibility and scenario adaptability.

[0091] The satellite IoT resource allocation method provided in this application first receives dynamic policy parameters generated by a ground station based on intent description data, and constructs a joint state vector based on network environment status information within the satellite coverage area and the security performance target threshold in the dynamic policy parameters. Then, the joint state vector is input into a pre-trained deep reinforcement learning decision network to obtain a joint resource allocation action, which is then executed. The deep reinforcement learning decision network includes a feature extraction and fusion network and a decision network. The feature extraction and fusion network performs feature fusion and dimensionality reduction on the joint state vector to generate an environmental feature representation vector. The decision network takes the environmental feature representation vector as input and outputs the joint resource allocation action. The joint resource allocation action includes a communication resource allocation action and a security resource allocation action. The security resource allocation action includes interference channel selection actions and noise power allocation actions to combat potential eavesdropping. Thus, by establishing an end-to-end learnable mapping system from semantic operational intent to the joint allocation actions of communication and security resources at the underlying physical layer, the satellite IoT system achieves real-time and accurate response to high-level dynamic operational intents, as well as intelligent endogenous collaborative optimization of communication performance and security strength.

[0092] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communications bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other through the communications bus 540. The processor 510 can call logical instructions in the memory 530 to execute a satellite IoT resource allocation method. This method includes: first, receiving dynamic policy parameters generated by a ground station based on intent description data, and constructing a joint state vector based on network environment state information within the satellite coverage area and a security performance target threshold in the dynamic policy parameters; then, inputting the joint state vector into a pre-trained deep reinforcement learning decision network to obtain a joint resource allocation action, and executing the joint resource allocation action; wherein the deep reinforcement learning decision network includes a feature extraction and fusion network and a decision network; the feature extraction and fusion network is used to perform feature fusion and dimensionality reduction on the joint state vector to generate an environmental feature representation vector; the decision network takes the environmental feature representation vector as input and outputs the joint resource allocation action; the joint resource allocation action includes a communication resource allocation action and a security resource allocation action, the security resource allocation action including interference channel selection action and noise power allocation action to combat potential eavesdropping. Thus, by establishing an end-to-end learnable mapping system that extends from semantic operational intent to the joint allocation of underlying physical layer communication and security resources, the satellite IoT system achieves real-time and accurate response to high-level dynamic operational intent, as well as intelligent endogenous collaborative optimization of communication performance and security strength.

[0093] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0094] On the other hand, this application also provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by the computer, the computer can execute the satellite Internet of Things resource allocation method provided by the above methods. The method includes: first, receiving dynamic policy parameters generated by a ground station based on intent description data, and constructing a joint state vector based on network environment state information within the satellite coverage area and a security performance target threshold in the dynamic policy parameters; then, inputting the joint state vector into a pre-trained deep reinforcement learning decision network to obtain a joint resource allocation action, and executing the joint resource allocation action; wherein, the deep reinforcement learning decision network includes: a feature extraction and fusion network and a decision network; the feature extraction and fusion network is used to perform feature fusion and dimensionality reduction on the joint state vector to generate an environmental feature representation vector; the decision network takes the environmental feature representation vector as input and outputs the joint resource allocation action; the joint resource allocation action includes a communication resource allocation action and a security resource allocation action, and the security resource allocation action includes an interference channel selection action and a noise power allocation action for combating potential eavesdropping. Thus, by establishing an end-to-end learnable mapping system that extends from semantic operational intent to the joint allocation of underlying physical layer communication and security resources, the satellite IoT system achieves real-time and accurate response to high-level dynamic operational intent, as well as intelligent endogenous collaborative optimization of communication performance and security strength.

[0095] In another aspect, this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the satellite IoT resource allocation methods provided above. The method includes: first, receiving dynamic policy parameters generated by a ground station based on intent description data, and constructing a joint state vector based on network environment state information within the satellite coverage area and a security performance target threshold in the dynamic policy parameters; then, inputting the joint state vector into a pre-trained deep reinforcement learning decision network to obtain a joint resource allocation action, and executing the joint resource allocation action; wherein the deep reinforcement learning decision network includes a feature extraction and fusion network and a decision network; the feature extraction and fusion network is used to perform feature fusion and dimensionality reduction on the joint state vector to generate an environmental feature representation vector; the decision network takes the environmental feature representation vector as input and outputs the joint resource allocation action; the joint resource allocation action includes a communication resource allocation action and a security resource allocation action, the security resource allocation action including an interference channel selection action and a noise power allocation action to combat potential eavesdropping. Thus, by establishing an end-to-end learnable mapping system that extends from semantic operational intent to the joint allocation of underlying physical layer communication and security resources, the satellite IoT system achieves real-time and accurate response to high-level dynamic operational intent, as well as intelligent endogenous collaborative optimization of communication performance and security strength.

[0096] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0097] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A satellite Internet of Things (IoT) resource allocation method, characterized in that, Applied to satellite intelligent agents, the method includes: The system receives dynamic policy parameters generated by the ground station based on intent description data, and constructs a joint state vector based on network environment status information within the satellite coverage area and the security performance target threshold in the dynamic policy parameters. The joint state vector is input into a pre-trained deep reinforcement learning decision network to obtain a joint resource allocation action, and the joint resource allocation action is executed. The deep reinforcement learning decision network includes a feature extraction and fusion network and a decision network. The feature extraction and fusion network is used to perform feature fusion and dimensionality reduction on the joint state vector to generate an environmental feature representation vector. The decision network takes the environmental feature representation vector as input and outputs a joint resource allocation action. The joint resource allocation action includes a communication resource allocation action and a security resource allocation action. The security resource allocation action includes an interference channel selection action and a noise power allocation action to combat potential eavesdropping.

2. The method according to claim 1, characterized in that, The joint state vector contains state information in the communication dimension and state information in the security dimension; the state information in the communication dimension includes at least one of the following: terminal traffic queue length, channel state information, co-channel interference intensity, historical throughput and latency statistics, and terminal traffic type priority label; the state information in the security dimension includes at least one of the following: channel characteristic difference degree, real-time security threat index, and the security performance target threshold.

3. The method according to claim 1, characterized in that, The dynamic strategy parameters also include: a reward function weight adjustment vector; The dynamic strategy parameters are generated based on the following steps: Based on a preset structured description template, operational instructions are transformed into structured intent description data, and expected effect keywords and intensity indicator values ​​are extracted from the structured intent description data. By querying the intent knowledge base, the desired effect keywords are mapped to the reward function weight adjustment vector, and the value of the reward function weight adjustment vector is determined according to the intensity indicator value; When the desired effect keyword includes security requirements, the intensity indicator value is quantified by combining historical security event records and the current network situation, and the quantified intensity indicator value is used as the security performance target threshold.

4. The method according to claim 1, characterized in that, The feature extraction and fusion network includes: parallel graph convolutional networks and convolutional neural networks; The step of inputting the joint state vector into a pre-trained deep reinforcement learning decision network to obtain the joint resource allocation action includes: The graph convolutional network is used to model the topological connection between the terminal and the channel, as well as the terminal attribute features, and outputs node feature vectors. The convolutional neural network is also used to extract local correlation features from the channel state information matrix and outputs a channel feature map. The node feature vector is concatenated with the channel feature map and then fused and reduced in dimensionality by a fully connected layer to generate the environment feature representation vector.

5. The method according to claim 1, characterized in that, The method further includes: Calculate an intent alignment index to assess the degree of match between the execution effect of the joint resource allocation action and the operational intent expressed by the intent description data; The intent alignment metric is stored or output as a basis for system performance monitoring.

6. The method according to claim 1, characterized in that, After performing the joint resource allocation action, the method further includes: The immediate reward obtained after performing the joint resource allocation action is calculated using a multi-objective reward function; The weights of each sub-reward item in the multi-objective reward function are dynamically adjusted by the reward function weight adjustment vector; the security gain sub-reward in the multi-objective reward function is calculated based on the security performance target threshold.

7. The method according to claim 6, characterized in that, The calculation steps for the security gain sub-reward include: Determine the security action efficiency factor, and calculate the security gain sub-reward based on the obtained legitimate channel capacity, eavesdropping channel capacity, and the security performance target threshold; The security action efficiency factor is used to characterize the degree of reduction in eavesdropping channel capacity caused by unit noise power.

8. The method according to claim 1, characterized in that, The deep reinforcement learning decision network is trained and optimized based on the following steps: Experience sample data is constructed using the state information, action information, reward information, and next state information generated after executing the joint resource allocation action; The empirical sample data is stored in the empirical replay buffer, and the parameters of the deep reinforcement learning decision network are updated based on the sample data in the empirical replay buffer using the empirical replay mechanism.

9. The method according to claim 8, characterized in that, The method further includes: After completing local training, the local model parameter update is uploaded to the federated aggregation server. The federated aggregation server then assigns differentiated aggregation weights to the model parameter update of each satellite agent based on the importance of the operational intent currently carried by each satellite agent, and generates global model parameters through weighted aggregation. Receive the global model parameters issued by the federated aggregation server, and update the local model using the global model parameters.

10. A satellite Internet of Things (IoT) resource allocation system, characterized in that, include: The system consists of a ground station and multiple satellites; the ground station transmits dynamic strategy parameters to the satellites via a satellite-to-ground link; each satellite is equipped with an intelligent agent. The system is used to execute the satellite Internet of Things resource allocation method as described in any one of claims 1 to 9.

Images