Satellite communication resource scheduling method and system, electronic equipment and storage medium

Abstract

The invention provides a satellite communication resource scheduling method and system, electronic equipment and a storage medium, and relates to the technical field of communication, and the method comprises the steps: obtaining a current state of a satellite communication system; determining a corresponding action for resource allocation according to the current state of the satellite communication system; executing the action, and obtaining a corresponding execution result; constructing a strategy optimization target, and updating a current state and an action value function of the satellite communication system according to the execution result and the strategy optimization target; the steps are repeated until the action value function is updated, and an updated action value function is obtained; and according to the updated action value function, taking a corresponding action in the current state of the satellite communication system so as to realize reasonable scheduling of resources of the satellite communication system. According to the invention, the business service quality of the satellite terminal user can be enhanced.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a satellite communication resource scheduling method, system, electronic equipment and non-transitory computer-readable storage medium. Background Art

[0002] Nowadays, satellite platform capabilities have been improved, antenna apertures have been increased, communication frequency bands have been increased, and the orbit of communication satellites has been lowered from geosynchronous orbit (GEO) to medium orbit or low orbit, forming a new communication mode in which satellites sparsely cover users with multiple movable narrow beams. The introduction of narrow beam technology has made satellite launch capabilities more concentrated, and the EIRP (equivalent isotropic radiated power) and G / T value (antenna gain to noise temperature ratio) have been improved, thereby increasing the user's communication rate. In this narrow beam situation, users need to apply for specific beam resources before communication so that the narrow beam can accurately cover their location. Resource scheduling involves the reasonable allocation of multiple wireless communication resources such as beams, frequencies, and time slots.

[0003] However, in the context of satellite communication resource scheduling, satellite attitude control has become a crucial link, because the satellite's attitude determines the precise pointing of the communication antenna. Satellite attitude control usually relies on magnetic torquers, which generate torque by interacting with the earth's magnetic field to adjust the satellite's attitude and ensure that the communication beam can accurately cover the target area. Therefore, the performance and polarity measurement of magnetic torquers have an important impact on satellite communication resource scheduling.

[0004] There are still many challenges in the current method of measuring the polarity of the magnetic torquer. When measuring the polarity of the magnetic torquer using the traditional compass method, the swing of the pointer is not sensitive, especially when the magnetic moment is small, the measurement accuracy is often not high, resulting in ambiguous test results and easy misjudgment. This not only affects the accuracy of attitude control, but also affects the pointing direction of the satellite antenna, which directly affects the efficiency of resource scheduling. In addition, after the magnetic torquer is installed on the satellite as a whole, due to location restrictions, the test becomes more complicated and easy to damage other components, increasing the safety risks of the test. Especially in small satellite systems, the polarity test steps are more cumbersome, and the operator needs to work closely with the test commander. The limitations of site and time further increase the complexity of the operation. Summary of the invention

[0005] The present invention aims at the technical problems existing in the prior art and provides a satellite communication resource scheduling method, system, electronic device and non-transitory computer-readable storage medium capable of enhancing the service quality of satellite terminal users.

[0006] The technical solution of the present invention to solve the above technical problems is as follows:

[0007] The present invention provides a satellite communication resource scheduling method, the method comprising:

[0008] Get the current status of the satellite communication system;

[0009] Determining a corresponding action for resource allocation according to a current state of the satellite communication system;

[0010] Execute the action and obtain the corresponding execution result;

[0011] Constructing a strategy optimization target, and updating the current state and action value function of the satellite communication system according to the execution result and the strategy optimization target;

[0012] Repeat the above steps until the action-value function is updated to obtain an updated action-value function;

[0013] According to the updated action-value function, corresponding actions are taken under the current state of the satellite communication system to achieve reasonable scheduling of resources of the satellite communication system.

[0014] Optionally, the construction strategy optimization objectives include:

[0015] Obtaining instant rewards for the execution results at various times;

[0016] Obtaining the discount factor of each instant reward and the strategy for selecting an action in each state;

[0017] The strategy optimization target is constructed based on the immediate reward, the discount factor and the strategy for selecting actions in each state.

[0018] Optionally, the strategy optimization objective is expressed as: π * = arg ma x π E[ ∑ t=0 ∞ γ t R t | π ];

[0019] in, is a strategy that defines the action selection in each state, is the discount factor, is at time t in state Execute an action After receiving the instant reward, is the expected value.

[0020] Optionally, updating the action value function of the satellite communication system according to the execution result and the strategy optimization target includes:

[0021] Get the maximum reward estimate among all possible actions to be performed in the next state;

[0022] The updated action-value function is determined according to the maximum reward estimate, the immediate reward and the reward estimate of the current state.

[0023] Optionally, the action value function is expressed as: Q(S t , A t ) ← Q(S t , A t )+α[ R t +γma x A ' Q S t+1 , A ' - Q(S t , A t )];

[0024] in, is the current state, is the current action, is the next state, is the time t in the state Execute an action After receiving the instant reward, is the learning rate, is the discount factor, For possible actions.

[0025] Optionally, the method further includes:

[0026] Preprocessing the original data to be allocated resources to obtain preprocessed data;

[0027] Processing each of the preprocessed data by using a judgment model to obtain a predicted magnetic torquer polarity;

[0028] According to the labels of the predicted magnetic torquer polarity and the actual polarity results, a loss function of the interpretation model is constructed, and the interpretation model is trained based on the loss function to obtain a trained interpretation model.

[0029] Optionally, the interpretation model is trained and optimized using deep reinforcement learning technology.

[0030] Optionally, the loss function of the interpretation model is expressed as:

[0031]

[0032] in, is the sample size, is the interpretation model, is the true polarity result, is the preprocessed data, The final judgment result.

[0033] Optionally, the action includes at least allocation of beam resources, frequency resources, and time slot resources.

[0034] The present invention also provides a satellite communication resource scheduling system, the system comprising:

[0035] A data acquisition module, used to obtain the current status of the satellite communication system;

[0036] an action determination module, used to determine a corresponding action for resource allocation according to a current state of the satellite communication system;

[0037] A result acquisition module, used to execute the action and obtain the corresponding execution result;

[0038] A function updating module, used for constructing a strategy optimization target, and updating a current state and an action value function of the satellite communication system according to the execution result and the strategy optimization target;

[0039] An iterative update module, used to repeat the above steps until the action-value function is updated to obtain an updated action-value function;

[0040] The resource scheduling module is used to take corresponding actions under the current state of the satellite communication system according to the updated action value function, so as to realize reasonable scheduling of resources of the satellite communication system.

[0041] In addition, to achieve the above-mentioned purpose, the present invention also proposes an electronic device, comprising: a memory for storing a computer software program; a processor for reading and executing the computer software program, thereby implementing a satellite communication resource scheduling method as described above.

[0042] In addition, to achieve the above-mentioned purpose, the present invention also proposes a non-transitory computer-readable storage medium, in which a computer software program is stored. When the computer software program is executed by a processor, a satellite communication resource scheduling method as described above is implemented.

[0043] The beneficial technical effects of the present invention are:

[0044] (1) By introducing deep reinforcement learning (DRL) technology, the present invention can dynamically adapt to changes in network conditions and computing requirements, realize intelligent scheduling and optimal allocation of resources, and thus improve the utilization efficiency of communication resources;

[0045] (2) The present invention reduces the possibility of human misjudgment and improves the accuracy and reliability of the interpretation results through an automated and intelligent interpretation model;

[0046] (3) Through intelligent scheduling and optimization strategies, the system can optimize resource utilization and energy consumption management while ensuring communication quality, thereby improving the energy efficiency of the entire system and reducing energy consumption;

[0047] (4) In the space-ground integrated network environment, the present invention improves the accuracy of task allocation through intelligent task scheduling and resource management, ensures the timeliness of data processing and transmission, and meets the needs of delay-sensitive applications;

[0048] (5) The present invention combines the powerful data processing capabilities of cloud computing and the low latency advantage of edge computing, and achieves efficient task load balancing and coordination in an integrated air-ground-space network environment, thereby improving service quality.

[0049] In summary, the present invention significantly improves the performance, efficiency and flexibility of satellite communication systems through deep reinforcement learning and intelligent resource scheduling methods, solves many problems in traditional methods, and has significant technical advantages and application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 It is a scenario diagram of a satellite communication resource scheduling method provided by an embodiment of the present invention;

[0052] Figure 2 is a flow chart of a satellite communication resource scheduling method provided by an embodiment of the present invention;

[0053] Figure 3 It is a structural schematic diagram of a satellite communication resource scheduling system provided by an embodiment of the present invention;

[0054] Figure 4 is a schematic diagram of a possible hardware structure of an electronic device provided by an embodiment of the present invention;

[0055] Figure 5 It is a schematic diagram of the hardware structure of a possible computer-readable storage medium provided by an embodiment of the present invention.

[0056] Explanation of the accompanying drawings: 301 -data acquisition module, 302 -action determination module, 303 -result acquisition module, 304 -function update module, 305 -iteration update module, 306 -resource scheduling module, 400 -electronic device, 410 -memory, 411 -computer program, 420 -processor, 500 -computer-readable storage medium. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0059] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration or explanation". Any embodiment described as "for example" in the present invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present invention.

[0060] Figure 1 A scenario diagram of a satellite communication resource scheduling method provided by the present invention. Figure 1As shown, the terminal and the server are connected via a network, such as a wired or wireless network connection. The terminal may include but is not limited to portable terminals such as mobile phones and tablets installed with various network platform applications, as well as fixed terminals such as computers, query machines, and advertising machines. The server provides users with various business services, including service push servers, user recommendation servers, etc.

[0061] It should be noted that Figure 1 The scenario diagram of a satellite communication resource scheduling method shown is only an example. The terminal, server and application scenario described in the embodiment of the present invention are intended to more clearly illustrate the technical solution of the embodiment of the present invention, and do not generate limitations on the technical solution provided by the embodiment of the present invention. Ordinary technicians in this field can know that with the evolution of the system and the emergence of new business scenarios, the technical solution provided by the embodiment of the present invention is also applicable to similar technical problems.

[0062] The present invention provides a satellite communication resource scheduling method, such as Figure 2 As shown, the method comprises the following steps:

[0063] Step 201: Obtain the current status of the satellite communication system.

[0064] In some embodiments, the current status may include communication quality parameters, system status parameters, environmental conditions, data link status, operational status, and mission performance.

[0065] In some embodiments, the communication quality parameters may include EIRP (equivalent isotropically radiated power): a measure of the power density of satellite transmitted signals, affecting communication coverage and signal strength. G / T value (antenna gain to noise temperature ratio): assesses the sensitivity and performance of the satellite receiving system. Bit error rate (BER): measures the error rate of the received signal, directly affecting the communication quality. Link quality parameters: such as signal strength, fading, etc.

[0066] In some embodiments, the system status parameters may include antenna pointing angle: the pointing angle of the satellite antenna, which affects the communication coverage area. Spectrum utilization: the utilization rate of the currently used communication frequency band and spectrum resources. Energy consumption: the current energy consumption of the satellite system, involving solar panel output, battery status, etc.

[0067] In some embodiments, environmental conditions may include ground station location and coverage area: the location of the satellite in orbit and the geographical area covered. Atmospheric and weather conditions: atmospheric transparency, cloud conditions, etc. that affect the propagation of communication signals.

[0068] In some embodiments, the data link status may include the data link status between the ground station and the satellite: such as delay, packet loss rate, etc.

[0069] In some embodiments, the operating status and task execution status may include executing tasks and instructions: communication tasks or instructions currently being executed.

[0070] Step 202: Determine a corresponding action for resource allocation according to the current state of the satellite communication system.

[0071] In some embodiments, the action includes allocation of beam resources, frequency resources, and time slot resources.

[0072] In some embodiments, beam resource allocation includes selecting a beam direction: selecting the optimal beam direction for communication based on the current location of the satellite and the geographical distribution of users. Configuring beam parameters: adjusting the beam width and coverage of the beam to maximize the number of covered users or improve the communication quality of a single user. Dynamic beam adjustment: dynamically adjusting the beam direction and parameters based on real-time user location and communication needs to optimize communication efficiency.

[0073] In some embodiments, frequency resource allocation may include spectrum monitoring and management: real-time monitoring of the utilization of the current frequency band, selecting idle frequency resources for communication, and avoiding interference and collision. Frequency switching and allocation: dynamically allocating available frequencies according to the real-time conditions of the communication link and user needs to optimize communication quality and bandwidth utilization.

[0074] In some embodiments, time slot resource allocation may include a time slot scheduling algorithm: using an efficient time slot scheduling algorithm to ensure that data transmission of different users or tasks can be completed within a predetermined time slot to maximize resource utilization. Time slot priority management: allocating and adjusting the order and length of time slot usage according to the priority and timeliness requirements of the communication task.

[0075] In the implementation process, technologies such as deep reinforcement learning (DRL) can be combined to adjust the action selection strategy in real time according to the preset reward function (such as communication quality, system efficiency, etc.) to adapt to the changing network conditions and user needs. Such an intelligent resource allocation strategy can significantly improve the efficiency, performance and user satisfaction of satellite communication systems.

[0076] In some embodiments, the action of determining resource allocation based on the current state can be implemented in the following ways: First, the state information of the current system can be collected and analyzed, including the availability, demand and priority of resources. This can be obtained through sensors, monitoring systems or user input. Then, the goal of resource allocation can be clarified, such as improving efficiency, reducing costs or meeting specific user needs. Then, a decision model can be established based on the current state and goals. For example, it can be a rule-based model, an optimization model or a machine learning model, depending on the complexity of the problem and the available data. Then, the decision model can be used to generate possible resource allocation actions based on the current state. For example, these actions can be to allocate certain resources to specific tasks, or to adjust the allocation method of existing resources. After the selected action is implemented, its effect is monitored and the results are fed back to the system for subsequent adjustment and optimization. Finally, the resource allocation strategy can be regularly re-evaluated and adjusted based on new state information and feedback to ensure that the system can always effectively respond to changing needs and conditions.

[0077] Step 203: execute the action and obtain the corresponding execution result.

[0078] In some embodiments, taking the execution of beam resource allocation as an example, the current satellite position, attitude and antenna pointing angle can be obtained by using sensors built into the satellite system, telemetry data or ground station feedback. The position and communication needs of each user in the current communication system are analyzed. The priority and timeliness requirements of each task can be determined based on a predetermined communication task list (such as a task set {T1, T2, ..., Tn}).

[0079] In some embodiments, the optimal beam direction can be selected based on the current position of the satellite and the distribution of users, so that the communication can cover as many users as possible or users in a specific area. The parameters of the beam, including beam width, gain, and coverage, are adjusted to optimize the communication quality and user experience. For example, for long-distance communication needs, a narrow beam can be selected to improve signal strength and receiving sensitivity.

[0080] Accordingly, the execution result acquisition may include communication quality parameters, user feedback and data collection, and system performance evaluation, specifically:

[0081] Communication quality monitoring refers to real-time monitoring of the communication quality parameters of each beam, such as signal strength, bit error rate, etc. User feedback and data collection refers to the collection of feedback information from the user end, such as communication quality reports and user experience evaluations. System performance evaluation refers to the evaluation of the communication performance and efficiency of the system in its current state based on the beam configuration and adjustments performed.

[0082] Step 204: construct a strategy optimization target, and update the current state and action value function of the satellite communication system according to the execution result and the strategy optimization target.

[0083] In some embodiments, step 204 may include:

[0084] Obtaining instant rewards for the execution results at various times;

[0085] Obtaining the discount factor of each instant reward and the strategy for selecting an action in each state;

[0086] The strategy optimization target is constructed based on the immediate reward, the discount factor and the strategy for selecting actions in each state.

[0087] In some embodiments, the policy optimization objective is expressed as:

[0088] π * = arg ma x π E[ ∑ t=0 ∞ γ t R t | π ];

[0089] in, is a strategy that defines the action selection in each state, is the discount factor, is at time t in state Execute an action After receiving the instant reward, is the expected value.

[0090] In the specific implementation, assume that the task set is , the status of each task is The action taken is The reward is .

[0091] Each task Indicates a specific task that needs to be performed, for example, a specific data transmission task in satellite communications. yes The current status. For example, the current network status, task priority, etc. is in state For example, the bandwidth allocated to the task, the transmission path selected, etc. Is to perform an action The reward obtained after the successful transmission of data, for example, positive feedback obtained from successfully transmitting data, or negative feedback caused by delays.

[0092] The strategy optimization goal is to maximize the cumulative reward:

[0093] π * = arg ma x π E[ ∑ t=0 ∞ γ t R t | π ];

[0094] in, is the strategy, defined in each state Next select action rules;

[0095] is a discount factor between 0 and 1 that measures the importance of future rewards. Indicates the importance of future rewards, lower Indicates a greater focus on current, immediate rewards;

[0096] is the expected value;

[0097] is the reward obtained at time t. The reward can be immediate feedback or delayed feedback;

[0098] In the specific implementation, It means starting from time step t=0, all rewards obtained at future time steps t The weighted sum of . Discount factor Used to reduce the weight of future rewards to ensure they are cumulative and finite.

[0099] The expected value represents the average cumulative reward that you hope to obtain after multiple task executions under the strategy 𝜋. Because the tasks in reinforcement learning may be random and each task execution may have different results, the expected value is used to represent the average situation.

[0100] arg ma x π E[ ∑ t=0 ∞ γ t R t | π ] The goal is to find the strategy , so that under this strategy, the expected cumulative reward is maximized.

[0101] In some embodiments, the following scenarios can be used:

[0102] Dynamic resource allocation: Dynamically allocate satellite resources (such as frequency, bandwidth, time slot, etc.) according to real-time communication needs and network status to maximize data transmission efficiency and user satisfaction.

[0103] Path optimization: Select the best transmission path to reduce delay and data loss.

[0104] Energy efficiency optimization: Minimize energy consumption while ensuring communication quality.

[0105] Assume that in a specific satellite communication mission:

[0106] state is the current network load and user request status, action Assign specific frequencies and bandwidths to current tasks, rewards It is the user feedback score after successfully transmitting data. Through the reinforcement learning algorithm, the policy optimization target model can learn the optimal actions to be taken under different states, thereby maximizing the user feedback score in the long term and improving the overall system performance and resource utilization efficiency.

[0107] In some embodiments, after the system selects and executes an action based on the current state, the system will receive corresponding feedback. This feedback can be positive (such as improved communication quality, increased resource utilization efficiency) or negative (such as communication failure, resource waste). The instant reward the system receives after each action is executed It reflects the effectiveness of the action in the current state. The value of the immediate reward is determined by the reward function preset by the system, which takes into account various factors, such as resource utilization efficiency, communication success rate, etc. The system can use the discount factor Future rewards are weighed to ensure that the system not only focuses on the current immediate reward, but also optimizes the overall performance in the long run. The immediate reward is combined with the discount factor to construct the policy optimization target.

[0108] In summary, the present invention can effectively solve the complex decision-making problems in satellite communication resource scheduling by defining the state, action and reward of the task and using the strategy optimization method in reinforcement learning, achieve optimal resource allocation and management, and improve the overall efficiency and performance of the communication system.

[0109] In some embodiments, step 204 may further include:

[0110] Get the maximum reward estimate among all possible actions to be performed in the next state;

[0111] The updated action-value function is determined according to the maximum reward estimate, the immediate reward and the reward estimate of the current state.

[0112] Step 205: Repeat the above steps until the action-value function is updated to obtain an updated action-value function.

[0113] In some embodiments, the action-value function is expressed as: Q(S t , A t ) ← Q(S t , A t )+α[ R t +γma x A ' Q S t+1 , A ' - Q(S t , A t )];

[0114] in, is the current state, is the current action, is the next state, is the time t in the state Execute an action After receiving the instant reward, is the learning rate, is the discount factor, For possible actions.

[0115] In the specific implementation, is the action value function, which is in state Next action The Q value can be understood as the expected estimate of the future cumulative reward obtained after performing the action;

[0116] is the current state;

[0117] is the current action;

[0118] is the next state;

[0119] For the status Execute an action The immediate reward obtained after the task is completed can be a reward directly fed back by the environment or other measurement criteria defined according to the task. The reward refers to the numerical feedback obtained by the system based on the action performed at a certain time step or state. The reward is used to evaluate the quality of an action or strategy, and it is the goal of reinforcement learning algorithm optimization.

[0120] Specifically, a reward is usually a scalar value that can be positive, negative, or zero, which is used to represent the immediate effect of the system after performing an action. The goal of reinforcement learning is to maximize the long-term cumulative rewards by adjusting the strategy, rather than just a single immediate reward.

[0121] In the scenario of satellite communication resource scheduling of the present invention, the reward can be a comprehensive evaluation of various factors, such as the improvement of communication quality, the increase of resource utilization efficiency, the reduction of system energy consumption, etc. For example, a successful communication connection may give a positive reward, while a communication failure or a waste of resources may result in a negative reward. Rewards play a very important role in reinforcement learning, and they are the driving force for system learning and optimization.

[0122] The learning rate controls the step size of each Q value update. The higher the learning rate, the greater the impact of the update, but it may cause instability. The lower the learning rate, the more stable the update process is, but the learning speed is slower.

[0123] The discount factor is used to measure the importance of future rewards relative to current rewards. The discount factor is between 0 and 1. The closer it is to 1, the more importance is attached to future rewards, and the closer it is to 0, the more importance is attached to immediate rewards.

[0124] For possible actions;

[0125] In the specific implementation, For the next state Select all possible actions The maximum Q value in the equation represents the expected maximization of the future state. Q-learning updates the Q value based on the feedback from the environment by continuously interacting with the environment. The update process is completed by comparing and adjusting the current Q value with the target Q value calculated according to the Bellman equation. The target Q value consists of the current immediate reward and the expected future reward.

[0126] By continuously updating the Q value, Q-learning can learn a set of strategies for selecting the best action in different states. The ultimate goal is to learn an optimal strategy that maximizes the cumulative reward, that is, to select the action that maximizes the Q value in each state.

[0127] Under certain conditions, Q-learning can guarantee convergence to the optimal Q value and thus the optimal strategy. These conditions include coverage of the state and action space, sufficient number of learning times, and appropriate selection of learning rate and discount factor.

[0128] Q-learning and its variants are widely used in various reinforcement learning problems, such as: learning action strategies in complex environments, such as path planning and obstacle avoidance in autonomous driving. Learning to play video games, such as Atari games, and board games such as Go and chess. Optimizing network resource allocation, energy management, etc. The power of Q-learning lies in its simplicity and wide applicability, especially for problems where the state space and action space are relatively small and discrete.

[0129] In some embodiments, the following conditions can be set to determine whether the update of the action value function is complete: for example, a threshold can be set. When the change in the value function is less than the threshold, the update can be considered complete, indicating that after multiple iterations, the change in the value function tends to be stable. For another example, a maximum number of iterations can be preset. When this number is reached, the update can be forced to stop, regardless of whether the value function converges. This helps prevent the calculation time from being too long. For another example, after each update, the effectiveness of the value function can be judged by evaluating the performance of the current strategy (such as rewards, efficiency, etc.). If the performance has not improved significantly over a period of time, it means that the value function has been updated.

[0130] It should be noted that the strategy optimization goal It is to find the strategy that maximizes the long-term cumulative reward globally, indicating that we need to find the long-term optimal strategy. The action value function is obtained by local update , gradually find the strategy that makes the actions performed in each state maximize the future cumulative reward, and approach this optimal strategy through local step-by-step iteration. It can be understood that the two complement each other in the algorithm and are used together to determine the optimal resource allocation strategy.

[0131] In some embodiments, the method further comprises:

[0132] Preprocessing the original data to be allocated resources to obtain preprocessed data;

[0133] Processing each of the preprocessed data by using a judgment model to obtain a predicted magnetic torquer polarity;

[0134] According to the labels of the predicted magnetic torquer polarity and the actual polarity results, a loss function of the interpretation model is constructed, and the interpretation model is trained based on the loss function to obtain a trained interpretation model.

[0135] In some embodiments, the judgment model can be a deep learning model, which is trained and optimized using deep reinforcement learning technology. The judgment model can be used to process key data in satellite communication resource scheduling and make accurate decisions. In particular, after preprocessing the data, the model further analyzes and judges the data and finally outputs the decision results.

[0136] In some embodiments, the loss function of the interpretation model is expressed as:

[0137]

[0138] in, is the sample size, is the interpretation model, is the true polarity result, is the preprocessed data, The final judgment result.

[0139] In the specific implementation, is the number of samples, the total number of samples used to train or evaluate the model;

[0140] It is a judgment model, which accepts preprocessed data As input, and output the predicted polarity result;

[0141] is the true polarity result, which is the true label of each sample and is used to measure the accuracy of the model prediction results. For example, the label can be 1 or 0;

[0142] It is preprocessed data, which refers to the data collected by sensors and preliminarily processed as the input of the model;

[0143] For the final interpretation result, the output of model f is used here, that is, , used in the loss function replace.

[0144] Among them, the magnetic torquer is a device used for satellite attitude control. It generates torque by interacting with the earth's magnetic field to adjust the satellite's attitude. The magnetic torquer is a common control device for spacecraft such as small satellites and low-orbit satellites. The correctness of the polarity of the magnetic torquer will affect the accuracy of satellite attitude control, thereby indirectly affecting the pointing accuracy of the satellite communication antenna and the effective use of satellite communication resources. The adjustment of satellite attitude (including antenna direction) is crucial to ensure the effective scheduling of satellite communication resources. If the attitude adjustment is inaccurate, communication may be interrupted or performance may degrade. Therefore, by accurately judging the polarity of the magnetic torquer, precise control of the satellite attitude can be ensured, thereby optimizing the use of communication resources.

[0145] In the specific implementation, a machine learning algorithm is used to intelligently interpret the polarity test data of the magnetic torquer. The calculation process of the loss function is as follows:

[0146] Input preprocessed data : Preprocessed data for each sample i Input into the interpretation model f.

[0147] Model prediction: Interpret the polarity results predicted by the output of model f .

[0148] Calculate the error: For each sample i, calculate the prediction result With real results The difference between .

[0149] Square the errors: Square each difference to ensure that the errors are positive and to amplify the impact of larger errors.

[0150] Average error: Sum the squared errors of all samples and take the average, which is divided by the total number of samples N.

[0151] As you can understand, mean square error (MSE) is a commonly used loss function in machine learning, which measures the average deviation between the predicted result and the true result. The smaller the loss value, the closer the model prediction result is to the true result. Model optimization: By minimizing the loss function, adjust the model parameters to improve the model's interpretation accuracy.

[0152] In this solution, the loss function is used to train and evaluate the interpretation model for magnetic torquer polarity testing, and the interpretation model training can be performed in the cloud. The specific steps include:

[0153] Data collection and preprocessing: The polarity data of the magnetic torquer is collected in real time through a high-sensitivity sensor. The collected data is preprocessed to generate Specifically, the preprocessing operations may include data cleaning, data standardization or normalization, feature extraction, time series processing, data compression, data format conversion, data grouping and classification, etc. The above preprocessing operations can be completed on the edge node.

[0154] Model training: Preprocessed data can be used And the corresponding true polarity result As training samples, train the judgment model. Define the judgment model f and input the preprocessed data , output the prediction result By minimizing the loss function L, the model parameters are optimized and the prediction accuracy of the model is improved.

[0155] Model evaluation and application: Use the trained model f to predict the newly collected data. Output the final judgment result . Compare the predicted results with the true results to evaluate the model performance.

[0156] Through the above method, the present invention can efficiently and accurately judge the polarity of the magnetic torquer, reduce the risk of human misjudgment, and improve the safety and reliability of the system.

[0157] Step 206: According to the updated action-value function, corresponding actions are taken under the current state of the satellite communication system to achieve reasonable scheduling of resources of the satellite communication system.

[0158] In some embodiments, the system status parameters may include the reliability of each data source. , transmission power, data volume, etc., as well as the usage of computing resources and storage resources in the system. The action space can include adjusting the specific allocation values ​​of computing resource allocation and storage resource allocation, or selecting different resource allocation strategies (such as giving priority to high-reliability data sources, optimizing data transmission paths, etc.).

[0159] In some embodiments, the action value function can be updated based on historical data and real-time feedback to evaluate the effect of each possible action in the current state. The action value function usually takes into account the overall performance indicators of the system, such as data transmission efficiency, energy utilization, data processing delay, etc., as well as the impact on data reliability and accuracy.

[0160] In some embodiments, the selected actions may be performed to adjust resource allocation of computing nodes and storage nodes, ensuring that the implementation of resource allocation is effective and timely in the system to meet the current data processing needs of the system and future changes.

[0161] In some embodiments, the resource allocation effect after implementation can be monitored and evaluated, including indicators such as data processing efficiency, system response time, data transmission success rate, etc. The action value function is updated according to real-time feedback to further optimize the resource scheduling strategy.

[0162] In the above manner, the action value function updated according to the present invention can effectively help the satellite communication system to achieve reasonable resource scheduling. The resource scheduling strategy based on the updated action value function can significantly improve the overall performance and resource management efficiency of the satellite communication system, so that it can better meet future data processing needs and service requirements.

[0163] Figure 3 A schematic diagram of the structure of a satellite communication resource scheduling system provided by the present invention. The satellite communication resource scheduling system proposed by the present invention includes:

[0164] The data acquisition module 301 is used to acquire the current status of the satellite communication system;

[0165] An action determination module 302, configured to determine a corresponding action for resource allocation according to a current state of the satellite communication system;

[0166] The result acquisition module 303 is used to execute the action and obtain the corresponding execution result;

[0167] A function updating module 304 is used to construct a strategy optimization target and update the current state and action value function of the satellite communication system according to the execution result and the strategy optimization target;

[0168] Iterative update module 305, used to repeat the above steps until the action value function is updated to obtain an updated action value function;

[0169] The resource scheduling module 306 is used to take corresponding actions in the current state of the satellite communication system according to the updated action value function, so as to realize reasonable scheduling of resources of the satellite communication system.

[0170] Figure 4 This is a schematic diagram of an electronic device provided by an embodiment of the present invention. An electronic device 400 provided by the present invention includes a memory 410, a processor 420, and a computer program 411 stored in the memory 410 and executable on the processor 420. When the processor 420 executes the computer program 411, the following steps are implemented:

[0171] Get the current status of the satellite communication system;

[0172] Determining a corresponding action for resource allocation according to a current state of the satellite communication system;

[0173] Execute the action and obtain the corresponding execution result;

[0174] Constructing a strategy optimization target, and updating the current state and action value function of the satellite communication system according to the execution result and the strategy optimization target;

[0175] Repeat the above steps until the action-value function is updated to obtain an updated action-value function;

[0176] According to the updated action-value function, corresponding actions are taken under the current state of the satellite communication system to achieve reasonable scheduling of resources of the satellite communication system.

[0177] Figure 5 A schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present invention. Figure 5As shown, this embodiment provides a computer-readable storage medium 500, on which a computer program 411 is stored. When the computer program 411 is executed by the processor 420, the following steps are implemented:

[0178] Get the current status of the satellite communication system;

[0179] Determining a corresponding action for resource allocation according to a current state of the satellite communication system;

[0180] Execute the action and obtain the corresponding execution result;

[0181] Constructing a strategy optimization target, and updating the current state and action value function of the satellite communication system according to the execution result and the strategy optimization target;

[0182] Repeat the above steps until the action-value function is updated to obtain an updated action-value function;

[0183] According to the updated action-value function, corresponding actions are taken under the current state of the satellite communication system to achieve reasonable scheduling of resources of the satellite communication system.

[0184] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0185] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0186] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or multiple boxes.

[0187] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction system, which is implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0188] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0189] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0190] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A satellite communication resource scheduling method, characterized in that: The method comprises: Get the current status of the satellite communication system; Determining a corresponding action for resource allocation according to a current state of the satellite communication system; Execute the action and obtain the corresponding execution result; Constructing a strategy optimization target, and updating the current state and action value function of the satellite communication system according to the execution result and the strategy optimization target; Repeat the above steps until the action-value function is updated to obtain an updated action-value function; According to the updated action-value function, corresponding actions are taken under the current state of the satellite communication system to achieve reasonable scheduling of resources of the satellite communication system.

2. The satellite communication resource scheduling method according to claim 1, characterized in that: The construction strategy optimization objectives include: Obtaining instant rewards for the execution results at various times; Obtaining the discount factor of each instant reward and the strategy for selecting an action in each state; The strategy optimization target is constructed based on the immediate reward, the discount factor and the strategy for selecting actions in each state.

3. The satellite communication resource scheduling method according to claim 2, characterized in that: The strategy optimization objective is expressed as: ； in, is a strategy that defines the action selection in each state, is the discount factor, is at time t in state Execute an action After receiving the instant reward, is the expected value.

4. The satellite communication resource scheduling method according to claim 3, characterized in that: The updating of the action value function of the satellite communication system according to the execution result and the strategy optimization target comprises: Get the maximum reward estimate among all possible actions to be performed in the next state; The updated action-value function is determined according to the maximum reward estimate, the immediate reward and the reward estimate of the current state.

5. The satellite communication resource scheduling method according to claim 4, characterized in that: The action value function is expressed as: ； in, is the current state, is the current action, is the next state, is the state at time t Execute an action After receiving the instant reward, is the learning rate, is the discount factor, For possible actions.

6. The satellite communication resource scheduling method according to claim 5, characterized in that: The method further comprises: Preprocessing the original data to be allocated resources to obtain preprocessed data; Processing each of the preprocessed data by using a judgment model to obtain a predicted magnetic torquer polarity; According to the labels of the predicted magnetic torquer polarity and the actual polarity results, a loss function of the interpretation model is constructed, and the interpretation model is trained based on the loss function to obtain a trained interpretation model.

7. The satellite communication resource scheduling method according to claim 6, characterized in that: The interpretation model is trained and optimized using deep reinforcement learning technology.

8. The satellite communication resource scheduling method according to claim 7, characterized in that: The loss function of the interpretation model is expressed as: ； in, is the sample size, is the interpretation model, is the true polarity result, is the preprocessed data, The final judgment result.

9. The satellite communication resource scheduling method according to claim 8, characterized in that: The actions include allocation of beam resources, frequency resources and time slot resources.

10. A satellite communication resource scheduling system, characterized in that: The system includes a data acquisition module, an action determination module, a result acquisition module, a function update module, an iterative update module and a resource scheduling module, which are used to execute the satellite communication resource scheduling method as described in any one of claims 1-9.

11. An electronic device, characterized in that: include: Memory for storing computer software programs; A processor, configured to read and execute the computer software program, thereby implementing the method according to any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium, characterized in that: The storage medium stores a computer software program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images