Star-ground link anti-interference decision method and system based on hybrid action mapping and online migration fine tuning

Abstract

The application discloses a kind of star-ground link anti-interference decision-making method and system based on hybrid action mapping and online migration fine-tuning, for the resource joint regulation and control problem under intelligent interference of star-ground link, method includes: obtaining the state space vector containing link and interference characteristics;The vector is input into policy generation network, and a hybrid action vector containing continuous and discrete control parameters is generated;Then the continuous output is converted into the physical execution gear suitable for star-ground link by smoothing mapping mechanism;After executing control, the instantaneous reward is calculated according to the evaluation system containing dynamic penalty mechanism, and the network parameters are fine-tuned using online learning mechanism, which can reduce the action space expansion problem caused by traditional high-dimensional discretization, realize the connection of offline pre-training and online adaptation, and improve the response capability and communication reliability of star-ground link under complex sudden electromagnetic interference.

Description

Technical Field

[0001] This invention relates to the fields of satellite communication, wireless communication anti-interference and intelligent decision-making, and in particular to a satellite-to-ground link anti-interference decision-making method and system based on hybrid action mapping and online migration fine-tuning. Background Technology

[0002] With the rapid development of space information networks, the satellite-to-ground link, as a key carrier channel of the integrated space-ground network, is directly related to the system's security in terms of high reliability and high survivability. Due to the broadcast characteristics of wireless communication links and the openness of spectrum resources, satellite-to-ground links face security threats from both suppression and intelligent deception interference in complex electromagnetic environments.

[0003] For the anti-jamming decision-making problem in satellite-to-ground links, traditional rule-based reasoning or static machine learning methods are effective under known interference patterns, but they often fail when facing intelligent interference with dynamic evolution capabilities due to a lack of generalization ability. In recent years, deep reinforcement learning has provided new ideas for intelligent anti-jamming, but it still faces three major technical bottlenecks in practical applications of satellite-to-ground links: First, the resource regulation of satellite-to-ground links involves a mixture of constraints on transmit power (a continuous variable), modulation and coding, and data rate (discrete gears). Traditional deep reinforcement learning is prone to the curse of dimensionality when dealing with such a high-dimensional continuous and discrete engineering action space, and it cannot directly output the gear instructions that can be executed at the engineering level.

[0004] Secondly, satellite-to-ground links have physical characteristics such as large propagation delays and extremely high costs for error correction and retransmission. Conventional trial-and-error online learning mechanisms are prone to causing long-term communication paralysis during the exploration phase.

[0005] Third, when faced with sudden and unknown interference, conventional algorithms have high cold start costs, slow convergence, and lack the ability to quickly migrate and adaptively reconstruct in heterogeneous electromagnetic environments.

[0006] Therefore, there is an urgent need for an intelligent anti-interference decision-making method that addresses the joint adjustment requirements of continuous power control and discrete coding rate, data rate, and anti-interference suppression state in satellite-to-ground links. This method aims to avoid the expansion of the action space caused by the full combination discretization of multi-dimensional actions and to convert the policy network output into engineering-executable link control parameters. Simultaneously, by combining offline pre-training with online updates, the method can improve the adaptability of the decision-making strategy to changes in the interference environment and reduce the cold start cost caused by training from scratch in a new interference environment. Summary of the Invention

[0007] The purpose of this invention is to solve the problem of high-dimensional optimization of the hybrid action space of satellite-to-ground links under intelligent interference, as well as the problem of online rapid adaptation and reliability assurance during cross-environment migration. A satellite-to-ground link anti-interference decision-making method and system based on hybrid action mapping and online migration fine-tuning is proposed.

[0008] The first aspect of this invention provides a satellite-to-ground link anti-jamming decision-making method based on hybrid action mapping and online migration fine-tuning, comprising: Obtain the real-time state characteristics of the satellite-to-ground link and construct a state space vector; The state space vector is input into the policy generation network to generate a hybrid action vector containing continuous control variables, discrete mapping variables, and control threshold variables; wherein, the initial parameters of the policy generation network are obtained through offline pre-training based on a preset interference environment; According to the preset smooth mapping rules, the hybrid action vector is converted into control commands adapted to the control gear of the satellite-to-ground link project, and the control commands are executed. Obtain the link performance feedback after executing the control command, and calculate the instantaneous reward based on throughput performance, bit error rate, and resource cost. Based on the empirical data constituted by the current state space vector, the control command, the instantaneous reward, and the state information at the next moment, the policy generation network is fine-tuned and updated using an online learning mechanism to output anti-interference decisions for the current interference environment.

[0009] Preferably, the step of acquiring the real-time state characteristics of the satellite-to-ground link and constructing the state space vector includes: Obtain the transmit power, coding rate, and data rate at the previous decision moment; Obtain the current environmental interference intensity and type; Statistically analyze the historical bit error rate sequence within a preset time window; The state space vector is constructed by combining the above features.

[0010] Preferably, the step of converting the hybrid motion vector into a control command adapted to the control level of the satellite-to-ground link project according to a preset smoothing mapping rule includes: For the continuous control variables in the hybrid action vector, constrain them to a preset value range to obtain continuous engineering control parameters; For the discrete mapping variables in the hybrid action vector, they are mapped to discrete coding rate levels and / or discrete data rate levels according to preset interval division rules or adjacent level rules. For the control threshold variable in the hybrid action vector, it is compared with a preset threshold. The activation state of the anti-interference notch filter is determined according to the comparison result, and mapped to notch depth and / or working bandwidth parameters in the activation state. The continuous engineering control parameters, the discrete encoding rate settings, the discrete data rate settings, and the activation state and operating parameters of the notch filter are encapsulated into the control instructions.

[0011] Preferably, the calculation of instantaneous reward based on throughput performance, bit error rate, and resource cost includes: Obtain the actual throughput, bit error rate, and transmit power of the link after executing the control command; Calculate the positive revenue item based on the actual throughput of the link; Calculate a bit error rate penalty term based on the bit error rate, and calculate a power consumption penalty term based on the transmit power; The instantaneous reward is obtained by weighted summation of the positive reward term, the bit error rate penalty term, and the power consumption penalty term.

[0012] Preferably, the calculation of the instantaneous reward also includes a suppressed resource matching determination and a dynamic penalty mechanism: Determine whether the data rate level corresponding to the current control command is a preset target level subject to interference; if so, increase the bit error rate penalty item according to the preset penalty amplification factor. The dynamic penalty mechanism also includes a time-adaptive logic based on the decision cycle: when the current resource configuration is continuously in the interfered resource area for multiple consecutive decision cycles, the weight coefficient is further increased according to a preset step rule or proportional rule; when the resource configuration is switched to a non-suppressed resource area and the link performance is restored to stability, the weight coefficient returns to the basic weight, or is smoothly reduced and reset according to a preset attenuation strategy.

[0013] Preferably, the fine-tuning and updating of the policy generation network using an online learning mechanism is initiated through the following triggering mechanism: A sudden change in the current interference intensity, interference type, or affected resource region relative to the reference time window has been detected; or, If the bit error rate is detected to be higher than a preset reliability threshold for multiple consecutive decision cycles; or, The average bit error rate within the sliding window was detected to have increased by more than a preset percentage relative to the historical baseline.

[0014] Preferably, the parameter configuration and optimization of the policy generation network adopts a two-stage transfer learning architecture, including: Phase 1: Offline pre-training is performed in a preset first interference environment to learn the basic physical mapping relationship between transmit power, coding rate, data rate and link bit error rate, and to obtain the initial parameters of the policy generation network. The second stage: In the second interference environment where the satellite-to-ground link is actually operating, online fine-tuning and updating are performed based on the empirical data acquired in real time, starting from the initial parameters.

[0015] A second aspect of the present invention provides a satellite-to-ground link anti-interference decision system based on hybrid action mapping and online migration fine-tuning, comprising: The status acquisition module is used to acquire the real-time status characteristics of the satellite-to-ground link and construct a state space vector. An action generation module is used to input the state space vector into a policy generation network to generate a hybrid action vector containing continuous control variables, discrete mapping variables, and control threshold variables; wherein, the initial parameters of the policy generation network are obtained through offline pre-training based on a preset interference environment; The mapping execution module is used to convert the hybrid action vector into control commands adapted to the control gear of the satellite-to-ground link project according to the preset smooth mapping rules, and to execute the control commands. The reward calculation module is used to obtain the link performance feedback after the execution of the control command, and calculate the instantaneous reward based on the throughput performance, bit error rate and resource cost. The policy update module is used to fine-tune and update the policy generation network based on empirical data composed of the current state space vector, the control command, the instantaneous reward and the state information at the next moment, using an online learning mechanism, so as to output anti-interference decisions for the current interference environment.

[0016] A third aspect of the present invention provides an electronic device, characterized in that it comprises: The memory is used to store the processing program; The processor, when executing the processing program, implements the satellite-to-ground link anti-interference decision method based on hybrid action mapping and online migration fine-tuning as described in any one of claims 1 to 7.

[0017] A fourth aspect of the present invention provides a readable storage medium storing a processing program, which, when executed by a processor, implements the above-described satellite-to-ground link anti-interference decision-making method based on hybrid action mapping and online migration fine-tuning.

[0018] Compared with the prior art, the present invention has the following beneficial effects: This invention provides a satellite-to-ground link anti-interference decision-making method based on hybrid action mapping and online transfer fine-tuning. By using preset interval constraints and smooth mapping rules, it achieves the connection between continuous decision output and discrete control levels of the communication physical layer, which helps reduce the action combination expansion problem caused by the high-dimensional hybrid action space. The use of a triggering mechanism combining environmental feature mutation detection and performance degradation verification, along with a two-stage transfer learning framework, reduces the cold start cost in highly adversarial environments and improves response capability and online transfer efficiency under environmental mutations. Through a dynamic penalty mechanism based on suppressed resource matching judgment, it promotes resource coordination under reliability constraints in satellite-to-ground links with long latency and high retransmission costs, which helps shorten the system's ineffective dwell time in suppressed resource areas. Attached Figure Description

[0019] Figure 1 This is an overall flowchart of the satellite-to-ground link anti-interference decision-making method based on hybrid action mapping and online migration fine-tuning in an embodiment of the present invention; Figure 2 This is a schematic diagram of the hybrid action vector interval constraint and engineering gear mapping mechanism in an embodiment of the present invention; Figure 3 This is a flowchart of the two-stage migration and online fine-tuning process triggered by two layers in an embodiment of the present invention; Figure 4 This is a schematic diagram of a satellite-to-ground link anti-interference decision system based on hybrid action mapping and online migration fine-tuning in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] like Figure 1-2 As shown, this invention provides a satellite-to-ground link anti-jamming decision-making method based on hybrid action mapping and online transfer fine-tuning. This method senses the link state and the interference environment, generates anti-jamming strategies using deep reinforcement learning, and achieves rapid adaptation through a two-stage transfer mechanism. The method specifically includes the following steps: Step S1: Obtain the real-time state characteristics of the satellite-to-ground link and construct a state space vector. Specifically, the system first collects real-time operational data of the link through onboard or ground station sensors. This operational data includes: transmit power, coding rate, and data rate from the previous decision cycle, used to characterize the current resource allocation; interference intensity and type in the current environment, used to characterize the external threat situation; and a historical bit error rate sequence within a preset time window, used to reflect the link's quality change trend. These multi-dimensional features are combined and encoded to construct the state space vector for the current moment, serving as the input for subsequent decisions.

[0022] Step S2: Input the state space vector into the policy generation network to generate a hybrid action vector containing continuous control variables, discrete mapping variables, and control threshold variables. First, the features of each dimension in the state space vector are normalized and preprocessed to match the preset input format of the policy generation network's input layer. The normalized state space vector is then input into the policy generation network, and after forward propagation calculation by the policy generation network, the corresponding original continuous action vector is output. The initial parameters of the policy generation network are obtained through offline pre-training based on a preset interference environment. The hybrid action vector specifically includes: a continuous control variable for transmit power, a discrete mapping variable for coding rate, a discrete mapping variable for data rate, and an anti-interference notch filter activation threshold variable. This hybrid output method balances the continuous power adjustment characteristics with the discrete level limitations specified by the communication standard.

[0023] Step S3: According to the preset smoothing mapping rules, the hybrid action vector is converted into control commands adapted to the engineering control levels of the satellite-to-ground link, and the control commands are executed. Because the onboard equipment is limited by physical hardware standards, the continuous values ​​output by the strategy generation network need to be smoothly mapped into engineering-executable commands. The specific mapping logic is as follows: For the continuous control variables in the hybrid action vector, they are constrained to a preset value range to obtain continuous engineering control parameters; For the discrete mapping variables in the hybrid action vector, they are mapped to discrete coding rate levels and / or discrete data rate levels according to preset interval division rules or adjacent level rules. For the control threshold variable in the hybrid action vector, it is compared with a preset threshold. The activation state of the anti-interference notch filter is determined according to the comparison result, and it is mapped to notch depth and / or working bandwidth parameters in the activation state. The continuous engineering control parameters, the discrete encoding rate settings, the discrete data rate settings, and the activation state and operating parameters of the notch filter are encapsulated into the control instructions.

[0024] Subsequently, the mapped discrete gears and continuous parameters are packaged and encapsulated into link control command protocol frames, and directly sent and written to the underlying hardware registers of the transmit power control unit, modulation and coding configuration unit, and anti-interference filtering control unit of the satellite-to-ground communication system, driving the physical devices to complete the adaptive reconstruction of link resources.

[0025] Step S4: Obtain the link performance feedback after executing the control command, and calculate the instantaneous reward based on the throughput performance, bit error rate, and resource cost.

[0026] First, the actual throughput, bit error rate, and transmit power of the link after executing the control command are collected; a positive reward term is calculated based on the actual throughput, a bit error rate penalty term is calculated based on the bit error rate, and a power consumption penalty term is calculated based on the transmit power; the above terms are weighted and summed to obtain the basic instantaneous reward.

[0027] Furthermore, this step also includes a suppressed resource matching determination and dynamic penalty mechanism: determining whether the frequency resources, rate resources, or modulation and coding resources corresponding to the current resource configuration match the interfered resource region. If a match is found, the weight coefficient of the bit error rate penalty term in the instantaneous reward is increased.

[0028] The mechanism incorporates time-adaptive logic based on decision cycles: when the current resource configuration remains in the interfered resource region for multiple consecutive decision cycles, a cumulative penalty mode is triggered, where the weight coefficient increases non-linearly according to a preset tiered or proportional rule; when the resource configuration switches to a non-suppressed resource region and link performance stabilizes, a penalty release mode is triggered, where the weight coefficient reverts to its base weight or undergoes a smooth reduction reset according to a preset attenuation strategy. This mechanism encourages the policy generation network to shorten its dwell time in the failed policy region.

[0029] Step S5: Based on the current state space vector, the control command, the instantaneous reward, and the state information at the next moment, empirical data is constructed. The policy generation network is then fine-tuned and updated using an online learning mechanism to output an anti-interference decision for the current interference environment.

[0030] The parameter configuration and optimization of the policy generation network adopts a two-stage transfer learning architecture, including: Phase 1: Offline pre-training is performed in a preset first interference environment to learn the basic physical mapping relationship between transmit power, coding rate, data rate and link bit error rate, and to obtain the initial parameters of the policy generation network. The second stage: In the second interference environment where the satellite-to-ground link is actually operating, online fine-tuning and updating are performed based on the empirical data acquired in real time, starting from the initial parameters.

[0031] The online fine-tuning update is initiated by the following triggering mechanisms: a sudden change in the current interference intensity, interference type, or interfered resource area relative to the reference time window is detected; or, the bit error rate is detected to be higher than a preset reliability threshold within multiple consecutive decision periods; or, the average bit error rate within the sliding window is detected to have increased by more than a preset proportion relative to the historical benchmark.

[0032] This step utilizes a two-layer triggering mechanism to initiate online fine-tuning in order to cope with sudden, unknown interference.

[0033] Triggering logic: The system continuously monitors environmental and status characteristics. When a sudden change is detected in the current interference intensity, interference type, or interfered resource area relative to the reference time window (primary trigger), or when the bit error rate exceeds the preset reliability threshold for multiple consecutive decision cycles, or the average bit error rate within the sliding window increases by more than a preset proportion relative to the historical benchmark (secondary trigger / performance degradation check), the system determines that the current strategy has failed or the environment has changed drastically, and immediately triggers the online fine-tuning process.

[0034] Fine-tuning execution: Once triggered, the system inherits the initial parameters from the offline pre-training phase as the starting point for migration, avoiding communication interruptions caused by starting from scratch. The system stores the current state, executed control instructions, calculated instantaneous reward, and next-time state information as a four-tuple of experience data in the experience pool, and uses an online learning mechanism to iteratively update the parameters of the policy generation network.

[0035] Exit Mechanism: During the fine-tuning process, the system continuously evaluates the effectiveness of the strategy. When the link performance recovers and stabilizes within the preset safe operating range, or when the number of online fine-tuning rounds reaches the safe limit, the current round of adaptive process is considered to have ended, the new strategy is solidified, and regular anti-interference control continues to be executed.

[0036] like Figure 3 As shown, the above online fine-tuning process forms a closed loop: after the system collects the status, it sequentially judges whether there are sudden environmental changes and whether the bit error rate exceeds the limit. If any condition is met, it enters the fine-tuning update process until the exit condition is met, thereby realizing the rapid adaptation and anti-interference decision of the satellite-to-ground link in a complex dynamic electromagnetic environment.

[0037] In one embodiment, this embodiment focuses on how the hybrid action vector output by the policy generation network is transformed into an actual executable engineering control command for the space-to-ground link through interval constraints and smooth mapping. The specific process is as follows: 1) Definition of hybrid action vector: The policy generation network outputs a continuous hybrid action vector with a fixed dimension based on the currently perceived state space vector. In this embodiment, the vector is defined as a continuous output containing four components, corresponding to the transmit power control variable, coding rate mapping variable, data rate mapping variable, and anti-interference notch filter control variable, respectively. This vector is usually processed by an activation function at the network output layer, such as the Sigmoid activation function or the Tanh activation function, and is initially normalized to the interval [0,1] or [-1,1]. 2) Interval constraints of continuous control variables: For the transmit power control variable, since the transmit power of the spaceborne equipment has strict physical upper and lower limits, the system performs interval constraints on it. Specifically, the normalized power output variable is directly mapped to the actual physical transmit power limited interval of the space-to-ground link through linear inverse transformation or a preset limiting function, where the lower limit is Pmin and the upper limit is Pmax. When the theoretical value of the network output exceeds the boundary, the system performs saturation truncation processing, forcibly constraining it within the physical safety threshold, thereby forming continuous transmit power control commands. 3) Coding rate mapping rules: The coding rate of the satellite-to-ground link is limited by the communication standard (such as a specified low-density parity-check code LDPC or convolutional code), and can only be switched in a limited number of discrete levels (the preset coding level index set may include multiple discrete level numbers, which correspond to the coding configuration table in the actual project). This system adopts an interval partitioning mapping mechanism: the output interval of the coding rate mapping variable (e.g., [0,1]) is divided into multiple sub-intervals proportionally or according to weights, and the number of sub-intervals is consistent with the number of coding rate levels available in the actual project. When the output continuous variable falls into a certain sub-interval, the system converts it into the corresponding discrete coding rate level according to the preset mapping rules. 4) Data Rate Mapping Rules: Data rates also have strict engineering discrete level limitations. For example, the preset rate level set can include discrete rate levels such as 4, 8, 16, 32, and 64 kbit / s. The above values ​​are only examples and do not constitute a limitation on the scope of protection of this invention. Similar to the coding rate mapping, the system segments the continuous interval where the data rate mapping variable is located. To ensure the smoothness of the mapping, the system determines the continuous value subspace where the output variable is located based on the proximity criterion or preset threshold comparison logic, and maps it to the nearest target discrete data rate level. Notch Filter Threshold Control Rules: For the anti-interference notch filter control variable, the system maps it to the filter's activation state or suppression depth parameter.In practical implementation, a soft activation threshold (e.g., 0.5) can be set: when the continuous variable is greater than or equal to this threshold, it is mapped to activate the anti-interference notch filter, and the continuous value is further linearly mapped to the suppression depth or operating bandwidth of the notch filter; when it is less than this threshold, it is mapped to bypass the notch filter module to reduce the attenuation of normal signals. 6) Engineering control command generation and execution process: After the above interval constraints and smooth mapping, the continuous hybrid action vector is completely converted into a discrete / continuous hybrid parameter set containing absolute power values, specific coding rate levels, specific data rate levels, and notch filter operating states. Subsequently, the mapped discrete levels and continuous parameters are packaged and encapsulated into a link control command protocol frame, and directly sent and written to the underlying hardware registers of the transmit power control unit, modulation and coding configuration unit, and anti-interference filtering control unit of the satellite-to-ground communication system, driving the physical equipment to complete the adaptive reconstruction of link resources. In this way, compared with the processing method of directly discretizing multi-dimensional actions through full combination, the smooth mapping mechanism described in this embodiment is beneficial to reducing the size of the action space and enabling the continuous output of the strategy generation network to adapt to the discrete control constraints of satellite communication hardware, thereby realizing the engineering connection between the decision layer output and the physical layer execution.

[0038] In one embodiment, this embodiment focuses on how the satellite-to-ground link establishes basic decision-making capabilities through offline pre-training when facing complex and ever-changing electromagnetic threats, and completes online fine-tuning in the target interference environment by relying on a two-layer triggering mechanism, thereby forming an adaptive anti-interference decision for unknown interference environments. The specific implementation process is as follows: including the following steps: S501: Perform offline pre-training in a preset first interference environment to obtain the initial parameters of the policy generation network; S502: Collect the current state of the satellite-to-ground link and the feedback information after executing control commands in real time; S503: Determine whether there are sudden changes in environmental or channel characteristics. If so, proceed to step S505; otherwise, proceed to step S504; S504: Determine whether the bit error rate continuously exceeds the limit and deteriorates. If so, proceed to step S505; otherwise, maintain the current decision policy and return to step S502; S505: Enter the online fine-tuning process, including: constructing empirical data containing the current state, control commands, instantaneous rewards, and the state at the next moment, and using an online learning mechanism to update the parameters of the policy generation network; S506: Determine whether the link performance has recovered to stability. If it has recovered to stability, the fine-tuning exits and the new policy is solidified; if it has not recovered to stability, return to step S505. The specific details are as follows: 1) First stage, offline pre-training and initial policy construction: In the initial stage of system deployment, the policy generation network is first pre-trained offline in a preset first interference environment (preferably a broadband background interference environment, characterized by full-band coverage and random power fluctuations). In this stage, the system learns the basic physical mapping relationship between transmit power, coding rate, data rate and link bit error rate through full interaction with the environment, converges and obtains the initial parameters of the policy generation network, which serve as the starting parameters for subsequent cross-environment migration and online fine-tuning. These initial parameters provide a starting point with basic physical knowledge for subsequent cross-environment migration, which helps to reduce cold start costs. 2) Second stage, two-layer triggering mechanism for online fine-tuning: In the actual satellite-to-ground communication process, the system continuously collects environmental and state features in the current state space vector. To accurately detect the intervention of unknown interference (such as sudden intelligent narrowband suppression interference), the system employs a two-layer triggering logic combining "environmental feature mutation detection" and "performance degradation verification": The main triggering mechanism (environmental feature mutation detection) extracts the current interference intensity, interference type, and distribution of the affected resource area, and performs a sliding comparison with historical features in a preset reference window or a previous time window. When a jump in the interference type identifier, an interference intensity change gradient exceeding a preset threshold, or a sudden change in the overlap between the current resource configuration (such as the frequency band corresponding to the highest data rate) and the affected spectrum region is detected, it is determined that the interference environment of the link has undergone a substantial change, and the online fine-tuning process is immediately triggered. This mechanism, acting as a feedforward sensing mechanism, can trigger the online fine-tuning process before significant link performance degradation occurs.Auxiliary Trigger / Backup Mechanism (Performance Degradation Verification): To prevent primary triggering from being missed due to concealed interference characteristics, the system synchronously monitors the transmission quality feedback of the service link. When the bit error rate (BER) is higher than the preset reliability threshold for N consecutive decision cycles, where N is a preset positive integer, or when the average BER within the current sliding window increases by more than a preset proportion compared to the historical baseline state, the system determines that the current strategy has failed, even if the environmental characteristics do not show obvious abrupt changes, and forcibly triggers the online fine-tuning process as a backup mechanism. 3) Online Experience Construction and Parameter Fine-tuning Execution: Once online fine-tuning is triggered, the system will inherit the initial network parameters obtained from offline pre-training as the migration starting point to avoid long-term communication interruptions caused by starting from scratch. Under target interference conditions, the system controls the satellite-to-ground link to execute engineering control instructions generated by the current strategy and collects in real time: the current state space vector, the actual executed control instructions, the instantaneous reward calculated according to the preset dynamic penalty function, and the state information at the next moment. The above four-tuple data are stored in the online experience pool. Subsequently, based on a preset online update cycle (or experience batch), the system extracts samples from the experience pool and uses an online learning mechanism to iteratively update the parameters of the policy generation network, enabling the system to gradually form an anti-interference decision-making strategy to avoid suppressed resources. 4) Fine-tuning exit and policy solidification: During the fine-tuning iteration process, the system continuously evaluates the execution effect of the updated strategy. When the link performance recovers and stabilizes within the preset safe operating range (e.g., the bit error rate is consistently lower than the reliability threshold and the throughput meets the minimum service requirements), or when the number of rounds of online fine-tuning reaches the preset safe upper limit, the system determines that the current round of online adaptive process has ended. After exiting the online fine-tuning state, the system outputs the updated policy generation network parameters and uses the solidified new strategy to execute subsequent conventional link anti-interference control. 5) Corresponding flowchart design description: e.g. Figure 3 As shown, after completing real-time status and feedback acquisition, the system enters the first decision node to determine whether there are sudden changes in environmental or channel characteristics. If the determination is yes, it enters the online fine-tuning and update process; if the determination is no, it enters the second decision node to determine whether the bit error rate has continuously exceeded the limit and deteriorated. If the determination is yes, it enters the online fine-tuning and update process; if the determination is no, it maintains the current decision strategy and continues to execute. After entering the online fine-tuning and update process, it sequentially executes online experience data construction, policy network parameter updates, and performance recovery decisions until the exit conditions are met to form a closed loop.

[0039] In one embodiment, this embodiment focuses on how to construct a reward calculation mechanism that balances throughput efficiency, communication reliability, and power cost during the anti-interference decision-making process of the satellite-to-ground link, and how to guide the policy generation network to exit the failed policy area and form an avoidance strategy by dynamically increasing the bit error rate penalty intensity when the current resource configuration is detected to fall into the suppressed resource area. 1) Reward calculation objective: The satellite-to-ground link has the physical characteristics of large propagation delay and high error correction and retransmission costs. Therefore, the reward calculation objective of this system includes: prioritizing ensuring that the link communication reliability meets the preset constraints, while taking into account the improvement of throughput efficiency and reasonable control of power resources; when the system detects that the current resource configuration falls into the suppressed resource area, by increasing the penalty intensity, prompting the policy generation network to shorten the stay time in the failed policy area. 2) Basic reward composition: In order to comprehensively evaluate the link execution status, the system designs a reward evaluation mechanism composed of multiple indicators. The instantaneous reward includes positive gain items and negative penalty items. Positive gain items characterize the effective transmission capacity of the link, including at least the gain corresponding to the actual throughput of the link. Negative penalty items characterize communication quality degradation and resource consumption, including at least bit error rate penalty items and power consumption penalty items. Under the basic state, the system sets preset basic weight coefficients for each component to maintain multi-objective coordination under normal communication conditions. 3) Dynamic penalty triggering mechanism: To cope with the targeted suppression of specific resources by intelligent interference, the system sets a suppressed resource matching judgment entry. When it is determined that the frequency resource, rate resource, or modulation and coding resource corresponding to the current resource configuration matches the interfered resource area, the dynamic penalty mechanism is triggered. The judgment entry includes, but is not limited to: the center frequency point and / or bandwidth corresponding to the currently executed configuration overlaps with the detected or estimated suppressed frequency band; the channel resource corresponding to the high-rate discrete gear of the current configuration is identified as a priority interfered resource area; the spectral overlap between the current resource configuration and the interference area exceeds a preset safety threshold; under the current configuration, the link bit error rate deteriorates rapidly, and its change characteristics are consistent with the characteristics of the identified interfered area. 4) Dynamic Adjustment Rules for Penalty Weights: After the dynamic penalty mechanism is triggered, the system dynamically adjusts the weight coefficients corresponding to the bit error rate penalty items. Specifically, this includes: increasing the weight coefficients corresponding to the bit error rate penalty items to increase the impact weight of the penalty items in the instantaneous reward, thereby applying negative feedback to the policy generation network; when the system detects that the current configuration is continuously in the interfered resource area for multiple consecutive decision cycles, the weight coefficients of the bit error rate penalty items are further increased according to preset tiered rules or proportional rules to shorten the maintenance time of the continuously failing configuration; when the system resource configuration switches to a non-suppressed resource area and the link performance recovers and remains stable, the weight coefficients are restored to the preset basic weights, or gradually fall back to the normal level according to preset attenuation rules.The basic weight coefficients and their dynamic adjustment rules can be preset according to link reliability requirements, resource consumption constraints, and preset service levels, or determined through the offline training phase. 5) Instantaneous reward calculation and policy update interface: At the end of each decision cycle, the system calculates the corresponding instantaneous reward value based on the actual collected throughput, bit error rate, and power consumption, combined with the current weight coefficients. This instantaneous reward value, together with the current state space vector, the executed engineering control instructions, and the state information at the next moment, constitutes empirical data and is written into the online empirical data pool. The online learning mechanism updates the policy generation network parameters by extracting the empirical data to achieve closed-loop optimization of anti-interference decision-making.

[0040] Therefore, by introducing a suppressed resource matching determination and a dynamic penalty mechanism, the reward calculation method provided in this embodiment is beneficial to shorten the ineffective trial and error time of the system in the suppressed resource area; improve the online adaptability of the policy generation network in complex electromagnetic environments; and facilitate the coordinated balance between throughput and power cost under link reliability constraints.

[0041] Based on the same concept, this invention provides a satellite-to-ground link anti-interference decision system based on hybrid action mapping and online migration fine-tuning, such as... Figure 4 As shown, it includes: The status acquisition module is used to acquire the real-time status characteristics of the satellite-to-ground link and construct a state space vector. An action generation module is used to input the state space vector into a policy generation network to generate a hybrid action vector containing continuous control variables, discrete mapping variables, and control threshold variables; wherein, the initial parameters of the policy generation network are obtained through offline pre-training based on a preset interference environment; The mapping execution module is used to convert the hybrid action vector into control commands adapted to the control gear of the satellite-to-ground link project according to the preset smooth mapping rules, and to execute the control commands. The reward calculation module is used to obtain the link performance feedback after the execution of the control command, and calculate the instantaneous reward based on the throughput performance, bit error rate and resource cost. The policy update module is used to fine-tune and update the policy generation network based on empirical data composed of the current state space vector, the control command, the instantaneous reward, and the state information at the next moment, using an online learning mechanism to output an anti-interference decision for the current interference environment. The implementation principle of the above module has been described in the foregoing embodiments and will not be repeated here.

[0042] Based on the same concept, an electronic device is also provided in some embodiments of this application. This electronic device includes a memory and a processor, wherein the memory stores a processing program, and the processor executes the processing program according to instructions. When the processor executes the processing program, the satellite-to-ground link anti-interference decision-making method based on hybrid action mapping and online migration fine-tuning described in the foregoing embodiments is realized.

[0043] In some embodiments of this application, a readable storage medium is also provided, which can be a non-volatile readable storage medium or a volatile readable storage medium. The readable storage medium stores instructions that, when executed on a computer, cause an electronic device containing such a readable storage medium to perform the aforementioned satellite-to-ground link anti-interference decision-making method based on hybrid action mapping and online migration fine-tuning.

[0044] It is understood that the aforementioned satellite-to-ground link anti-interference decision-making methods based on hybrid action mapping and online migration fine-tuning, if implemented as software functional modules and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0045] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0046] The program code for executing the technical solutions disclosed in this application can be written in any combination of one or more programming languages. These programming languages ​​include object-oriented programming languages—such as Java and C++—and conventional procedural programming languages—such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0047] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A satellite-to-ground link anti-interference decision-making method based on hybrid action mapping and online migration fine-tuning, characterized in that, include: Obtain the real-time state characteristics of the satellite-to-ground link and construct a state space vector; The state space vector is input into the policy generation network to generate a hybrid action vector containing continuous control variables, discrete mapping variables, and control threshold variables; wherein, the initial parameters of the policy generation network are obtained through offline pre-training based on a preset interference environment; According to the preset smooth mapping rules, the hybrid action vector is converted into control commands adapted to the control gear of the satellite-to-ground link project, and the control commands are executed. Obtain the link performance feedback after executing the control command, and calculate the instantaneous reward based on throughput performance, bit error rate, and resource cost. Based on the empirical data constituted by the current state space vector, the control command, the instantaneous reward, and the state information at the next moment, the policy generation network is fine-tuned and updated using an online learning mechanism to output anti-interference decisions for the current interference environment.

2. The satellite-to-ground link anti-interference decision-making method based on hybrid action mapping and online migration fine-tuning as described in claim 1, characterized in that, The process of acquiring the real-time state characteristics of the satellite-to-ground link and constructing the state space vector includes: Obtain the transmit power, coding rate, and data rate at the previous decision moment; Obtain the current environmental interference intensity and type; Statistically analyze the historical bit error rate sequence within a preset time window; The state space vector is constructed by combining the above features.

3. The satellite-to-ground link anti-interference decision-making method based on hybrid action mapping and online migration fine-tuning as described in claim 1, characterized in that, The step of converting the hybrid motion vector into control commands adapted to the adjustment levels of the satellite-to-ground link project according to a preset smoothing mapping rule includes: For the continuous control variables in the hybrid action vector, constrain them to a preset value range to obtain continuous engineering control parameters; For the discrete mapping variables in the hybrid action vector, they are mapped to discrete coding rate levels and / or discrete data rate levels according to preset interval division rules or adjacent level rules. For the control threshold variable in the hybrid action vector, it is compared with a preset threshold. The activation state of the anti-interference notch filter is determined according to the comparison result, and it is mapped to notch depth and / or working bandwidth parameters in the activation state. The continuous engineering control parameters, the discrete encoding rate settings, the discrete data rate settings, and the activation state and operating parameters of the notch filter are encapsulated into the control instructions.

4. The satellite-to-ground link anti-interference decision-making method based on hybrid action mapping and online migration fine-tuning as described in claim 1, characterized in that, The instantaneous reward calculation based on throughput performance, bit error rate, and resource cost includes: Obtain the actual throughput, bit error rate, and transmit power of the link after executing the control command; Calculate the positive revenue item based on the actual throughput of the link; Calculate a bit error rate penalty term based on the bit error rate, and calculate a power consumption penalty term based on the transmit power; The instantaneous reward is obtained by weighted summation of the positive reward term, the bit error rate penalty term, and the power consumption penalty term.

5. The satellite-to-ground link anti-interference decision-making method based on hybrid action mapping and online migration fine-tuning as described in claim 4, characterized in that, The calculation of the instantaneous reward also includes a suppressed resource matching determination and a dynamic penalty mechanism: Determine whether the data rate level corresponding to the current control command is a preset target level subject to interference; if so, increase the bit error rate penalty item according to the preset penalty amplification factor. The dynamic penalty mechanism also includes a time-adaptive logic based on the decision cycle: when the current resource configuration is continuously in the interfered resource area for multiple consecutive decision cycles, the weight coefficient is further increased according to a preset step rule or proportional rule; when the resource configuration is switched to a non-suppressed resource area and the link performance is restored to stability, the weight coefficient returns to the basic weight, or is smoothly reduced and reset according to a preset attenuation strategy.

6. The satellite-to-ground link anti-interference decision-making method based on hybrid action mapping and online migration fine-tuning as described in claim 1, characterized in that, The fine-tuning and updating of the policy generation network using an online learning mechanism is initiated through the following triggering mechanism: A sudden change in the current interference intensity, interference type, or affected resource area relative to the reference time window has been detected. or, The bit error rate was detected to be higher than the preset reliability threshold for multiple consecutive decision cycles. or, The average bit error rate within the sliding window was detected to have increased by more than a preset percentage relative to the historical baseline.

7. The satellite-to-ground link anti-interference decision-making method based on hybrid action mapping and online migration fine-tuning as described in claim 1, characterized in that, The parameter configuration and optimization of the policy generation network adopts a two-stage transfer learning architecture, including: Phase 1: Offline pre-training is performed in a preset first interference environment to learn the basic physical mapping relationship between transmit power, coding rate, data rate and link bit error rate, and to obtain the initial parameters of the policy generation network. The second stage: In the second interference environment where the satellite-to-ground link is actually operating, online fine-tuning and updating are performed based on the empirical data acquired in real time, starting from the initial parameters.

8. A satellite-to-ground link anti-interference decision-making system based on hybrid action mapping and online migration fine-tuning, characterized in that, include: The status acquisition module is used to acquire the real-time status characteristics of the satellite-to-ground link and construct a state space vector. An action generation module is used to input the state space vector into a policy generation network to generate a hybrid action vector containing continuous control variables, discrete mapping variables, and control threshold variables; wherein, the initial parameters of the policy generation network are obtained through offline pre-training based on a preset interference environment; The mapping execution module is used to convert the hybrid action vector into control commands adapted to the control gear of the satellite-to-ground link project according to the preset smooth mapping rules, and to execute the control commands. The reward calculation module is used to obtain the link performance feedback after the execution of the control command, and calculate the instantaneous reward based on the throughput performance, bit error rate and resource cost. The policy update module is used to fine-tune and update the policy generation network based on empirical data composed of the current state space vector, the control command, the instantaneous reward and the state information at the next moment, using an online learning mechanism, so as to output anti-interference decisions for the current interference environment.

9. An electronic device, characterized in that, include: The memory is used to store the processing program; The processor, when executing the processing program, implements the satellite-to-ground link anti-interference decision method based on hybrid action mapping and online migration fine-tuning as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that, The readable storage medium stores a processing program, which, when executed by a processor, implements the satellite-to-ground link anti-interference decision-making method as described in any one of claims 1 to 7.

Images