Space-air-ground integrated secure transmission method and system based on laser radio frequency fusion link

Abstract

The application discloses a space-air-ground integrated secure transmission method and system based on a laser radio frequency fusion link, calculates an instantaneous security rate according to a current time slot upper layer service object selection result and a lower layer power distribution result, constructs a learning reward according to the instantaneous security rate, updates an upper layer discrete selection network and a lower layer continuous power control network according to the learning reward, and obtains a joint resource allocation result according to the updated upper layer discrete selection network and the lower layer continuous power control network. The application can quickly generate a joint resource allocation strategy meeting system constraints under a dynamic multi-user, radio frequency / laser fusion link and potential eavesdropping environment, improve system secure transmission performance, resource utilization efficiency and online scheduling real-time performance, and reduce online solving complexity of a traditional iterative optimization method in a complex dynamic scene.

Description

Technical Field

[0001] This invention relates to a secure air-space-ground integrated transmission method and system based on a laser-radio frequency fusion link, belonging to the field of wireless communication technology. Background Technology

[0002] With the rapid development of sixth-generation mobile communication systems and integrated air-space-ground networks, collaborative transmission between satellites, high-altitude platforms, and terrestrial communication networks has gradually become an important technological direction for improving wide-area coverage, link reliability, and communication security. In scenarios such as communication in remote areas, emergency communication, maritime communication, disaster relief, and military security communication, traditional terrestrial base stations are easily affected by factors such as deployment costs, terrain obstruction, infrastructure damage, and limited coverage, making it difficult to continuously provide stable and reliable communication services. High-altitude platform stations have advantages such as high deployment altitude, wide coverage, flexible deployment, and the ability to act as airborne relay nodes. They can establish relay links between satellites and ground users, thereby enhancing the service capabilities of integrated air-space-ground networks.

[0003] In high-altitude platform auxiliary communication scenarios, communication systems not only need to meet the requirements of wide-area coverage and high-speed transmission, but also need to ensure the secure transmission of sensitive business data in open wireless environments. While traditional radio frequency (RF) communication links offer advantages such as wide coverage, good link stability, and strong resistance to obstruction, their limited spectrum resources and the broadcast nature of wireless signals make them susceptible to co-channel interference and eavesdropping by unauthorized nodes, thus limiting the system's secure transmission capabilities. Free-space optical (FSO) communication offers advantages such as high bandwidth, high speed, strong directionality, and resistance to electromagnetic interference, which can improve link capacity and physical layer security to some extent. However, its transmission performance is easily affected by beam alignment, atmospheric turbulence, pointing errors, and obstruction conditions. In scenarios where user location and channel conditions change dynamically, link stability and continuous service capabilities still face challenges. Therefore, combining RF links with free-space optical links to form a hybrid optoelectronic transmission mechanism can leverage the complementary advantages of both types of links, providing high-altitude platform relay systems with higher capacity, stronger robustness, and better secure transmission capabilities. However, in real-world multi-user secure communication scenarios, systems must simultaneously address issues such as changing user access demands, dynamic fluctuations in link status, uncertainties in eavesdropping threats, and the coupling of multi-dimensional communication resources. Existing resource allocation methods typically optimize for a single link, a single user, or a single resource variable, making it difficult to simultaneously coordinate user service selection, optoelectronic link collaborative transmission, and transmit power allocation in dynamic environments. While traditional convex optimization, alternating optimization, and heuristic search methods can achieve certain performance improvements under specific conditions, they usually rely on accurate mathematical models and multiple rounds of iterative solutions, resulting in high computational complexity. When user location, link status, and eavesdropping environment change over time, traditional methods struggle to recalculate within a short period, failing to fully meet the real-time scheduling and online decision-making requirements of high-altitude platform relay systems. Especially in secure communication scenarios, resource allocation strategies need to respond promptly to eavesdropping risks and changes in link quality; significant decision-making delays directly impact the system's secure transmission capabilities.

[0004] Therefore, how to quickly generate an executable laser / RF fusion link secure transmission resource allocation strategy based on the current link status and changes in security threats in a complex and dynamic environment has become a key issue in improving the security performance and resource utilization efficiency of high-altitude platform relay communication systems.

[0005] In recent years, research has begun to focus on resource allocation in secure transmission scenarios for laser / RF fusion links. Related methods typically revolve around user scheduling, link selection, power allocation, physical layer security enhancement, and intelligent optimization decision-making. These methods attempt to utilize optimization algorithms or deep reinforcement learning to generate resource allocation strategies based on user states, link states, and security requirements in the communication environment. This reduces the online solution burden of traditional iterative optimization methods in dynamic scenarios and improves the adaptive capability of resource scheduling.

[0006] However, in high-altitude platform-assisted integrated air-ground secure transmission systems, laser links and radio frequency (RF) links have different transmission characteristics. The status of legitimate users, link quality, eavesdropping threats, and service demands can all change over time. Simultaneously, the system needs to jointly consider multiple resource variables, such as laser link target user selection, RF NOMA user pairing, and RF / FSO transmit power allocation. Therefore, existing methods still have room for improvement when directly applied to such complex and dynamic scenarios.

[0007] In summary, the existing technology still has the following shortcomings:

[0008] (1) Existing methods usually optimize for a single link or a single resource variable, focusing more on a certain part of user selection, link allocation or power control. It is difficult to coordinate RF NOMA user pairing, laser link target selection and transmit power allocation simultaneously in the laser / RF fusion link scenario. There is still room for improvement in system resource utilization efficiency and secure transmission performance.

[0009] (2) Existing methods do not adequately consider changes in link status, user service requirements and eavesdropping threats in dynamic multi-user scenarios. It is difficult to adjust resource allocation strategies in a timely manner according to the current communication environment. In complex time-varying channels and potential eavesdropping environments, the system security, robustness and adaptability are still limited.

[0010] (3) Existing methods still have shortcomings in dealing with the coupling problem of discrete service selection and continuous power control. Laser link target selection and radio frequency NOMA user pairing are discrete decisions, while transmit power allocation is a continuous control. The two affect each other and are subject to multiple communication constraints. Traditional methods are difficult to meet the time-slot-by-time real-time scheduling requirements of high-altitude platform relay systems while ensuring the feasibility of the strategy. Summary of the Invention

[0011] Objective: To overcome the problems of dynamic link state changes, multi-user service selection, uncertain eavesdropping threats, and heterogeneous resource coupling optimization in the collaborative transmission of laser and radio frequency links in integrated air-space-ground networks, this invention provides a secure transmission method and system for integrated air-space-ground networks based on laser-radio frequency fusion links. This method is designed for secure transmission scenarios assisted by high-altitude platforms. It incorporates RF user pair selection, FSO target user selection, and RF / FSO transmit power allocation into a unified joint optimization framework. The upper-layer discrete decision network determines the service object combination of the current time slot, and the lower-layer continuous control network completes the transmit power allocation. By combining secure transmission performance and resource constraints to construct learning rewards, it achieves time-slot-by-time adaptive generation of resource allocation strategies in complex dynamic environments, thereby improving the system's secure transmission capability, resource utilization efficiency, and real-time online scheduling.

[0012] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0013] Firstly, a secure air-space-ground integrated transmission method based on a laser-radio frequency fusion link specifically includes:

[0014] Step 1: Obtain the joint resource scheduling optimization model for the laser / RF fusion link.

[0015] Step 2: Construct the upper-layer discrete service selection state based on the laser / RF fusion link joint resource scheduling optimization model, input the upper-layer discrete service selection state into the upper-layer discrete selection network, and output the upper-layer service object selection result.

[0016] Step 3: Construct the lower-level continuous power control state based on the upper-level service object selection result, and input the lower-level continuous power control state into the lower-level continuous power control network to obtain the lower-level power allocation result.

[0017] Step 4: Calculate the instantaneous security rate based on the upper-layer service object selection result and the lower-layer power allocation result of the current time slot. Construct a learning reward based on the instantaneous security rate. Update the upper-layer discrete selection network and the lower-layer continuous power control network based on the learning reward. Obtain the joint resource allocation result based on the updated upper-layer discrete selection network and the lower-layer continuous power control network.

[0018] Optionally, the expression for the joint resource scheduling optimization model of the laser / RF fusion link is as follows:

[0019]

[0020] in, Let be the objective function. - As constraints, For time slot index, Let n be the total number of time slots within the scheduling period, and n be the index of valid users. The total number of legitimate users, Index for eavesdroppers As an auxiliary variable for the rate of secrecy, For FSO target switching penalty coefficient, Switch variables for FSO target. Select variables for RF-NOMA users. Select variables for the FSO objective. As a comprehensive service indicator variable, Startup variables for FSO services For RF-NOMA weak user transmit power, For RF-NOMA strong user transmit power, For FSO transmit power, For maximum total transmission power, This is the minimum rate threshold for SIC. For SIC decoding rate, For the legally reachable link rate, For legal rate auxiliary variables, eavesdropper The eavesdropping rate for user n, The equivalent eavesdropping rate of the most powerful eavesdropper. For the lowest legal rate, For the minimum security rate, For the minimum average service ratio, For continuous service slot index, For the shortest continuous service duration of FSO, Select variables for the FSO target of user n in the previous time slot.

[0021] Optionally, the step of inputting the upper-layer discrete service selection state into the upper-layer discrete selection network and outputting the upper-layer service object selection result specifically includes:

[0022] Input the upper-layer discrete service selection state information into the upper-layer discrete selection network, and output the action value vector corresponding to all candidate joint discrete actions in the current time slot. .

[0023] Construct an instant service object evaluation value for each candidate joint discrete action. .

[0024] Action value vector Evaluation value of immediate service recipients By fusing the results, a comprehensive action score is obtained for the candidate joint discrete actions. .

[0025] Based on the FSO remaining residency count in the current time slot Feasibility screening is performed on candidate joint discrete actions to obtain a set of upper-level feasible actions that satisfy the FSO shortest continuous service constraint. .

[0026] The upper-layer discrete selection network selects the joint discrete action with the highest comprehensive action score from the set of upper-layer actionable actions, and uses it as the upper-layer service object selection result for the current time slot.

[0027] Optionally, the step of inputting the lower-level continuous power control state into the lower-level continuous power control network to obtain the lower-level power allocation result specifically includes:

[0028] Lower-level continuous power control status information Input the lower-level continuous power control network and output the mean vector of the original power control action. and standard deviation vector .

[0029] Generate the original power control factor .

[0030] Constructing a safety-aware power control decision quantity based on the original power control factor. .

[0031] Safety-aware power control decision quantity After performing bounded processing, the normalized power control factor is obtained. , .

[0032] Based on the normalized power control factor The unbounded continuous action output by the lower-level continuous power control network is converted into RF-NOMA weak user transmit power, RF-NOMA strong user transmit power, and FSO transmit power, which can be used for power mapping, as the lower-level power allocation result.

[0033] Optionally, step 4 specifically includes:

[0034] For any legitimate user n, calculate the legitimate reachability rate of user n in the second hop based on the upper-layer service object selection result and the lower-layer power allocation result of the current time slot. .

[0035] Based on the legal reachability rate of user n in the second hop Calculate the end-to-end legal reachability rate for user n. .

[0036] For the eavesdropping link, the calculation is performed based on the upper-layer service object selection result and the lower-layer power allocation result of the current time slot. The total eavesdropping rate of user n information by a single eavesdropper .

[0037] The rate corresponding to the highest eavesdropping ability among all eavesdroppers is taken as the equivalent eavesdropping rate of user n. .

[0038] Calculate the instantaneous confidentiality rate for user n based on the end-to-end legal reach rate and the equivalent eavesdropping rate. .

[0039] Construct the current time slot learning reward based on the instantaneous confidentiality rate of each legitimate user. .

[0040] For the upper-layer discrete selection network, after completing the selection of the upper-layer service object and obtaining the learning reward in each time slot, the upper-layer discrete service selection state of the current time slot is recorded. Upper-level joint discrete actions Learning rewards and the upper-layer discrete service selection state for the next time slot. Stored in the upper-level experience replay pool.

[0041] During network training, a small batch of training samples is randomly drawn from the upper-layer experience replay pool. A value loss function is constructed based on the drawn training samples, comparing the current action value with the target action value. Furthermore, by minimizing the value loss function, the parameters of the upper-layer discrete selection network are updated, enabling the upper-layer discrete selection network to gradually learn the selection strategies for FSO target users and RF-NOMA service users under different communication states.

[0042] For the lower-level continuous power control network, after completing continuous power control and obtaining learning rewards in the current time slot, the lower-level continuous power control state of the current time slot will be... Lower-level continuous power control action Learning rewards and the lower-level continuous power control state of the next time slot. Stored in the lower-level experience replay pool.

[0043] During network training, a small batch of training samples is randomly drawn from the lower-layer experience replay pool. The action value corresponding to the continuous power control action is evaluated based on the training samples, and loss functions for two critic networks are constructed. After updating the commentator network parameters, the loss function of the lower-level policy network is further constructed. Furthermore, by minimizing the policy network loss function, the parameters of the lower-level policy network are updated, enabling the lower-level continuous power control network to progressively learn the continuous power allocation policy under the given conditions of FSO target users and RF-NOMA serving user pairs.

[0044] During training, the corresponding target network parameters are synchronized according to the preset update method, and the process of empirical sample extraction, loss function calculation and network parameter update is repeated until the preset number of training rounds is reached or the learning reward tends to stabilize, thus obtaining the trained upper-layer discrete selection network and lower-layer continuous power control network.

[0045] Based on the user location in the current time slot, the instantaneous channel state of RF / FSO, the instantaneous reachability rate from the satellite to the high-altitude platform, the FSO target selection result of the previous time slot, and the remaining FSO residency count, an upper-layer discrete service selection state is constructed. This upper-layer discrete service selection state is then input into the trained upper-layer discrete selection network to determine the FSO target user and RF-NOMA service user pair in the current time slot. Based on the upper-layer service object selection result and the corresponding legitimate link state, eavesdropping link state, and historical service state, a lower-layer continuous power control state is constructed. This state is then input into the trained lower-layer continuous power control network to determine the RF-NOMA weak user transmit power, the RF-NOMA strong user transmit power, and the FSO transmit power, thus forming the joint resource allocation result for the current time slot.

[0046] Beneficial Effects: The air-space-ground integrated secure transmission method and system based on laser-RF fusion links provided by this invention are designed for secure transmission scenarios using laser / RF fusion links assisted by high-altitude platforms. Employing hierarchical deep reinforcement learning technology, it integrates RF-NOMA user pair selection, FSO target user selection, and RF / FSO transmit power allocation into a unified joint resource allocation process. Specifically, this invention uses an upper-layer D3QN discrete selection network to determine the FSO target user and RF-NOMA service user pair for the current time slot based on the current communication environment and historical scheduling information, achieving intelligent decision-making for discrete service object selection. By constructing a set of actionable actions through FSO remaining residency counts, the selection of FSO target users satisfies the shortest continuous service constraint, thereby reducing the impact of frequent handovers on link stability. Through a lower-layer SAC continuous power control network, under the condition that the service objects are determined, it generates RF-NOMA weak user power, strong user power, and FSO transmit power, and ensures that the output results satisfy the total power budget and NOMA power hierarchical constraints through power mapping. Simultaneously, this invention constructs a learning reward based on the legal link rate, the equivalent eavesdropping rate of the strongest eavesdropper, and the instantaneous security rate, and utilizes an experience replay mechanism to update the upper-layer discrete selection network and the lower-layer continuous power control network respectively. This invention can rapidly generate joint resource allocation strategies that satisfy system constraints in dynamic multi-user, RF / laser fusion link, and potential eavesdropping environments, improving the system's secure transmission performance, resource utilization efficiency, and real-time online scheduling, while reducing the online solution complexity of traditional iterative optimization methods in complex dynamic scenarios. Compared to existing technologies, this invention has the following advantages:

[0047] 1. This invention adopts a hierarchical deep reinforcement learning structure, which decomposes laser link target user selection, RF-NOMA user pairing, and transmit power allocation into upper-layer discrete service decision-making and lower-layer continuous power control. It can effectively handle the joint optimization complexity problem caused by the coupling of discrete selection variables and continuous power variables, and improve the feasibility and online solution efficiency of resource scheduling strategies in complex dynamic communication scenarios.

[0048] 2. This invention introduces an action value evaluation mechanism and a laser link constraint action screening mechanism in the upper-level discrete decision-making process. Combining the current communication status, historical scheduling information, link quality and service constraints, it adaptively determines the FSO target user and RF-NOMA service user pair in the current time slot from the set of actionable actions, thereby improving the intelligence of service object selection, link switching stability and security scheduling robustness in dynamic multi-user scenarios.

[0049] 3. This invention introduces security-aware power control decision quantities and normalized power control factors into the lower-level continuous power control process. It comprehensively considers the quality of legitimate links, the state of eavesdropping links, user fair debt, and the confidentiality advantages between links to perform feasible mapping of RF / FSO transmit power and generate power allocation results that meet the total power budget. At the same time, it constructs learning rewards by combining confidentiality rate, eavesdropping threat, scheduling cost, and constraint violation, so that the system can balance security, resource utilization efficiency, and online scheduling real-time performance in complex dynamic environments, thereby improving the overall performance of high-altitude platform-assisted confidential transmission. Attached Figure Description

[0050] Figure 1 This is a model diagram of the secure transmission system of the integrated air-space-ground laser / RF fusion link of the present invention.

[0051] Figure 2 This is a comparison chart of the training convergence of the upper-layer user selection algorithm described in this invention with DQN, A2C, PPO, and random selection algorithms.

[0052] Figure 3 This is a comparison chart showing the impact of the number of eavesdroppers on the average security rate in this invention.

[0053] Figure 4 This is a comparison chart showing the impact of the number of users on the average security rate in this invention. Detailed Implementation

[0054] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0055] The present invention will be further described below with reference to specific embodiments.

[0056] Example 1:

[0057] This embodiment introduces a secure air-space-ground integrated transmission method based on a laser-radio frequency fusion link, specifically including the following steps:

[0058] Step 1: The integrated air-space-ground laser / RF fusion link secure transmission system includes a satellite transmitter, a high-altitude platform relay node, multiple legitimate ground users, and multiple passive eavesdroppers. The satellite transmitter first transmits information to the high-altitude platform via the first-hop link. The high-altitude platform, acting as a relay node, simultaneously utilizes both the RF-NOMA (Radio Frequency Non-Orthogonal Multiple Access) link and the FSO (Free Space Optics) link in the second hop to collaboratively transmit information to the legitimate ground users.

[0059] The RF link is used to select two legitimate users to form an RF-NOMA service user pair within the same time slot, and the FSO link is used to select one legitimate user as the FSO target user within the same time slot. Passive eavesdroppers can listen to the RF signal and eavesdrop on the FSO signal when the beam leakage or interception conditions are met.

[0060] Since there are multiple passive eavesdroppers in the system, this embodiment adopts a security modeling approach oriented towards the strongest eavesdropper. Specifically, for each legitimate user, the eavesdropping rate corresponding to the strongest eavesdropper among all eavesdroppers is taken as the user's equivalent eavesdropping rate. The instantaneous security rate of the user is calculated based on the difference between the achievable rate of the legitimate link and the equivalent eavesdropping rate. Furthermore, to balance the system's secure transmission performance and FSO link switching stability, this embodiment aims to maximize the long-term average security benefit for all legitimate users within the scheduling period. An FSO target switching penalty term is introduced into the objective function, and a joint resource scheduling optimization model for the laser / RF fusion link is established using multiple constraints. Its specific form is as follows:

[0061]

[0062] in, Let be the objective function. - As constraints, For time slot index, Let n be the total number of time slots within the scheduling period, and n be the index of valid users. The total number of legitimate users, Index for eavesdroppers The total number of eavesdroppers. As an auxiliary variable for the rate of secrecy, For FSO target switching penalty coefficient, Switch variables for FSO target. Select variables for RF-NOMA users. Select variables for the FSO objective. As a comprehensive service indicator variable, Startup variables for FSO services For RF-NOMA weak user transmit power, For RF-NOMA strong user transmit power, For FSO transmit power, For maximum total transmission power, This is the minimum rate threshold for SIC (Successive Interference Cancellation). For SIC decoding rate, For the legally reachable link rate, For legal rate auxiliary variables, eavesdropper The eavesdropping rate for user n, The equivalent eavesdropping rate of the most powerful eavesdropper. For the lowest legal rate, For the minimum security rate, For the minimum average service ratio, For continuous service slot index, The shortest continuous service duration for FSO. Select variables for the FSO target of user n in the previous time slot.

[0063] The RF-NOMA user choice variable in the above optimization model and FSO target selection variables Together, they constitute the discrete decision variables for service target selection, corresponding to the RF-NOMA serving user pair and the FSO target user in the subsequent upper-level discrete selection process; RF-NOMA weak user transmit power Strong user transmission power and FSO transmit power These are the continuous decision variables that constitute power allocation, corresponding to subsequent lower-level continuous power control processes. FSO target switching variables. FSO service startup variables and integrated service indicator variables The service selection results for the current time slot and historical time slots are used to determine the legitimate link rate, equivalent eavesdropping rate, and confidentiality rate, which are calculated jointly by the current communication state and the aforementioned resource allocation results. Based on this, this embodiment decomposes the above-mentioned joint resource scheduling optimization model into two interconnected decision-making processes: upper-layer service object selection and lower-layer continuous power allocation.

[0064] Step 2: Addressing the issue that in secure transmission scenarios using high-altitude platform-assisted laser / RF fusion links, FSO target user selection and RF-NOMA user pairing are influenced by link status, user location, and historical service conditions, making adaptive service object selection difficult to achieve through fixed rules, this invention proposes a state construction method based on an upper-layer D3QN discrete selection network. This method unifies the current time slot's communication environment information, first-hop link status, user fair debt, previous time slot's FSO target selection result, and FSO remaining residency count into upper-layer state information, which serves as the decision input for subsequent FSO target user selection and RF-NOMA user pairing. Therefore, the upper-layer discrete service selection state information is represented as:

[0065]

[0066] in, For time slots The upper-layer discrete service selection status information. For the location state of user n, For user n, the instantaneous RF channel state For user n, the instantaneous FSO channel state, The instantaneous achievable speed of the satellite to the high-altitude platform. For user n, the fair service debt Based on the target selection results of the previous time slot FSO, This is the remaining residency count information corresponding to the minimum service duration constraint for FSO.

[0067] Step 3: This embodiment uses an upper-layer D3QN discrete selection network built based on the existing D3QN algorithm to evaluate the action value of candidate joint service object combinations. Addressing the problem that traditional discrete action selection relies solely on network output action value, making it difficult to simultaneously reflect current time slot security benefits, user service status, and FSO link switching constraints, this embodiment introduces an immediate evaluation mechanism for candidate joint actions and a feasible action screening mechanism based on the remaining FSO residency status during the upper-layer discrete service selection process. For each joint discrete action consisting of a candidate FSO target user and a candidate RF-NOMA service user pair, based on the long-term action value output by the network, an immediate service object evaluation value is constructed by combining the estimated security benefits, user service fairness status, FSO target switching cost, and communication constraint violations corresponding to the candidate action. This immediate service object evaluation value is then fused with the long-term action value to obtain a comprehensive action score for the candidate joint action. Simultaneously, to ensure that the FSO target selection result output by the network meets the shortest continuous service requirement, feasibility screening of candidate actions is performed based on the current remaining FSO residency count before executing the joint discrete action selection. This allows the discrete action selection process to directly satisfy the FSO target switching constraint, reducing the situation where inactive actions are repeatedly selected by the network.

[0068] Specifically, the candidate joint discrete action is defined as:

[0069]

[0070] in, For the first One candidate joint discrete action, For the candidate FSO target users corresponding to this action, This represents the candidate RF-NOMA user pair corresponding to the action. Therefore, each candidate action in the upper-layer D3QN discrete selection network represents a user scheduling scheme.

[0071] The upper-layer discrete service selection state information obtained in step 2 is input into the upper-layer D3QN (Dueling Double DeepQ-Network) reinforcement learning network, which outputs the action value vector corresponding to all candidate joint discrete actions in the current time slot. Its specific form is as follows:

[0072]

[0073] in, This represents the action value vector corresponding to all candidate joint discrete actions in the current time slot. In the state Select action The value of the action, This refers to the upper-layer state information of the current time slot. These are the parameters for the upper-layer D3QN network. For the first One candidate joint discrete action, This is the upper-level joint discrete action space.

[0074] Meanwhile, to ensure that upper-level decisions can explicitly reflect the confidentiality benefits, fairness, and laser link switching costs of the current time slot, this example constructs an instant service object evaluation value for each candidate joint discrete action, in the following specific form:

[0075]

[0076] in, For candidate joint discrete actions In the time slot Service recipient evaluation score, For the estimated confidentiality rate of user n, For user n in time slot Serving fair debt, For fair debt weighting coefficient, For candidate joint discrete actions The comprehensive service indicator variable for user n. The target switching penalty coefficient for the laser link. For candidate joint discrete actions The corresponding FSO target switching indicator variable, To constrain the penalty coefficient for violations, For candidate joint discrete actions The corresponding constraint violation penalty item.

[0077] By fusing the long-term action value output by the upper-layer D3QN discrete selection network with the immediate service object evaluation value, a comprehensive action score for candidate joint discrete actions is obtained:

[0078]

[0079] in, The overall score for candidate joint discrete actions. The weighting of the evaluation values ​​of the real-time service recipients.

[0080] Since FSO links have directional and alignment stability requirements, to avoid frequent handovers by FSO target users, this embodiment uses the remaining FSO residency count in the current time slot. The feasibility of candidate joint discrete actions is screened to obtain a set of upper-level feasible actions that satisfy the FSO shortest continuous service constraint, which takes the following form:

[0081]

[0082] in, This is the set of upper-level possible actions for the current time slot. For the target user of the previous time slot FSO. When When this occurs, it indicates that the current FSO target user has not yet met the minimum continuous service requirement. In this case, switching the FSO target user is not allowed; only the joint action of the candidate FSO target user consistent with the previous time slot is retained. When this condition is met, it indicates that the current FSO target user has met the shortest continuous service requirement. At this point, it is permissible to reselect the FSO target user and RF-NOMA user pair in the complete upper-layer joint discrete action space.

[0083] After obtaining the action value vector and the set of possible actions, the upper-layer D3QN network selects only the joint discrete action with the highest action value from the set of possible actions as the upper-layer service object selection result for the current time slot, that is:

[0084]

[0085] The optimal upper-level joint discrete action is defined as:

[0086]

[0087] in, For the FSO target user selected in the current time slot, The RF-NOMA serving user pair is selected for the current time slot. Therefore, the upper-layer D3QN network completes the selection of serving users for the current time slot and uses the result as the input condition for subsequent lower-layer continuous power control.

[0088] The upper-layer discrete service selection process and the lower-layer continuous power control process are executed sequentially within the same time slot. For each time slot... First, an upper-layer discrete service selection state is constructed based on the current communication environment. The upper-layer discrete selection network determines the FSO target user and RF-NOMA service user pair for the current time slot. Subsequently, the service object selection result, together with the corresponding user's legitimate link state, eavesdropping link state, and historical service state, constitutes the lower-layer continuous power control state. The lower-layer continuous power control network determines the RF-NOMA weak user transmit power, RF-NOMA strong user transmit power, and FSO transmit power for the current time slot. The upper-layer service object selection result and the lower-layer power allocation result together constitute the joint resource allocation action for the current time slot, which is applied to the communication environment to calculate the legitimate transmission rate, equivalent eavesdropping rate, and instantaneous security rate for the current time slot.

[0089] Step 4: Given that the target users are known, the lower-layer continuous power control network continuously controls the RF-NOMA weak user power, RF-NOMA strong user power, and FSO transmit power in the current time slot. Specifically, the lower-layer network constructs lower-layer continuous power control state information around weak users, strong users, and FSO target users, i.e.:

[0090]

[0091] in, For time slots The lower-level continuous power control status information. and For the two users in the current RF-NOMA service user pair, and users respectively and users RF link status, The link status of the current FSO target user. The instantaneous achievable speed of the satellite to the high-altitude platform. and eavesdroppers For users and users RF eavesdropping link status of information eavesdropper The FSO eavesdropping link status for the current FSO target user information. The total number of eavesdroppers. , and users respectively ,user and current FSO target users In the time slot The fair value of debt.

[0092] Step 5: After obtaining the lower-layer continuous power control state information, this embodiment uses a lower-layer continuous power control network constructed based on the existing SAC (SoftActor-Critic) algorithm to control the RF / FSO transmit power. The lower-layer SAC continuous power control network includes a random policy network, two Q-critic networks, and target critic networks corresponding to the two Q-critic networks. The random policy network generates continuous power control actions based on the current lower-layer state, and the critic networks evaluate the value of the actions corresponding to the state and the continuous actions. Addressing the issue that the continuous actions directly output by the continuous power control network cannot simultaneously reflect legitimate link quality, eavesdropping threats, and the security advantages of different links, and cannot directly satisfy the total power budget and RF-NOMA power hierarchical constraints, this embodiment introduces a security-aware power adjustment and feasible mapping mechanism during the lower-layer continuous power control process. Based on the continuous control results obtained from the lower-level continuous power control network, and combined with the legitimate link status, eavesdropping risk, service fairness status, and relative confidentiality advantages of RF links and FSO links of the current served users, the total power enable decision quantity, the power hierarchical decision quantity among RF-NOMA users, and the power allocation decision quantity among RF / FSO links are constructed respectively.

[0093] Specifically, the lower-level state information obtained in step 4 Input the lower-level SAC policy network, and the network outputs the mean vector of the original power control action. and standard deviation vector During training, to maintain the exploration capability of the SAC stochastic policy, the original power control factor is generated through reparameterization, i.e.:

[0094]

[0095] in, The original power control factor. Used to control the proportion of total power used. Used to control the power allocation ratio between weak and strong users in RF-NOMA. Used to control the power distribution ratio between the RF link and the FSO link. This is the mean vector output by the lower-level SAC network. This is the standard deviation vector output by the lower-level SAC network. These are the parameters for the lower-level policy network. For element-wise multiplication, It is a standard normal random noise vector. It follows a Gaussian distribution.

[0096] Construct a safety-aware power control decision quantity based on the original power control factor:

[0097]

[0098] in, Enable decision variables for total power decisions. For RF-NOMA user power stratification decision-making, This is the power allocation decision quantity between the FSO link and the RF link.

[0099] The total power activation decision quantity is represented as:

[0100]

[0101] in, The normalized fair average debt of the current users being served. This serves as a normalized eavesdropping risk indicator for the current users being served. For the normalized legitimate link quality indicators of the currently served users, , and Enable control weights for total power.

[0102] The power stratification decision quantity among RF-NOMA users is expressed as:

[0103]

[0104] in, and These are the normalized link states for RF-NOMA weak users and RF-NOMA strong users, respectively. and The normalized fair debts for weak and strong users of RF-NOMA are respectively. As a preset positive stable term, , Weights for power stratification control among RF-NOMA users.

[0105] The power allocation decision between the laser link and the RF link is expressed as:

[0106]

[0107] in, For the normalized fair debt of the current FSO link target users, , Assign control weights to power distribution between links. As an indicator of the current security advantages of FSO links, This represents the average security advantage metric for current RF links.

[0108] The security advantages of FSO links are expressed as follows:

[0109]

[0110] The average security advantage metric for RF links is expressed as:

[0111]

[0112] in, The normalized FSO link state for the current FSO target user. eavesdropper For the current FSO target user's normalized FSO eavesdropping link status, This indicates whether a user is a weak or strong user in the current RF-NOMA service user pair. For the normalized RF link state of user n, eavesdropper Normalized RF eavesdropping link status for user n information.

[0113] Since the security-aware power control decision quantity is an unbounded continuous quantity and cannot be directly used as the actual transmit power, this embodiment further performs bounded processing, converting it into a normalized power control factor, namely:

[0114]

[0115] in, It is the hyperbolic tangent function. Let be the normalized power control factor, and satisfy . .

[0116] Finally, through the above bounded transformation process, the unbounded continuous action output by the lower-level SAC network is converted into a bounded control quantity that can be used for power mapping, the specific form of which is as follows:

[0117]

[0118] in, This represents the total power used in the current time slot. The maximum total transmit power allowed by the system. For FSO link transmit power, This represents the total transmit power of the RF-NOMA link. For RF-NOMA weak user transmit power, For RF-NOMA strong user transmit power. Specifically, Used to control the proportion of total power used in the current time slot relative to the maximum power. Used to control the distribution ratio of total power usage between FSO links and RF links. Used to control the power allocation ratio between weak and strong users in RF-NOMA.

[0119] Step 6: For any legitimate user n, based on the scheduling result and power allocation result of the current time slot, calculate the legitimate reachability rate of user n in the second hop, i.e.:

[0120]

[0121] in, For user n in time slot The second hop legal reachability rate, For the achievable rate on the RF link, This represents the achievable rate on the FSO link.

[0122] Since this embodiment uses a relay forwarding transmission method, the legally reachable end-to-end rate of user n is limited by both the first-hop link from the satellite to the high-altitude platform and the second-hop link from the high-altitude platform to the user. Therefore, the legally reachable end-to-end rate of user n is expressed as follows: For the eavesdropping link, calculate the first... The total eavesdropping rate of n users' information by a single eavesdropper is:

[0123]

[0124] in, eavesdropper The total rate of eavesdropping on user information eavesdropper The rate at which user n's information is eavesdropped via RF link. The rate at which an eavesdropper can eavesdrop on user n's information via the FSO link. For RF-NOMA user selection variables, when user n is selected into the RF-NOMA serving user pair in the current time slot... , Select variables for the FSO target when user n is selected as the FSO target user for the current time slot. .

[0125] To obtain a more conservative security performance assessment, this embodiment adopts the strongest eavesdropper model, taking the rate corresponding to the eavesdropper with the strongest eavesdropping ability among all eavesdroppers as the equivalent eavesdropping rate of user n, that is:

[0126]

[0127] in, For user n in time slot The equivalent eavesdropping rate, A gathering of eavesdroppers.

[0128] Furthermore, based on the end-to-end legal reach rate and the equivalent eavesdropping rate, the instantaneous security rate of user n is calculated, in the following specific form:

[0129]

[0130] in, For user n in time slot Instantaneous security rate, .

[0131] After obtaining the instantaneous security rate of each legitimate user, this embodiment combines the FSO target switching cost and constraint violation penalty to construct the current time slot learning reward, which is specifically in the following form:

[0132]

[0133] in, For time slots The learning rewards The total instantaneous confidentiality benefits for all legitimate users in the current time slot. For FSO target switching penalty coefficient, Switch variables for FSO target. The penalty coefficient is... The constraint violation penalty term is used to characterize the violation of conditions such as SIC feasibility constraints, minimum legal rate constraints, and minimum security rate constraints. Through the above-described learning reward construction method, the hierarchical deep reinforcement learning network can not only optimize for improving the instantaneous security rate of the system, but also suppress unnecessary frequent switching of FSO objectives and reduce the probability of resource allocation actions that do not meet communication constraints being continuously selected.

[0134] Step 7: After completing the service object selection, power allocation, and learning reward calculation in each time slot, store the upper-layer state, upper-layer joint discrete action, learning reward, and the upper-layer state of the next time slot in the upper-layer experience replay pool for the current time slot. Simultaneously, store the lower-layer state, lower-layer continuous power control action, learning reward, and the lower-layer state of the next time slot in the lower-layer experience replay pool. During network training, randomly select small batches of training samples from both the upper-layer and lower-layer experience replay pools to update network parameters, thereby reducing the correlation between samples in consecutive time slots and improving training stability.

[0135] For the upper-layer D3QN discrete selection network, a value loss function is constructed based on the extracted training samples, which is of the form:

[0136]

[0137] in, The value loss function for the upper-layer discrete selection network is determined by the upper-layer state in the next time slot, the set of possible actions that satisfy the FSO shortest continuous service constraint, and the target network. This is achieved by minimizing... Update upper-layer D3QN online network parameters This enables the upper-layer network to gradually learn discrete scheduling strategies for selecting FSO target users and RF-NOMA user pairs under different upper-layer states. This represents the expectation of a small batch of training samples randomly drawn from the upper-level experience replay pool. For time slots The corresponding upper-level target action value, For the upper-level discrete selection network in the state The lower layer executes the discrete actions of the upper layer. The value of the action.

[0138] For the lower-layer SAC continuous power control network, the action value corresponding to the continuous power control action is evaluated based on the training samples extracted from the lower-layer empirical replay pool, and the loss functions of the two critic networks are constructed, with the following specific forms:

[0139]

[0140] in, For the first The loss function of a network of critics. For the first A network of critics, Its network parameters. By minimizing and Update the network parameters for the two commentators respectively. The original power control factor. The target action value for the lower-level continuous power control network.

[0141] After updating the commentator network parameters, the loss function of the lower-level SAC policy network is further updated, and its specific form is as follows:

[0142]

[0143] in, The temperature coefficient is used to adjust the weights of the policy entropy term. For parameters The lower-level random policy network in state The probability distribution for generating continuous power control actions. These are the parameters for the lower-level policy network.

[0144] By minimizing the loss function of the policy network and updating the parameters of the lower-level policy network, the lower-level continuous power control network can progressively learn the continuous allocation strategy of RF-NOMA weak user transmit power, RF-NOMA strong user transmit power, and FSO transmit power under the condition of a given FSO target user and RF-NOMA serving user pair.

[0145] During training, the corresponding target network parameters are synchronized according to a preset update method, and the processes of empirical sample extraction, loss function calculation, and network parameter update are repeatedly executed until a preset number of training rounds are reached or the learning reward tends to stabilize, resulting in the trained upper-layer discrete selection network and lower-layer continuous power control network. In the online execution phase, based on the communication status of the current time slot, the upper-layer discrete selection network is first used to determine the FSO target user and RF-NOMA serving user pair. Then, the lower-layer continuous power control network is used to determine the corresponding RF-NOMA weak user transmit power, RF-NOMA strong user transmit power, and FSO transmit power, thereby forming the joint resource allocation result for the current time slot.

[0146] Example 2:

[0147] This embodiment describes a computer-readable storage medium storing a computer program that, when executed by a processor, implements a secure air-space-ground integrated transmission method based on a laser / RF fusion link as described in any of Embodiment 1.

[0148] Example 3:

[0149] This embodiment describes a computer device, including:

[0150] Memory is used to store instructions.

[0151] A processor is configured to execute the instructions, causing the computer device to perform a secure air-space-ground integrated transmission method based on a laser / RF fusion link as described in any of Embodiment 1.

[0152] Example 4:

[0153] This embodiment introduces a simulation experiment of an integrated air-space-ground secure transmission method based on a laser / RF fusion link. The simulation scenario is set as an integrated air-space-ground dual-hop secure downlink system consisting of a satellite transmitter, a fixed HAPS relay node, multiple legitimate mobile users on the ground, and multiple fixed passive eavesdroppers. The number of legitimate users is set to 6, the number of eavesdroppers to 3, the total system transmit power budget to 5W, the RF link noise power to 0.05, the FSO link noise power to 0.03, the minimum legitimate rate threshold to 0.35, and the minimum secure rate threshold to 0.12. The HAPS altitude is set to 20km, the satellite altitude to 520km, the ground service area radius to 4.5km, the user movement disturbance velocity parameter to 0.05km, each round to 100 time slots, and the minimum FSO dwell time to 5 time slots.

[0154] To address the characteristic of the joint resource scheduling model in this invention, which simultaneously includes discrete service object selection variables and continuous power allocation variables, a hierarchical adaptation is performed on the reinforcement learning agent. The selection of FSO target users and RF-NOMA user pairing are mapped to upper-layer discrete actions, decided by a D3QN (Dueling Double Deep Q-Network) agent. The RF-NOMA weak user power, strong user power, and FSO transmit power are mapped to lower-layer continuous actions, controlled by a SAC (Soft Actor-Critic) agent. During the online execution phase, the upper-layer discrete selection network and the lower-layer continuous power control network operate collaboratively in a time-slot-level sequence. Within each time slot, the high-altitude platform first determines the FSO target user and RF-NOMA service user pair based on the current communication status through the upper-layer discrete selection network. Subsequently, the lower-layer continuous power control state is constructed based on the service object selection results, and the lower-layer continuous power control network determines the RF-NOMA weak user transmit power, RF-NOMA strong user transmit power, and FSO transmit power. The service object selection result and the power allocation result together constitute the joint resource allocation result of the current time slot, and the data transmission and system status update of the current time slot are completed accordingly.

[0155] The reinforcement learning network parameters are set as follows: Both the upper-layer D3QN online network and the target network adopt a two-hidden-layer fully connected Dueling structure. Each hidden layer contains 128 neurons and uses the ReLU activation function. The output is divided into a one-dimensional state value branch and an action advantage branch corresponding to the number of candidate joint actions. D3QN uses the Adam optimizer with a learning rate of [missing information]. The discount factor is 0.97, and the target network is synchronized every 200 parameter updates. The lower-level SAC's policy network and two commentator networks each use two ReLU fully connected hidden layers with 128 neurons each. The policy network outputs the distribution parameters of the 3D continuous power control actions, and the two commentator networks output the corresponding action values. All SAC networks use the Adam optimizer with a learning rate of [missing information - likely a specific value]. The discount factor is 0.97, and the soft update coefficient of the target commentator network is 0.01. The experience replay pool capacity for both networks is... The number of training samples in each mini-batch is 128.

[0156] like Figure 2 The figure shows a comparison of the convergence performance of the upper-layer user selection algorithm during training, with the shaded area representing the fluctuation range under different random seeds. As can be seen from the figure, the average round reward of the random selection strategy remains at a low level in the long term, indicating that in complex scenarios with multiple users, multiple eavesdroppers, and mixed RF / FSO links, random scheduling struggles to achieve effective secure transmission performance. A2C (Advantage Actor-Critic) and PPO (Proximal Policy Optimization) achieve some improvement in the early stages of training, but their convergence level and stability are limited in the later stages. In contrast, DQN (Deep Q-Network) and D3QN can gradually improve the upper-layer service object selection capability through value learning. Among them, D3QN has the highest overall reward during training, stabilizing at approximately 1.8 or higher after convergence, significantly outperforming DQN, A2C, PPO, and the random strategy. This result demonstrates that the D3QN upper-layer selection network used in this invention can more accurately evaluate the long-term benefits of different FSO target users and RF-NOMA user pair combinations, thereby improving the convergence performance and stability of system resource scheduling.

[0157] like Figure 3The figure shows the impact of the number of eavesdroppers on the average security rate of different methods. As the number of eavesdroppers increases from 3 to 6, the average security rate of all methods decreases. This is because with an increased number of eavesdroppers, the most dangerous eavesdropping links faced by the system become stronger, increasing the equivalent eavesdropping rate for legitimate users and thus reducing the achievable secure transmission rate. The figure shows that the Random+Equal and Single-Agent PPO methods exhibit the most significant decreases, indicating that fixed power allocation or single-agent joint control is insufficient to adapt to the complex environment with increased eavesdropping risk. Although D3QN+Equal possesses good user selection capabilities, its performance is still significantly lower than the method using SAC power control due to the lack of adaptive power allocation. D3QN+SAC achieves the highest average security rate under all eavesdropper number settings and maintains a significant advantage even as the number of eavesdroppers increases. This demonstrates that the present invention, through upper-layer D3QN security-aware service object selection and combined with lower-layer SAC continuous adaptive power allocation, can effectively suppress the degradation of security performance caused by an increase in the number of eavesdroppers.

[0158] like Figure 4 The figure shows the impact of the number of legitimate users on the average security rate of different methods. As the number of legitimate users increases from 4 to 10, the average security rate of each method decreases overall. This is because, under the condition of limited total power budget and link resources, an increase in the number of users intensifies scheduling competition, and the combination of channel differences and eavesdropping risks among different users becomes more complex, thus increasing the difficulty of resource allocation. As shown in the figure, Random+Equal and Single-Agent PPO show a significant performance decline when the number of users increases, making it difficult to stably adapt to changes in user scale. D3QN+Equal can achieve some performance improvement by relying on the upper-layer selection strategy, but due to the fixed power allocation, it cannot fully utilize the resource differences of different users and link states. DQN+SAC outperforms the fixed power scheme at all user scales, and D3QN+SAC consistently achieves the highest average security rate, indicating that the hierarchical deep reinforcement learning method proposed in this invention can maintain strong generalization ability and robustness when the number of users changes, and achieve better user selection and power allocation coordination in multi-user secure transmission scenarios.

[0159] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0160] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0161] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0162] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0163] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A secure air-space-ground integrated transmission method based on a laser-radio frequency fusion link, characterized in that: Specifically, it includes: Step 1: Obtain the joint resource scheduling optimization model for the laser / RF fusion link; Step 2: Construct the upper-layer discrete service selection state based on the laser / RF fusion link joint resource scheduling optimization model, input the upper-layer discrete service selection state into the upper-layer discrete selection network, and output the upper-layer service object selection result; Step 3: Construct the lower-level continuous power control state based on the upper-level service object selection result, and input the lower-level continuous power control state into the lower-level continuous power control network to obtain the lower-level power allocation result; Step 4: Calculate the instantaneous security rate based on the upper-layer service object selection result and the lower-layer power allocation result of the current time slot. Construct a learning reward based on the instantaneous security rate. Update the upper-layer discrete selection network and the lower-layer continuous power control network based on the learning reward. Obtain the joint resource allocation result based on the updated upper-layer discrete selection network and the lower-layer continuous power control network.

2. The air-space-ground integrated secure transmission method based on laser-RF fusion link according to claim 1, characterized in that: The expression for the joint resource scheduling optimization model of the laser / RF fusion link is as follows: ； in, Let be the objective function. - As constraints, For time slot index, Let n be the total number of time slots within the scheduling period, and n be the index of valid users. The total number of legitimate users, Index for eavesdroppers As an auxiliary variable for the rate of secrecy, For FSO target switching penalty coefficient, Switch variables for FSO target. Select variables for RF-NOMA users. Select variables for the FSO objective. As a comprehensive service indicator variable, Startup variables for FSO services For RF-NOMA weak user transmit power, For RF-NOMA strong user transmit power, For FSO transmit power, For maximum total transmission power, This is the minimum rate threshold for SIC. For SIC decoding rate, For the legally reachable link rate, For legal rate auxiliary variables, eavesdropper The eavesdropping rate for user n, The equivalent eavesdropping rate of the most powerful eavesdropper. For the lowest legal rate, For the minimum security rate, For the minimum average service ratio, For continuous service slot index, For the shortest continuous service duration of FSO, Select variables for the FSO target of user n in the previous time slot.

3. The air-space-ground integrated secure transmission method based on laser-RF fusion link according to claim 1, characterized in that: The step of inputting the upper-layer discrete service selection state into the upper-layer discrete selection network and outputting the upper-layer service object selection result specifically includes: Input the upper-layer discrete service selection state information into the upper-layer discrete selection network, and output the action value vector corresponding to all candidate joint discrete actions in the current time slot. ; Construct an instant service object evaluation value for each candidate joint discrete action. ; Action value vector Evaluation value of immediate service recipients By fusing the results, a comprehensive action score is obtained for the candidate joint discrete actions. ; Based on the FSO remaining residency count in the current time slot Feasibility screening is performed on candidate joint discrete actions to obtain a set of upper-level feasible actions that satisfy the FSO shortest continuous service constraint. ; The upper-layer discrete selection network selects the joint discrete action FSO target user and RF-NOMA service user pair with the highest comprehensive action score from the set of upper-layer actionable actions, and uses this as the upper-layer service object selection result for the current time slot.

4. The air-space-ground integrated secure transmission method based on laser-RF fusion link according to claim 1, characterized in that: The step of inputting the lower-level continuous power control state into the lower-level continuous power control network to obtain the lower-level power allocation result specifically includes: Lower-level continuous power control status information Input the lower-level continuous power control network and output the mean vector of the original power control action. and standard deviation vector ; Generate the original power control factor ; Constructing a safety-aware power control decision quantity based on the original power control factor. ; Safety-aware power control decision quantity After performing bounded processing, the normalized power control factor is obtained. , ; Based on the normalized power control factor The unbounded continuous action output by the lower-level continuous power control network is converted into RF-NOMA weak user transmit power, RF-NOMA strong user transmit power, and FSO transmit power, which can be used for power mapping, as the lower-level power allocation result.

5. The air-space-ground integrated secure transmission method based on laser-RF fusion link according to claim 1, characterized in that: Step 4 specifically includes: For any legitimate user n, calculate the legitimate reachability rate of user n in the second hop based on the upper-layer service object selection result and the lower-layer power allocation result of the current time slot. ; Based on the legal reachability rate of user n in the second hop Calculate the end-to-end legal reachability rate for user n. ; For the eavesdropping link, the calculation is performed based on the upper-layer service object selection result and the lower-layer power allocation result of the current time slot. The total eavesdropping rate of user n information by a single eavesdropper ; The rate corresponding to the highest eavesdropping ability among all eavesdroppers is taken as the equivalent eavesdropping rate of user n. ; Calculate the instantaneous confidentiality rate for user n based on the end-to-end legal reach rate and the equivalent eavesdropping rate. ; Construct the current time slot learning reward based on the instantaneous confidentiality rate of each legitimate user. ; For the upper-layer discrete selection network, after completing the selection of the upper-layer service object and obtaining the learning reward in each time slot, the upper-layer discrete service selection state of the current time slot is recorded. Upper-level joint discrete actions Learning rewards and the upper-layer discrete service selection state for the next time slot. Stored in the upper-level experience replay pool; During network training, a small batch of training samples is randomly drawn from the upper-layer experience replay pool; a value loss function is constructed based on the drawn training samples to compare the current action value with the target action value. And by minimizing the value loss function Update the parameters of the upper-level discrete selection network; For the lower-level continuous power control network, after completing the lower-level continuous power control and obtaining the learning reward in the current time slot, the lower-level continuous power control state of the current time slot will be... Lower-level continuous power control action Learning rewards and the lower-level continuous power control state of the next time slot. Store in the lower-level experience replay pool; During network training, a small batch of training samples is randomly drawn from the lower-level experience replay pool. The action value corresponding to the continuous power control action is evaluated based on the training samples, and the loss function of the critic network is constructed. By minimizing the loss function Update the critic network; after updating the critic network parameters, further construct the loss function for the lower-level policy network. And by minimizing the network loss function of the policy Update the parameters of the lower-level policy network; Repeat the process of extracting empirical samples, calculating the loss function, and updating network parameters until the preset number of training rounds is reached or the learning reward tends to stabilize, and the trained upper-layer discrete selection network and lower-layer continuous power control network are obtained. Based on the user location in the current time slot, the instantaneous channel state of RF / FSO, the instantaneous reachability rate from the satellite to the high-altitude platform, the FSO target selection result of the previous time slot, and the remaining FSO residency count, an upper-layer discrete service selection state is constructed. This upper-layer discrete service selection state is then input into the trained upper-layer discrete selection network to determine the FSO target user and RF-NOMA service user pair in the current time slot. Based on the upper-layer service object selection result and the corresponding legitimate link state, eavesdropping link state, and historical service state, a lower-layer continuous power control state is constructed and input into the trained lower-layer continuous power control network to determine the RF-NOMA weak user transmit power, RF-NOMA strong user transmit power, and FSO transmit power, thus forming the joint resource allocation result for the current time slot.

6. The air-space-ground integrated secure transmission method based on laser-RF fusion link according to claim 3, characterized in that: The action value vector The expression is as follows: ； in, In the state Select action The value of the action, This provides the upper-layer state information for the current time slot. Select network parameters for the upper discrete layers. For the first One candidate joint discrete action, For the upper-level joint discrete action space; The service recipient evaluation value The expression is as follows: ； in, For the estimated confidentiality rate of user n, For user n in time slot Serving fair debt, For fair debt weighting coefficient, For candidate joint discrete actions The comprehensive service indicator variable for user n. The target switching penalty coefficient for the laser link. For candidate joint discrete actions The corresponding FSO target switching indicator variable, To constrain the penalty coefficient for violations, For candidate joint discrete actions The corresponding constraint violation penalty item; The comprehensive action score The expression is as follows: ； in, The fusion weight for the evaluation value of the real-time service recipients; The set of upper-level actions The expression is as follows: ； in, For the target user of the previous time slot FSO, Indicates the target users of FSO. Indicates RF-NOMA service user pair; The result of the selection of the upper-level service object The expression is as follows: ； in, For the FSO target user selected in the current time slot, Select the RF-NOMA service user pair for the current time slot.

7. The air-space-ground integrated secure transmission method based on laser-RF fusion link according to claim 1, characterized in that: The expression for the upper-layer discrete service selection state is as follows: ； in, For time slots The upper-layer discrete service selection status information. For the location state of user n, For user n, the instantaneous RF channel state For user n, the instantaneous channel state of FSO. The instantaneous achievable speed of the satellite to the high-altitude platform. For user n, the fair service debt Based on the target selection results of the previous time slot FSO, This is the remaining residency count information corresponding to the minimum service duration constraint for FSO.

8. The air-space-ground integrated secure transmission method based on laser-RF fusion link according to claim 4, characterized in that: The original power control factor The expression is as follows: ； in, The original power control factor. Used to control the proportion of total power used. Used to control the power allocation ratio between weak and strong users in RF-NOMA. Used to control the power distribution ratio between the RF link and the FSO link. These are the parameters for the lower-level continuous power control network. For element-wise multiplication, It is a standard normal random noise vector; The security-aware power control decision quantity The expression is as follows: ； in, Enable decision variables for total power decisions. For RF-NOMA user power stratification decision-making, For power allocation decision-making between FSO link and RF link; The normalized power control factor The expression is as follows: ； in, It is the hyperbolic tangent function; The expression for the lower-level power allocation result is as follows: ； in, This represents the total power used in the current time slot. The maximum total transmit power allowed by the system. For FSO link transmit power, This represents the total transmit power of the RF-NOMA link. For RF-NOMA weak user transmit power, For RF-NOMA strong user transmit power, Used to control the proportion of total power used in the current time slot relative to the maximum power. Used to control the distribution ratio of total power usage between FSO links and RF links. Used to control the power allocation ratio between weak and strong users in RF-NOMA.

9. The air-space-ground integrated secure transmission method based on laser-RF fusion link according to claim 1, characterized in that: The lower-level continuous power control state The expression is as follows: ； in, For time slots The lower-level continuous power control status information. and For the two users in the current RF-NOMA service user pair, and users respectively and users RF link status, The link status of the current FSO target user. The instantaneous achievable speed of the satellite to the high-altitude platform. and eavesdroppers For users and users RF eavesdropping link status of information eavesdropper The FSO eavesdropping link status for the current FSO target user information. The total number of eavesdroppers. , and users respectively ,user and current FSO target users In the time slot The fair value of debt.

10. The air-space-ground integrated secure transmission method based on laser-RF fusion link according to claim 5, characterized in that: The legal reachability rate of user n in the second hop The expression is as follows: ； in, For the achievable rate on the RF link, For the achievable rate on the FSO link, For RF-NOMA user selection variables, when user n is selected into the RF-NOMA serving user pair in the current time slot... , Select variables for the FSO target when user n is selected as the FSO target user for the current time slot. ; The end-to-end legal reachability rate of user n The expression is as follows: ； in, The instantaneous achievable speed of the satellite to the high-altitude platform; The first The total eavesdropping rate of user n information by a single eavesdropper The expression is as follows ； in, eavesdropper The rate at which user n's information is eavesdropped via RF link. The rate at which an eavesdropper can eavesdrop on user n's information via the FSO link; The equivalent eavesdropping rate of user n The expression is as follows: ； The instantaneous confidentiality rate of user n The expression is as follows: ； in, ; The time slot learning reward The expression is as follows: ； in, For time slots The learning rewards The total instantaneous confidentiality benefits for all legitimate users in the current time slot. For FSO target switching penalty coefficient, Switch variables for FSO target. The penalty coefficient is... To constrain violations and penalties.

Images