Method for selecting node working mode of cellular-free NAFD system in low-altitude three-dimensional coverage scene

Abstract

The invention discloses a method for selecting a node working mode of a cellular-free NAFD system in a low-altitude three-dimensional coverage scene, and belongs to the technical field of network communication. The method comprises the following steps: modeling a non-cellular communication system in a low-altitude three-dimensional coverage scene; analyzing a signal transmission model; analyzing the downlink spectral efficiency of the system using the RZF precoding; analyzing the spectrum efficiency of the uplink using the MMSE receiver; modeling is carried out on the optimization problem of the solved RRU working mode selection; and solving the optimization problem by using a deep Q-learning algorithm to obtain the uplink and downlink working modes suitable for all nodes. According to the invention, in a cellular network-assisted full-duplex communication system in a low-altitude three-dimensional coverage scene, proper node uplink and downlink working modes can be selected, the large downlink problem of an unmanned aerial vehicle user in the low-altitude three-dimensional coverage scene is effectively solved, resource waste is avoided, and possibility is provided for further development of a cellular network-free network in the low-altitude three-dimensional coverage scene.

Description

Technical Field

[0001] The present invention relates to the field of network communication technology, and in particular to a method for selecting a node working mode of a non-cellular NAFD system in a low-altitude stereoscopic coverage scenario. The method can be used in a non-cellular network-assisted full-duplex communication system in a low-altitude stereoscopic coverage scenario to determine the uplink and downlink working modes of nodes in the system. Background Art

[0002] In the low-altitude three-dimensional coverage scenario of drones, there are problems such as rapid changes in user data demand and asymmetric uplink and downlink traffic. The collaboration capability and scalability of the cell-free radio access network (CF-RAN) are used to support dynamic scenarios, and network-assisted full-duplex (NAFD) is adopted to support flexible duplex communication by selecting the appropriate node duplex mode according to the traffic in real time.

[0003] CF-RAN is a distributed network architecture with collaborative processing capabilities. The architecture consists of remote radio units (RRUs), edge distributed units (EDUs), and user-centric distributed units (UCDUs). Each EDU connects to multiple RRUs and performs signal processing. By introducing distributed data processing units, CF-RAN can achieve a compromise between system scalability and performance. It also has more joint processing capabilities than traditional fully distributed unit-free systems.

[0004] NAFD technology can achieve flexible duplex communication in the space domain and is an effective and suitable technology to support low-altitude CF-RAN architecture. NAFD was proposed to improve resource utilization. It allows RRU to flexibly select duplex mode according to the traffic distribution in the environment. Each RRU can choose different working modes such as uplink reception or downlink transmission. However, drone users in low-altitude stereo coverage scenarios have a huge amount of downlink data. The existing technology cannot select the appropriate working mode for the access point according to the characteristics of the uplink and downlink traffic of the communication network, which will lead to resource waste. Summary of the invention

[0005] In view of this, the present invention proposes a method for selecting node working modes of a non-cellular NAFD system in a low-altitude stereoscopic coverage scenario. The method can select appropriate node uplink and downlink working modes in a non-cellular network-assisted full-duplex communication system in a low-altitude stereoscopic coverage scenario, effectively solve the large downlink problem of drone users in low-altitude stereoscopic coverage scenarios, avoid waste of resources, and provide possibilities for the further development of non-cellular networks in low-altitude stereoscopic coverage scenarios.

[0006] The technical solution adopted by the present invention is:

[0007] The method for selecting the node working mode of the non-cellular NAFD system in the low-altitude stereo coverage scenario includes the following steps:

[0008] Step 1: Model the non-cellular NAFD system in the low-altitude stereo coverage scenario and establish a signal transmission model;

[0009] Step 2, analyzing the downlink spectrum efficiency of a system using regularized zero-forcing precoding and the uplink spectrum efficiency of a system using a minimum mean square error receiver;

[0010] Step 3, with the goal of maximizing spectrum efficiency, establish an optimization problem for node working mode selection;

[0011] Step 4: Use the deep Q-learning method to solve the optimization problem established in step 3 to obtain the uplink and downlink working modes of all nodes.

[0012] Furthermore, the specific method of step 1 is:

[0013] Step 1.1, modeling a non-cellular NAFD system in a low-altitude stereoscopic coverage scenario, wherein the non-cellular NAFD system includes a cloud control unit, multiple user-centric distribution units and switches, M edge distribution units with cache and computing capabilities, and multiple access points, where N U Access points work in uplink mode and perform uplink reception. D The access points work in downlink mode and transmit downlink. The cellular-free NAFD system contains K active users, of which K U Uplink users, K D downlink users; each access point is equipped with L antennas;

[0014] Step 1.2, use the Rice channel model to model the channel of the cellular-free NAFD system, where the channel g between the nth access point and the i-th user is n,i It is expressed as:

[0015]

[0016] In the formula, represents the large-scale attenuation parameter between the nth access point and the i-th user, k n,i is the Rice factor between the nth access point and the i-th user, represents the LoS link, h n,i represents the path loss and spatial correlation of the transmission signal between the access point and the user, where h n,i It obeys the complex Gaussian symmetric distribution with mean 0 and variance as the identity matrix of dimension L;

[0017] Step 1.3, processing uplink data, the signal received by the edge distribution unit is expressed as:

[0018]

[0019] In the formula, r ul,m is the signal received by the edge distribution unit, K U is the number of uplink users in the system, K D is the number of downlink users, g ul,i ,g ul,j are the channel vectors of the i-th and j-th uplink users, p ul,i and p ul,j is the transmit power of the ith and jth uplink active users, p dl,k is the downlink transmission power to the kth user, x i 、x j and k denote the normalized signals sent by uplink user i, user j and downlink user k respectively, represents the interference matrix between access points working in downlink mode, w k is the channel precoding vector between the access point working in downlink mode and downlink user k, which is z ul Gaussian white noise of the uplink with a mean of 0;

[0020] Step 1.4: The signal y received by downlink user k is dl,k It is expressed as:

[0021]

[0022] In the formula, g dl,k is the channel vector of the kth downlink user, z dl,k is the Gaussian white noise of the downlink with a mean of 0, x i 、s k and j denote the normalized signals sent by uplink user i, downlink user k and user j respectively, and w k and w jrepresents the precoding vector of the channel between the access point working in downlink mode and the kth and jth downlink users, is the cross-link interference from uplink users to downlink users, u k,i represents the interference channel between user k and user i.

[0023] Furthermore, in step 2, the regularized zero-forcing precoding vector of the channel between the access point operating in the downlink mode and the kth downlink user is It is expressed as:

[0024]

[0025] Among them, the symbol Q dl Specifically represents the downlink channel estimation matrix, where Indicates that the dimension is N D ×L identity matrix, N D is the number of access points operating in downlink mode, L is the number of antennas, represents the estimated vector of the uplink user k channel, α>0 represents the regularization coefficient;

[0026] In the downlink transmission process of the UAV, based on the finite block length mechanism, the lower bound of the downlink spectrum efficiency corresponding to the kth user of the system is obtained The closed expression is:

[0027]

[0028] Among them, τ p is the length of the pilot sequence, the total coherent block length is T, f k The expression of (x) function is:

[0029]

[0030] In formula (5), γ dl,k is the closed expression of SINR of the kth user, expressed as:

[0031]

[0032] In the formula, in represents the line-of-sight path between user k and access point n, which obeys the unit complex Gaussian symmetric distribution, tr(×) represents the trace of the matrix, υ n,j for in P k represents the set of users that use the same pilot as user k; is the downlink noise power;

[0033] Uplink MMSE receive vector It is expressed as:

[0034]

[0035] Among them, the covariance matrix of residual interference and noise of the kth user is Σ k for:

[0036]

[0037] The spectral efficiency of the kth uplink user using an MMSE receiver is:

[0038]

[0039] The closed-form expression of the spectrum efficiency of the kth uplink user is:

[0040]

[0041] in, Indicates whether the nth access point and the edge distribution unit are connected, if yes, it is 1, otherwise it is 0.

[0042] Furthermore, the specific method of step 3 is:

[0043] Step 3.1, model the uplink spectrum efficiency optimization problem, expressed as:

[0044]

[0045] stC1:γ ul,k ≥γ ul,k,min k=1,2,3,…,K U (17)

[0046] C2:d n ={0,1} n=1,2,3,…,N (18)

[0047] C3:p ul,k ≤P ul k=1,2,3,…,K U (19)

[0048] C4:p ul,k ≥0 (20)

[0049] Where D = [d1,…,d N ] represents the duplex mode selection state of the access point; constraint C1 is the uplink service quality constraint for the UAV and ground users, where γ ul,k,min>0 indicates the required minimum uplink signal-to-noise ratio; constraint C2 indicates the working mode of the access point, d n Taking 0 means that the nth access point works in downlink mode, taking 1 means that it works in uplink mode, and each access point can only choose one mode; Constraints C3 and C4 are power constraints for uplink transmission, where P ul is the maximum value of each user's uplink transmission power;

[0050] Step 3.2, model the downlink spectrum efficiency optimization problem. The optimization goal is to maximize the total downlink spectrum efficiency under the limited block length mechanism, which is expressed as:

[0051]

[0052] stC5:γ dl,k ≥γ dl,k,min ,k=1,2,3,…,K D (17)

[0053] C6:ε≤ε max (18)

[0054]

[0055] C8:p dl,k ≥0 (20)

[0056] C2-C4 (21)

[0057] Among them, constraint C5 is the downlink service quality constraint between the UAV and the ground user, γ dl,k,min >0 indicates the required minimum downlink signal-to-noise ratio, constraint C6 is the bit error probability constraint under the finite block length mechanism, ε max represents the maximum bit error probability, constraint C7 is the maximum power constraint for downlink transmission precoding, and constraint C8 is the minimum power constraint for downlink transmission;

[0058] Step 3.3, a multi-objective optimization model is established for the uplink and downlink spectrum efficiency optimization problem, which is expressed as:

[0059]

[0060] stC1-C8 (23)

[0061] By solving the problem, the total uplink and downlink spectrum efficiency can be maximized simultaneously.

[0062] Furthermore, the specific method of step 4 is:

[0063] Step 4.1, set the basic parameters of the deep Q-learning method, where:

[0064] The state function s(t) is defined as A(t) represents the uplink and downlink selection vectors of the access point at step t;

[0065] The action function a(t) is defined as in represents the change of uplink and downlink selection of access points in step t, where only one access point mode can be changed in each step;

[0066] The reward function r(t) is defined as in, represents the total uplink spectrum efficiency of the user, represents the total downlink spectrum efficiency of the user under the finite block length mechanism, γ is the regularization parameter;

[0067] Step 4.2, set the operating parameters of the deep Q-learning method, specifically:

[0068] According to the state-action Q value function and ε-greedy strategy, with a fixed probability ε t Select the action a(t) with the largest Q value, ε t =0.001;

[0069] After executing action a(t), the forward function r(t) is obtained and the next state s(t+1) is reached;

[0070] Using the joint transition vector d(t) = [s(t), a(t), r(t), s(t+1)], the variables of the next state s(t+1) are stored in memory and uniformly sampled through batch training;

[0071] This is repeated, and at each step the state s(t) of the system at this step is input into the convolutional neural network, and then the Q value function Q(s(t), a(t)) is output;

[0072] During the optimization process, the parameters of the convolutional neural network are obtained by minimizing the squared error loss function:

[0073]

[0074] in, represents the maximum value of the Q-value function of the next state s(t+1), A is the action set, and Q(s(t), a(t)) is the Q-value function of selecting action a(t) in state s(t);

[0075] Step 4.3: Run the set deep Q-learning method to obtain the optimal node duplex mode selection scheme. The specific method is as follows:

[0076] Initialize the convolutional neural network, according to the state-action Q value function and ε-greedy strategy, with a fixed probability ε t Select the action a(t) with the largest Q value;

[0077] Based on the current state and the selected action, obtain the Q value and observe the next state s′;

[0078] Calculate the reward r(t) for action a(t) in step t, store the transition vector d(t) in memory, and then enter the next state s(t+1);

[0079] After the neural network training is completed, the system parameters of the previously modeled communication system, such as the total number of users, the number of uplink users, the serial number of uplink users, the number of access points, the number of antennas at each access point, the channel state information between all users and access points, the channel state information between all users, and the channel state information between all access points, are input into the trained neural network, and the neural network predicts the optimal duplex mode selection s with the highest reward. max , in s max In the working mode of all nodes represented, the system will have balanced and excellent uplink and downlink communication transmission rates.

[0080] The beneficial effects of the present invention are:

[0081] The present invention can select appropriate node uplink and downlink working modes in a non-cellular network-assisted full-duplex communication system in a low-altitude three-dimensional coverage scenario, effectively solve the large downlink problem of drone users in the low-altitude three-dimensional coverage scenario, avoid waste of resources, and provide possibility for the further development of non-cellular networks in low-altitude three-dimensional coverage scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 It is a flow chart of a method for selecting a node working mode of a non-cellular NAFD system in a low-altitude three-dimensional coverage scenario of the present invention. DETAILED DESCRIPTION

[0083] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0084] A node working mode selection method for a cellular-free NAFD system in a low-altitude stereo coverage scenario is proposed. This method analyzes the closed-form expression of the uplink spectrum efficiency under an infinite block length communication system and the lower bound of the downlink spectrum efficiency under a finite block length communication system under a scalable minimum mean-square-error (MMSE) receiver and a regularized zero-forcing (RZF) precoding scheme. A multi-objective optimization problem is established to maximize and balance the spectrum efficiency of the uplink and downlink links, and a deep Q-learning algorithm is used to solve it, ultimately achieving the selection of appropriate node uplink and downlink working modes.

[0085] like Figure 1 As shown, the method comprises the following steps:

[0086] (1) Modeling of non-cellular communication system in low-altitude stereo coverage scenario

[0087] Specifically, the steps include:

[0088] (1.1) First, the NAFD system with low-altitude 3D coverage without cellular is modeled. The system is equipped with a cloud control unit (CCU), multiple UCDUs and switches, M EDUs with cache and computing capabilities, and a large number of RRUs. RRUs are equivalent to access points (APs) and are equipped with L antennas. Each EDU is connected to multiple RRUs and performs signal processing. In the NAFD system, N ul RRUs work in uplink mode to perform uplink reception, N dl RRUs perform downlink transmission in downlink mode. The system contains K active users, of which K u Uplink users, K d Downlink users.

[0089] (1.2) Then, the Rician channel model is used to model the channel. The channel between the nth RRU and the i-th user is expressed as:

[0090]

[0091] in, k represents the large-scale attenuation parameter between the nth RRU and the ith user, and the other part in the bracket is the small-scale attenuation parameter, where the first term represents the LoS part of the channel and the second term represents the NLoS part of the channel. n,i is the Ricean factor between the nth RRU and the i-th user, defined as the ratio of the LoS link to the NLoS link. Indicates a LoS link, Represents the path loss and spatial correlation of the signal transmitted between the RRU and the user.

[0092] (2) Signal transmission model analysis

[0093] Specifically, the method includes the following steps:.

[0094] (2.1) During uplink transmission, all uplink RRUs receive signals from uplink users together. After the UCDU merges the data streams from the EDU, it can process the uplink data. The signal received by the EDU can be expressed as:

[0095]

[0096] p ul,i is the transmit power of the ith uplink active user. i and k are the normalized signals sent by uplink user i and downlink user k respectively. represents the interference matrix between downlink RRUs. k is the channel precoding vector between the downlink RRU and downlink user k. ul is Gaussian white noise.

[0097] (2.2) The signal received by downlink user k is expressed as:

[0098]

[0099] in, is the noise, w k represents the precoding vector of the channel between the downlink RRU and the kth downlink user. The third part of the formula is the cross-link interference of the uplink user to the downlink user.

[0100] (3) Analysis of downlink spectrum efficiency of systems using RZF precoding

[0101] Specifically, the steps include:

[0102] (3.1) First, in order to reduce interference, the RZF precoding scheme implemented in the EDU is considered. The RZF precoding vector of the channel between the downlink RRU and the kth downlink user can be expressed as:

[0103]

[0104] in Downlink channel estimation matrix, α>0 represents the regularization coefficient. By selecting an appropriate regularization parameter α, the interference between users can be flexibly controlled and the downlink spectrum efficiency can be improved. When α=0, it corresponds to ZF precoding; when α approaches infinity, it corresponds to matched filter precoding.

[0105] (3.2) Then, the downlink spectrum efficiency of the system is analyzed. Based on RZF precoding, the downlink signal-to-noise ratio (SINR) of the kth downlink user can be calculated as:

[0106]

[0107] In the downlink transmission process of the UAV, considering the low latency and high reliability of communication, the downlink spectrum efficiency of the kth UAV is analyzed based on the finite block length communication (FBLC) mechanism. It is obtained that, among which, τ p is the leader length.

[0108] The formula for calculating the achievable rate is:

[0109]

[0110] In the formula, V(γ)=1-(1+γ) -2 is the channel scattering, ε is the block error probability, e is a natural constant, Q -1 (.) is the inverse function of the complementary cumulative distribution function (Q function) of a standard Gaussian random variable. μ = B·T is the number of bits used by each channel, and B is the system bandwidth. Directly putting (5) into (6) can calculate the achievable downlink spectral efficiency of the kth user, but it will bring high complexity, so the downlink closed expression lower bound is used.

[0111] (3.3) Finally, the closed expression of the lower bound of the downlink spectrum efficiency of the system is obtained to facilitate the selection of subsequent node modes. The lower bound expression of the downlink spectrum efficiency under the finite block length mechanism is:

[0112]

[0113] The expression of the f(x) function is:

[0114]

[0115] In formula (7), γ dl,k is the closed expression of SINR of the kth user, which can be expressed as:

[0116]

[0117] (4) Analysis of uplink spectrum efficiency using MMSE receiver

[0118] Specifically, the method includes the following steps:.

[0119] (4.1) First, consider the MMSE receiver scheme implemented in the EDU. In the low-altitude CF-RAN system,

[0120] The uplink baseband signal and the downlink baseband signal can be processed collaboratively through the coordination of the EDU, so the cross-link interference of the downlink RRU to the uplink RRU can be alleviated. is the channel estimation error of the interference channel matrix. After interference elimination, the residual cross-link interference between RRUs can be expressed as

[0121] The covariance matrix of residual interference and noise is:

[0122]

[0123] The MMSE received vector is expressed as:

[0124]

[0125] definition Then the MMSE receiving vector can be expressed as:

[0126]

[0127] (4.2) Under the MMSE receiver, the SINR of the kth uplink user can be expressed as:

[0128]

[0129] The first term in the denominator represents the interference from other uplink user channels, and the second term in the denominator represents the noise and interference caused by the downlink RRU, specifically: According to Shannon channel theory, the spectrum efficiency of the kth uplink user using an MMSE receiver is:

[0130]

[0131] The closed-form expression of the spectrum efficiency of the kth uplink user can be expressed as:

[0132]

[0133] (5) Modeling the optimization problem of RRU working mode selection

[0134] Specifically, the method includes the following steps:.

[0135] (5.1) First, the uplink spectrum efficiency optimization problem is modeled. Since drones and ground users have different requirements for uplink and downlink services, in the low-altitude CF-RAN system where drones and GUEs coexist, the duplex mode of RRU can be flexibly scheduled according to the real-time needs of users based on NAFD technology. This multi-objective optimization problem (MOOP) can be modeled as follows:

[0136]

[0137] stC1:γ ul,k ≥γ ul,k,min k∈K U (17)

[0138] C2:d n ={0,1} n∈N (18)

[0139] C3:p ul,k ≤P ul k∈K U (19)

[0140] C4:p ul,k ≥0 (20)

[0141] Where D = [d1,…,d d,N ] represents the duplex mode selection state of RRU; C1 is the uplink quality of service (QoS) constraint of UAV and GUE, where γ ul,k,min >0 indicates the required minimum uplink SINR. Constraint C2 indicates the working mode of the RRU. n The value 0 indicates that the nth RRU works in downlink mode, and the value 1 indicates that it works in uplink mode. Each RRU can only select one mode. Constraints C3 and C4 are power constraints for uplink transmission, where P ul is the maximum uplink transmission power of each user.

[0142] (5.2) Then, the downlink spectrum efficiency optimization problem is modeled. The second optimization goal is to maximize the total downlink spectrum efficiency under the FBLC mechanism, which can be expressed as:

[0143]

[0144] stC5:γ dl,k ≥γ dl,k,min ,k∈K D (17)

[0145] C6:ε≤ε max (18)

[0146]

[0147] C8:p dl,k ≥0 (20)

[0148] C2-C4 (21)

[0149] Where C5 is the downlink QoS constraint of UAV and GUE, γ dl,k,min >0 indicates the required minimum downlink SINR, C6 is the bit error probability constraint under the FBLC mechanism, ε max represents the maximum tolerable bit error probability, C7 is the maximum power constraint for downlink transmission precoding, and C8 is the minimum power constraint for downlink transmission.

[0150] (5.3) Then, a multi-objective optimization model is established for the uplink and downlink spectrum efficiency optimization problem. Since the nth RRU can only select one working mode at the same time, and when the number of RRUs is fixed, the above two optimization goals are conflicting and cannot reach the optimal value at the same time. The MOOP can be expressed as:

[0151]

[0152] stC1-C8 (23)

[0153] By solving MOOP, the total uplink and downlink spectrum efficiency can be maximized simultaneously.

[0154] (6) Use the deep Q-learning (DQN) algorithm to solve MOOP and obtain the appropriate uplink and downlink working modes for all nodes.

[0155] Specifically, the method includes the following steps:.

[0156] (6.1) First, we need to clarify the basic parameter settings in the DQN algorithm. The state function s(t) is defined as:

[0157]

[0158] In the above formula, A(t) represents the uplink and downlink selection vector of the RRU at step t. The action function a(t) is defined as:

[0159]

[0160] The reward function r(t) is defined as:

[0161]

[0162] in, represents the total uplink spectrum efficiency of the user, represents the total downlink spectrum efficiency of the user under the FBLC mechanism. γ is the regularization parameter, which is set to 1 to ensure network convergence. is a constant related to the overall spectral efficiency.

[0163] (6.2) Then, the operating parameter settings of the DQN network are clarified. In this algorithm, according to the state-action Q value function and the ε greedy scheme, the fixed probability ε t Select the action a(t) with the largest Q value and set the probability to 0.2. After executing this action, the forward function r(t) can be obtained and the next state s(t+1) can be reached. Use the joint transition vector d(t)=[s(t),a(t),r(t),s(t+1)] to store the above variables in memory and uniformly sample them through batch training. At each step, the state s(t) of the system at this step is input into the convolutional neural network, and then the Q value function Q(s(t),a(t)) is output. During the optimization process, the parameters of the convolutional neural network are obtained by minimizing the squared error loss function:

[0164]

[0165] in, represents the maximum value of the Q-value function of the next state s￠, A is the action set, and Q(s(t), a(t)) is the Q-value function of selecting action a(t) in state s(t).

[0166] (6.3) Finally, run the complete DQN algorithm. First, initialize the network, and according to the state-action Q value function and the ε greedy scheme, follow the fixed probability ε t Select the action a(t) with the largest Q value, obtain the Q value based on the current state and the selected action, observe the next state s', calculate the reward r(t) of action a(t) in the tth step, store the transition vector d(t) in the memory, and then enter the next state s(t+1). After the network training is completed, the specific parameters of the communication system are input into the network to obtain the optimal duplex mode selection s with the highest reward. max In s max In the working mode of all nodes represented, the system will have balanced and excellent uplink and downlink communication transmission rates.

[0167] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0168] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for selecting a node working mode of a non-cellular NAFD system in a low-altitude stereo coverage scenario, characterized in that: The following steps are involved: Step 1: Model the non-cellular NAFD system in the low-altitude stereo coverage scenario and establish a signal transmission model; Step 2, analyzing the downlink spectrum efficiency of a system using regularized zero-forcing precoding and the uplink spectrum efficiency of a system using a minimum mean square error receiver; Step 3, with the goal of maximizing spectrum efficiency, establish an optimization problem for node working mode selection; Step 4: Use the deep Q-learning method to solve the optimization problem established in step 3 to obtain the uplink and downlink working modes of all nodes.

2. The method for selecting a node working mode of a non-cellular NAFD system in a low-altitude stereoscopic coverage scenario according to claim 1, characterized in that: The specific method of step 1 is: Step 1.1, modeling a non-cellular NAFD system in a low-altitude stereoscopic coverage scenario, wherein the non-cellular NAFD system includes a cloud control unit, multiple user-centric distribution units and switches, M edge distribution units with cache and computing capabilities, and multiple access points, where N U Access points work in uplink mode and perform uplink reception. D The access points work in downlink mode and transmit downlink. The cellular-free NAFD system contains K active users, of which K U Uplink users, K D downlink users; each access point is equipped with L antennas; Step 1.2, use the Rice channel model to model the channel of the cellular-free NAFD system, where the channel g between the nth access point and the i-th user is n,i It is expressed as: In the formula, represents the large-scale attenuation parameter between the nth access point and the i-th user, k n,i is the Rice factor between the nth access point and the i-th user, represents the LoS link, h n,i represents the path loss and spatial correlation of the transmission signal between the access point and the user, where h n,i It obeys the complex Gaussian symmetric distribution with mean 0 and variance as the identity matrix of dimension L; Step 1.3, processing uplink data, the signal received by the edge distribution unit is expressed as: In the formula, r ul,m is the signal received by the edge distribution unit, K U is the number of uplink users in the system, K D is the number of downlink users, g ul,i ,g ul,j are the channel vectors of the i-th and j-th uplink users, p ul,i and p ul,j is the transmit power of the ith and jth uplink active users, p dl,k is the downlink transmission power to the kth user, x i 、x j and k denote the normalized signals sent by uplink user i, user j and downlink user k respectively, represents the interference matrix between access points working in downlink mode, w k is the channel precoding vector between the access point working in downlink mode and downlink user k, which is z ul Gaussian white noise of the uplink with a mean of 0; Step 1.4: The signal y received by downlink user k is dl,k It is expressed as: In the formula, g dl,k is the channel vector of the kth downlink user, z dl,k is the Gaussian white noise of the downlink with a mean of 0, x i 、s k and j denote the normalized signals sent by uplink user i, downlink user k and user j respectively, and w k and w j represents the precoding vector of the channel between the access point working in downlink mode and the kth and jth downlink users, is the cross-link interference from uplink users to downlink users, u k,i represents the interference channel between user k and user i.

3. The method for selecting a node working mode of a non-cellular NAFD system in a low-altitude stereoscopic coverage scenario according to claim 1, characterized in that: In step 2, the regularized zero-forcing precoding vector of the channel between the access point working in the downlink mode and the kth downlink user is It is expressed as: Among them, the symbol Q dl Specifically represents the downlink channel estimation matrix, where Indicates that the dimension is N D ×L identity matrix, N D is the number of access points operating in downlink mode, L is the number of antennas, represents the estimated vector of the uplink user k channel, a>0 represents the regularization coefficient; In the downlink transmission process of the UAV, based on the finite block length mechanism, the lower bound of the downlink spectrum efficiency corresponding to the kth user of the system is obtained The closed expression is: Among them, t p is the length of the pilot sequence, the total coherent block length is T, f k The expression of (x) function is: In formula (5), γ dl,k is the closed expression of SINR of the kth user, expressed as: In the formula, in represents the line-of-sight path between user k and access point n, which obeys the unit complex Gaussian symmetric distribution, tr(×) represents the trace of the matrix, T k,n =β k,n / (k k,n +1)I L , υ n,j for in P k represents the set of users that use the same pilot as user k; is the downlink noise power; Uplink MMSE reception vector It is expressed as: Among them, the covariance matrix of residual interference and noise of the kth user is Σ k for: The spectral efficiency of the kth uplink user using an MMSE receiver is: The closed-form expression of the spectrum efficiency of the kth uplink user is: in, Indicates whether the nth access point and the edge distribution unit are connected, if yes, it is 1, otherwise it is 0.

4. The method for selecting a node working mode of a non-cellular NAFD system in a low-altitude stereoscopic coverage scenario according to claim 1, characterized in that: The specific method of step 3 is: Step 3.1, model the uplink spectrum efficiency optimization problem, expressed as: s.t.C1:γ ul,k 3g ul,k,min k=1,2,3,…,K U (17) C2:d n ={0,1} n=1,2,3,…,N (18) C3:p ul,k ￡P ul k=1,2,3,…,K U (19) C4:p ul,k 30(20) Where D = [d1,...,d N ] represents the duplex mode selection state of the access point; constraint C1 is the uplink service quality constraint for the UAV and ground users, where γ ul,k,min >0 indicates the required minimum uplink signal-to-noise ratio; constraint C2 indicates the working mode of the access point, d n Taking 0 means that the nth access point works in downlink mode, taking 1 means that it works in uplink mode, and each access point can only choose one mode; Constraints C3 and C4 are power constraints for uplink transmission, where P ul is the maximum value of each user's uplink transmission power; Step 3.2, model the downlink spectrum efficiency optimization problem. The optimization goal is to maximize the total downlink spectrum efficiency under the limited block length mechanism, which is expressed as: s.t.C5:γ dl,k 3g dl,k,min ,k=1,2,3,…,K D (17) C6:e￡e max (18) C8:p dl,k 30(20)C2-C4(21) Among them, constraint C5 is the downlink service quality constraint between the UAV and the ground user, γ dl,k,min >0 indicates the required minimum downlink signal-to-noise ratio, constraint C6 is the bit error probability constraint under the finite block length mechanism, ε max represents the maximum bit error probability, constraint C7 is the maximum power constraint for downlink transmission precoding, and constraint C8 is the minimum power constraint for downlink transmission; Step 3.3, a multi-objective optimization model is established for the uplink and downlink spectrum efficiency optimization problem, which is expressed as: stC1-C8(23) solves the problem so that the total uplink and downlink spectrum efficiency can be maximized at the same time.

5. The method for selecting a node working mode of a non-cellular NAFD system in a low-altitude stereoscopic coverage scenario according to claim 1, characterized in that: The specific method of step 4 is: Step 4.1, set the basic parameters of the deep Q-learning method, where: The state function s(t) is defined as A(t) represents the uplink and downlink selection vectors of the access point at step t; The action function a(t) is defined as in represents the change of uplink and downlink selection of access points in step t, where only one access point mode can be changed in each step; The reward function r(t) is defined as in, represents the total uplink spectrum efficiency of the user, represents the total downlink spectrum efficiency of the user under the finite block length mechanism, γ is the regularization parameter; Step 4.2, set the operating parameters of the deep Q-learning method, specifically: According to the state-action Q value function and ε-greedy strategy, with a fixed probability ε t Select the action a(t) with the largest Q value, ε t =0.001; After executing action a(t), the forward function r(t) is obtained and the next state s(t+1) is reached; Using the joint transition vector d(t) = [s(t), a(t), r(t), s(t+1)], the variables of the next state s(t+1) are stored in memory and uniformly sampled through batch training; This is repeated, and at each step the state s(t) of the system at this step is input into the convolutional neural network, and then the Q value function Q(s(t), a(t)) is output; During the optimization process, the parameters of the convolutional neural network are obtained by minimizing the squared error loss function: in, represents the maximum value of the Q-value function of the next state s(t+1), A is the action set, and Q(s(t), a(t)) is the Q-value function of selecting action a(t) in state s(t); Step 4.3: Run the set deep Q-learning method to obtain the optimal node duplex mode selection scheme. The specific method is as follows: Initialize the convolutional neural network, according to the state-action Q value function and ε-greedy strategy, with a fixed probability ε t Select the action a(t) with the largest Q value; Based on the current state and the selected action, obtain the Q value and observe the next state s￠; Calculate the reward r(t) for action a(t) in step t, store the transition vector d(t) in memory, and then enter the next state s(t+1); After the neural network training is completed, the system parameters of the previously modeled communication system, such as the total number of users, the number of uplink users, the serial number of uplink users, the number of access points, the number of antennas at each access point, the channel state information between all users and access points, the channel state information between all users, and the channel state information between all access points, are input into the trained neural network, and the neural network predicts the optimal duplex mode selection s with the highest reward. max , in s max In the working mode of all nodes represented, the system will have balanced and excellent uplink and downlink communication transmission rates.

Images