Node mode selection and beamforming method in collaborative synaesthesia integration scenario

Abstract

The present invention discloses a node mode selection and beamforming method in a collaborative synaesthesia integration scenario, and belongs to the field of wireless communication. The present invention proposes a system node mode selection and beamforming scheme that can achieve the best balance between communication and sensing performance. In the first stage, the channel model and data transmission model of the system are first established, and the uplink and downlink communication signal-to-noise ratio expressions and the perception signal-to-noise ratio expressions are derived; in the second stage, the expressions of maximum ratio transmission MRT beamforming, zero-forcing ZF beamforming and conjugate perception beamforming are derived; in the third stage, a node mode selection and beamforming algorithm based on a multi-agent hierarchical reinforcement learning algorithm is designed; in the fourth stage, the algorithm is called to obtain duplex mode selection and beam vector design for each access node. This method can improve system efficiency and promote the actual deployment of a non-cellular collaborative synaesthesia integration system, so the present invention has certain practical value.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and in particular to a node mode selection and beamforming method in a collaborative synaesthesia integration scenario. Background Art

[0002] Synaesthesia integration technology effectively integrates communication and perception functions through the sharing of spectrum resources and the reuse of hardware architecture. This technology significantly improves the spectrum efficiency of the system and greatly reduces the deployment cost, making it particularly suitable for the rapidly developing technology fields such as Internet of Vehicles and UAVs. Compared with the traditional synaesthesia integration system that uses single-station perception, where the transmitter and receiver are located in the same location, the multi-station collaborative synaesthesia integration system has gained wide attention in the industry due to its higher perception accuracy and wider coverage.

[0003] Cell-free massive multiple-input multiple-output (MIMO) technology provides a solid architectural foundation for realizing cooperative interawareness integration in the existing communication system. In this system, multiple access nodes AP are connected to the central processing unit CPU, which is commanded by the CPU to perform coherent transmission and reception tasks to serve all users. These access nodes can operate in the same frequency band and time to achieve communication with user devices and synchronous perception of targets. In the cell-free cooperative interawareness integration system, the realization of dual functions depends on beamforming technology. At present, a large number of studies have focused on the optimization of power allocation, the adoption of fixed beam design, and the formulation of benchmark schemes, and algorithms for joint base station selection and beamforming design have been proposed. However, these studies mainly focus on the scenarios where uplink or downlink communication and radar perception coexist, and fail to fully consider the complex situation where both requirements exist simultaneously in actual applications. In addition, directly deploying full-duplex access nodes may limit communication performance. Summary of the invention

[0004] The present invention provides a node mode selection and beamforming method in a collaborative synaesthesia integration scenario. Existing studies have mostly focused on the coexistence scenarios of uplink or downlink communication and radar perception, while ignoring the actual situation where both requirements coexist in practical applications. In addition, the direct deployment of full-duplex access nodes may limit communication performance. In response to this problem, the present invention proposes a non-cellular collaborative synaesthesia integration architecture that comprehensively considers uplink and downlink communications and radar perception. Although the architecture has potential advantages, the challenges it faces include high coupling and difficult-to-handle interference problems. To this end, the present invention provides an innovative node mode selection and beamforming scheme that can achieve an optimal balance between communication and sensing performance, thereby significantly improving the overall efficiency of the system. Through a carefully designed scheme, we can effectively deal with complex interference problems in the collaborative synaesthesia integration system, ensuring that the system can maintain efficient perception capabilities while meeting communication requirements.

[0005] An embodiment of the present invention provides a node mode selection and beamforming method in a collaborative synaesthesia integration scenario, comprising the following steps:

[0006] Step 1: establish a channel model and a data transmission model for a non-cellular cooperative interaceptive integrated system, and derive an expression for the uplink and downlink communication signal-to-interference-to-noise ratio and an expression for the perception signal-to-interference-to-noise ratio;

[0007] Step 2, deriving the expressions of maximum ratio transmission MRT beamforming and zero forcing ZF beamforming for communication signals in the non-cellular cooperative synaesthesia integrated system, and the expression of conjugate beamforming for perception signals;

[0008] Step 3, design a node mode selection and beamforming algorithm based on a multi-agent hierarchical reinforcement learning algorithm;

[0009] Step 4: Call the designed node mode selection and beamforming algorithm to obtain the duplex mode selection and beamforming vector design of each access node.

[0010] The node mode selection and beamforming method in the collaborative synaesthesia integration scenario of the embodiment of the present invention aims at the problems caused by the high coupling of the non-cellular collaborative synaesthesia integration system, makes full use of the advantages of high concentration and high coverage of the non-cellular architecture, and uses the multi-agent hierarchical reinforcement learning method to achieve the optimal balance between communication and perception performance. The present invention first proposes a deployment framework for a non-cellular collaborative synaesthesia integration system, establishes a channel model and a data transmission model according to the system scenario; then derives the uplink and downlink communication signal-to-noise ratio expression and the perception signal-to-noise ratio expression; then designs the mode selection and beamforming algorithm; finally, adopts the mode selection and beamforming optimization algorithm based on hierarchical reinforcement learning to perform duplex mode selection and beamforming design for each access node AP. The method proposed in the present invention makes full use of the excellent performance of the non-cellular collaborative synaesthesia integration system, can achieve the optimal balance between communication and perception performance, thereby further improving the system efficiency, and is suitable for meeting the diverse scenarios and service requirements in the sixth generation of communications, which is of great significance to the actual deployment of 6G synaesthesia integration.

[0011] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0013] Figure 1A flowchart of a node mode selection and beamforming method in a collaborative synaesthesia integration scenario provided according to an embodiment of the present invention;

[0014] Figure 2 A comparison diagram of the cumulative distribution function (CDF) of the communication performance between the method according to the embodiment of the present invention and the existing access node mode selection and beamforming design method;

[0015] Figure 3 The figure is a comparison diagram of the cumulative distribution function (CDF) of the perceived performance between the method according to the embodiment of the present invention and the existing access node mode selection and beamforming design method. DETAILED DESCRIPTION

[0016] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0017] Based on the problems raised in the background technology, the present invention explores a non-cellular collaborative synaesthesia integration scenario that takes into account uplink and downlink communications and radar perception. Although this architecture has obvious advantages, there is a highly coupled and difficult-to-handle interference problem. Therefore, in this system, the design of synaesthesia integrated beamforming and the selection of access node mode are particularly important, and these design decisions are closely related to many aspects of the system. Therefore, effective solutions should be designed from these key perspectives.

[0018] Figure 1 The present invention provides a flowchart of a node mode selection and beamforming method in a collaborative synaesthesia integration scenario according to an embodiment of the present invention.

[0019] like Figure 1 As shown, the node mode selection and beamforming method in the collaborative synaesthesia integration scenario includes the following steps:

[0020] Step 1: Establish the channel model and data transmission model of the non-cellular cooperative interaceptive integrated system, and derive the expression of the uplink and downlink communication signal-to-interference-noise ratio and the perception signal-to-interference-noise ratio.

[0021] In one embodiment of the present invention, step 1 specifically includes:

[0022] Step 101, establish a scenario model of a non-cellular cooperative synaesthesia integrated system managed by a central processor; wherein M The access nodes AP are K Provide services to users and sense a point target on the same time-frequency resource; each access node AP is equipped withN Half-duplex antennas, each user is equipped with a half-duplex antenna; K Users include K ul Uplink demand users and K dl Users with downlink demand; and Represents the index set of uplink users and downlink users respectively; use two 1-line M Binary vector of columns z ul and z dl To model AP mode selection; m For the set [1, M ] is any integer in m The access node AP is used for uplink reception (downlink transmission) mode, then z ul ( z dl ) m elements equal to 1, z dl No. m The element is equal to 0; if the mth access node AP is used in downlink transmission mode, then z ul No. m elements are equal to 0, z dl No. m elements are equal to 1; using the quasi-static block fading model, in each coherent time block, there are M ul =[ z ul ] sum The access node AP works in uplink mode. M dl =[ z dl ] sum The access node AP works in downlink mode, [·] sum Represents the sum of all elements in a vector; and Respectively represent the index sets of uplink APs and downlink APs.

[0023] set up k For the set [1, K ] is any integer in the set [1,M], and m is any integer in the set [1,M]. m The channel vector of the access node AP is recorded as N A vector with 1 row and 1 column ;in, Indicates k User and m The large-scale fading coefficient between access nodes AP, express The small-scale fast fading vector has a mean of 0 and a correlation matrix of I N The multivariate cyclically symmetric complex Gaussian distribution of N Indicates that the rows and columns are N The identity matrix of k The channel vector from each user to all access nodes AP is recorded as MN A vector with 1 row and 1 column , Indicates k The channel vector from the user to the first access node AP, Indicates k Users to M The channel vector of the access node AP, with superscript (·) T Represents the transpose operation on a vector.

[0024] set up j For the set [1, K ul ], l For the set [1, K dl ], from j Uplink users to l The channel of a downlink user is denoted as a complex number ;set up i For the set [1, M ] is any integer in the range of , the i-th access node AP and the m The direct channel between access nodes AP without target reflection is recorded as N OK N Column complex matrix ;in, represents the large-scale fading coefficient, express The small-scale fast fading matrix of i = m hour, The value of is 0; combine the direct channel matrices between all access nodes AP to obtain all direct channels between access nodes AP , is a MN OK MN A complex matrix of columns.

[0025] From i Access node AP to the mThe access node AP passes through the target reflection channel and is represented as a N OK N Column complex matrix ;in, It is i Access node AP to the m The complex amplitude of the target reflection channel of the access node AP, is a N A vector with 1 row and 1 column, indicating that m The steering vector from the access node AP to the sensing target, Indicates that from i The steering vector from the access node AP to the sensing target, with superscript (·) H Indicates the conjugate transpose operation of the vector; combine the target reflection channel matrices between all access nodes AP to obtain the total target reflection channel matrix between access nodes AP .

[0026] Step 102, each coherence time symbols are used for uplink pilot training; the minimum mean square error channel estimation method is used to obtain the k Channel vectors from users to all access nodes AP Estimated value of ,in, It is k Users to m The estimated channel of the access node AP, yes The equivalent large-scale fading coefficient of express The small-scale fast fading vector of The channel estimation error is recorded as ,in It is k Users to m The estimated channel error of the access node AP.

[0027] Step 103: During the remaining symbol of each coherence time, the non-cellular cooperative synaesthesia integrated system performs full-duplex communication and target perception simultaneously; for downlink transmission, select M dl Downlink access node AP K dl Each downlink user sends a communication data signal and sends a radar signal to sense the speed and direction of the target; l For the set [1, K dl ] is any integer in l The received downlink user signal is expressed as:

[0028] (1)

[0029] in, represents the node downlink mode selection matrix, It means to build a matrix with the vector in brackets as the diagonal. represents the Kronecker product; and Indicates l and The estimated channels from downlink users to all access nodes AP, Indicates l The actual channel vector from downlink users to all access nodes AP, Indicates l The estimated channel error vector from downlink users to all access nodes AP; , and Respectively indicate l , and k Communication precoding for downlink users; , and Respectively indicate l , and k Communication signals sent by downlink users; It is u The transmission power of uplink users is Indicates u Uplink users to l Downlink user channels, Indicates u Uplink data sent by uplink users; is additive Gaussian white noise; yes MN Row 1 and column 1 are beamforming sensing signals, which are defined as follows:

[0030] (2)

[0031] Formula (2) indicates: i For the set [1, M ] is any integer in i The access node AP is in downlink mode. No. Go to The value of the row is equal to ,in Indicates i The perceptual beamforming of the access nodes AP, Indicates i The sensing signal sent by the access node AP; if the i The access node AP is in uplink mode. No. Go to The values ​​of the rows are all 0.

[0032] No. l The signal-to-interference-to-noise ratio of a downlink user is:

[0033] (3)

[0034] And the l The achievable rate for a downlink user is ;in, , , , and They correspond to items 1 to 5 on the right side of formula (1), indicating the l The desired signal of each downlink user, as well as the interference signal caused by the downlink users, channel estimation errors, perception signals and uplink users' signals.

[0035] Step 104, for uplink communication, j The signal-to-interference-to-noise ratio of an uplink user is:

[0036] (4)

[0037] And the j The achievable rate for an uplink user is ; Indicates j The expected signal of the uplink user is It is for j The receiving vector of the uplink user is represents the node uplink mode selection matrix, It is j The estimated channels from uplink users to all access nodes AP, It is j The transmission power of uplink users is Indicates j Uplink data sent by uplink users; Indicates the interference caused by other uplink users. It is The estimated channels from uplink users to all access nodes AP, It is The transmission power of uplink users is Indicates Uplink data sent by uplink users; represents the interference caused by channel estimation error, It is u The transmission power of uplink users is Indicates u The estimated channel error vector from uplink users to all access nodes AP, Indicates u Uplink data sent by uplink users; Indicates the interference caused by the interfering signal. is the CPU’s estimation error matrix of the inter-AP channel, and are the inaccurate estimation coefficients of the direct channel between APs and the target reflection channel, respectively; Indicates the cross-link interference caused by the signal sent by the downlink access node AP. Expressing the k Communication precoding for downlink users, Expressing the k Communication signals sent by downlink users; Indicates noise interference.

[0038] Step 105, for target perception, m The received signal of an uplink access node AP is expressed as follows after partial interference elimination:

[0039] (5)

[0040] in, is the perception expectation signal containing target detection information, Indicates i Access node AP to the m The access node AP passes through the target reflection channel, Indicates i The perceptual beamforming of the access nodes AP, Indicates i The sensing signal sent by the access node AP; Represents the residual uplink communication interference, is the transmission power of the u-th uplink user, It is u The uplink user and m Uplink access node AP channel The estimated error vector is It is the first u Uplink data sent by uplink users The decoding error of represents the residual cross-link interference, Indicates iAccess node AP to the m The direct channel of the access node AP, Expressing the i Communication signals sent by downlink users; represents the residual interference caused by the downlink communication signal; is additive noise.

[0041] No. m The maximum value of the perceived signal-to-interference-noise ratio output by an uplink AP is:

[0042] (6)

[0043] in, ; Define the criteria used to evaluate the perceived performance of the system.

[0044] Step 2, derive the expressions of maximum ratio transmission MRT beamforming and zero forcing ZF beamforming for communication signals in the non-cellular cooperative synaesthesia integrated system, and the expression of conjugate beamforming for perception signals.

[0045] In one embodiment of the present invention, step 2 specifically includes:

[0046] Step 201, the central processing unit CPU obtains the maximum ratio transmission MRT beamforming vector and the zero forcing ZF beamforming vector for the communication signal according to the estimated channel between the user and the access node AP; when the maximum ratio transmission MRT beamforming scheme is adopted, the first k The MRT communication beamforming vector of a downlink user is expressed as MN Complex vector with rows and columns ,in, It is for k Estimate the channels between downlink users and all access nodes AP, represents the Euclidean norm of the vector; when the maximum ratio transmission zero-forcing ZF beamforming scheme is adopted, the k The ZF communication beamforming vector of a downlink user is expressed as ,in, is a matrix No. k Column, and , and Represent the first downlink user and the K dl Estimate the channels between downlink users and all access nodes AP.

[0047] Step 202: The central processor obtains a conjugate beamforming vector for the sensing signal according to the estimated channel between the access nodes AP. When the conjugate beamforming scheme is adopted, i The sensing beamforming vector of an access node AP is expressed as ,in, Indicates that from i The steering vector from the access node AP to the sensing target.

[0048] Step 3: Design a node mode selection and beamforming algorithm based on a multi-agent hierarchical reinforcement learning algorithm.

[0049] In one embodiment of the present invention, step 3 specifically includes:

[0050] Step 301, the joint optimization problem of node mode selection and beamforming design in the non-cellular cooperative synaesthesia integrated system is modeled as:

[0051] (7a)

[0052] (7b)

[0053] (7c)

[0054] (7d)

[0055] (7e)

[0056] (7f)

[0057] In the optimization goal, is the sum of the communication rates achievable by all users, R sen is the sum of the perceived signal-to-interference-and-noise ratios of all uplink APs, and μ represents the perceived performance weight; among the optimization variables, z dl represents the downstream node mode selection vector, Indicates i The ratio of the power of the communication signal sent by the downlink AP, φ i Indicates i The ratio of MRT beamforming in the communication beamforming vector of the downlink AP; and They represent the communication requirements of uplink users and downlink users respectively, It is i The total power consumed by the downlink access nodes AP.

[0058] Step 302, adopt a multi-agent hierarchical reinforcement learning method to solve the above optimization problem; at a high level, the central processor is regarded as a meta-agent; the meta-agent is in the first t The observations in a semi-Markov step are defined as a vector ; The meta-agent is in the t The action in the semi-Markov step is the node selected. The working mode is defined as a Dimensional array ,in z m ( t ) has a value of 0 or 1, corresponding to the m AP in the t semi-Markov steps are scheduled to work in uplink mode or downlink mode; when the meta-agent performs actions After that, the lower-level agent will quickly take small steps to design the beamforming matrix of the corresponding downlink access node, and the reward obtained by the meta-agent is the average of the lower-level rewards obtained by each small step of the lower-level agent.

[0059] At the low level, each access node AP is considered as an agent, and in each low-level small step, only the agent corresponding to the access node AP selected as the downlink mode will be selected The beamforming matrix is ​​designed based on the value of t The first of the semi-Markov steps t mstep The observation of a small step is defined as , This is the vector composed of the communication rate of each user and the system perceived performance at this time; m The lower-level agent is t The first of the semi-Markov steps t mstep Small steps of local action Defined as:

[0060] (8)

[0061] Formula (8) shows that t In the semi-Markov steps, if m The access node AP is set to uplink mode. In all the small steps of this semi-Markov step, the first m The actions of all lower-level agents are 0. m The access node AP is set to downlink mode. In each small step of this semi-Markov step, the m The low-level agents need to choose ,in Indicatesm The ratio of the power of the communication signal sent by the access node AP, φ m Indicates m The ratio of MRT beamforming in the communication beamforming vector of the access node AP; the global action of the lower-level agent is the combination of the beamforming matrices of all downlink APs expressed as , is the combination of all lower-level agent actions; the reward of the lower-level agent in a small step is defined as , when the communication and perception performance of the system satisfies equations (7a), (7b) and (7c) ,otherwise .

[0062] Step 4: Call the designed node mode selection and beamforming algorithm to obtain the duplex mode selection and beamforming vector design of each access node.

[0063] Optionally, in one embodiment of the present invention, step 4 specifically includes:

[0064] Step 401, in the offline training phase of the joint optimization design of node mode selection and beamforming design based on multi-agent hierarchical reinforcement learning: t In the semi-Markov step, the meta-agent will obtain the observation of this step , and use a high-level action deep neural network to select the optimal AP mode action ; After executing the action, the meta-agent will receive the reward for this step and get the next moment's observation ; Meta-agent’s experience tuple will be stored in the high-level experience buffer; the meta-agent periodically takes out some tuples from the high-level experience buffer, uses the high-level comment deep neural network output to evaluate the action selection strategy, and updates the parameters of the high-level action deep neural network and the high-level comment deep neural network until convergence;

[0065] Similarly, for the m A low-level agent, in the t The first half of the Markov step Small steps to gain observation And use its low-level action deep neural network to select the best local action ; Performing low-level global actions After that, each agent receives t The first half of the Markov step t mstep Rewards for small steps And the next small step of observation ; Experience tuples of low-level agents will be stored in the low-level experience buffer; each low-level agent periodically takes out some tuples from the low-level experience buffer, uses the low-level comment deep neural network output to evaluate the action selection strategy, and updates the parameters of the low-level action deep neural network and the low-level comment deep neural network until convergence; finally, the optimal high-level and low-level strategies after training are obtained;

[0066] Step 402, in the online execution stage of the joint optimization design of node mode selection and beamforming design based on multi-agent hierarchical reinforcement learning: the estimated channel and prior channel state information within each coherence time are input into the trained multi-agent hierarchical reinforcement learning architecture, and the optimal node mode selection and beamforming matrix are output.

[0067] The node mode selection and beamforming method in the collaborative synaesthesia integration scenario of the present invention is described in detail below through a specific embodiment.

[0068] Assume a scenario of a non-cellular cooperative interawareness integrated system. There is a circular area with a radius of R = 250 meters in the scenario. All M = 6 access nodes AP and K = 6 single-antenna users are randomly distributed in the area. Among these K users, there are K dl = 4 downlink users and K ul =2 uplink users. Each access node is equipped with N=4 half-duplex antennas. The path loss exponent of the channel between the access node and the user is set to α=3.7, and the path loss exponents of the interference channel between the user and the access node are α U =3, α A = 3. A coherent block contains τ c = 200 symbols, of which τ p = 5 channel usage opportunities are dedicated to pilot transmission. The maximum power for each user to transmit pilot and uplink data is P UE =ρ P =100mw. The maximum power of each access node is P UE =2w, the noise signal power is The inaccuracy coefficient of the channel state information estimation between access nodes is and The perceived performance weight is set to μ=5.

[0069] Figure 2 The cumulative distribution function (CDF) comparison of the communication performance of the method of the present invention and the existing access node mode selection and beamforming design method is shown. Based on 500 randomly generated scenarios, Figure 2The advantages of the proposed node mode selection and beamforming scheme based on multi-agent hierarchical reinforcement learning (MAHRL (proposed)) in terms of communication performance are highlighted. Among them, QL+MRT-CS means using Q-learning reinforcement learning method for node mode selection, and using maximum ratio transmission MRT for beamforming of communication signals, and using conjugate precoding for beamforming of perception signals; Ran+MRT-CS means using random node mode selection, and using maximum ratio transmission MRT for beamforming of communication signals, and using conjugate precoding for beamforming of perception signals; QL+ZF-CS means using Q-learning reinforcement learning method for node mode selection, and using zero forcing ZF for beamforming of communication signals, and using conjugate precoding for beamforming of perception signals; FPMM represents the existing node mode selection and beamforming scheme with the goal of maximizing perception performance. By comparison and It can be seen that under the same access node mode selection method, increasing the proportion of communication beamforming transmission can bring significant benefits in terms of spectrum efficiency. Since the proposed scheme performs a more sophisticated scheduling design for the communication power ratio of each downlink access node, it can achieve better performance compared with the MRT-CS scheme with a fixed communication power ratio. , as well as In comparison, the average total rates obtained by the proposed method are 58.6%, 43.7%, 28.9% and 8% higher, respectively, and are almost twice that of the FPMM method.

[0070] Figure 3 The cumulative distribution function (CDF) comparison of the perceived performance of the proposed method and the existing access node mode selection and beamforming design methods is shown. The perceived performance of the proposed scheme is almost equivalent to that of the FPMM method, with only a slight decrease of 0.19. The average R sen The value is approximately Ran+MRT-CS, as well as Four times. and In comparison, the MAHRL-based AMJBD improves by 80% and 60% respectively. Therefore, the proposed MAHRL-based AMJBD can improve the communication and perception performance, making it very suitable for non-cellular cooperative synaesthesia integrated system.

[0071] In summary, the present invention aims at the problems caused by the high coupling of the non-cellular collaborative synaesthesia integration system, and proposes a node mode selection and beamforming scheme in the collaborative synaesthesia integration scenario. The present invention first proposes a deployment framework for non-cellular collaborative synaesthesia integration, and establishes a channel model and a data transmission model according to the system scenario; then the uplink and downlink communication signal-to-interference-noise ratio expression and the perception signal-to-interference-noise ratio expression are derived; then the mode selection and beamforming algorithm are designed; finally, the mode selection and beamforming optimization algorithm based on hierarchical reinforcement learning is adopted to perform duplex mode selection and beam design for each access node AP. The method proposed in the present invention makes full use of the excellent performance of the non-cellular collaborative synaesthesia integration system, and can achieve the optimal balance between communication and perception performance, thereby further improving the system efficiency. It is suitable for meeting the diverse scenarios and service requirements in the sixth generation of communications, and is of great significance to the actual deployment of 6G synaesthesia integration.

[0072] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0073] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0074] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

Claims

1. A method for node mode selection and beamforming in a collaborative synaesthesia integration scenario, characterized in that: The following steps are involved: Step 1: establish a channel model and a data transmission model for a non-cellular cooperative interaceptive integrated system, and derive an expression for the uplink and downlink communication signal-to-interference-to-noise ratio and an expression for the perception signal-to-interference-to-noise ratio; Step 2, deriving the expressions of maximum ratio transmission MRT beamforming and zero forcing ZF beamforming for communication signals in the non-cellular cooperative synaesthesia integrated system, and the expression of conjugate beamforming for perception signals; Step 3, designing a node mode selection and beamforming algorithm based on a multi-agent hierarchical reinforcement learning algorithm, specifically including: taking the node mode and beamforming vector as optimization variables, and the communication and perception performance as optimization targets, establishing an optimization problem, and optimizing the optimization problem using a multi-agent hierarchical reinforcement learning method. At a high level, the central processor is regarded as a meta-agent, and the meta-agent takes action to select the uplink and downlink modes of each access node by observing the communication achievable rate of all users in the current system and the sum of the perceived signal-to-interference-noise ratio of the uplink access node; at a low level, each access node is regarded as an agent, and only the agent corresponding to the access node selected as the downlink mode by the meta-agent will design the beamforming matrix of the corresponding downlink access node by selecting the power ratio of the communication signal sent by this access node and the ratio of the MRT beamforming; Step 4: Call the designed node mode selection and beamforming algorithm to obtain the duplex mode selection and beamforming vector design of each access node.

2. The method according to claim 1, characterized in that Step 1 specifically includes: Step 101, establish a scenario model of a non-cellular cooperative synaesthesia integrated system managed by a central processor; wherein M The access nodes AP are K Provide services to users and sense a point target on the same time-frequency resource; each access node AP is equipped with N Half-duplex antennas, each user is equipped with a half-duplex antenna; K Users include K ul Uplink demand users and K dl Users with downlink demand; and Represents the index set of uplink users and downlink users respectively; use two 1-line M Binary vector of columns z ul and z dl To model AP mode selection; m For the set [1, M ] is any integer in m Access nodes AP are used for uplink receiving mode, then z ul No. m elements equal to 1, z dl No. m The element is equal to 0; if the mth access node AP is used in downlink transmission mode, then z ul No. m elements are equal to 0, z dl No. m elements are equal to 1; using the quasi-static block fading model, in each coherent time block, there are M ul =[ z ul ] sum The access node AP works in uplink mode. M dl =[ z dl ] sum The access node AP works in downlink mode, [·] sum Represents the sum of all elements in a vector; and Respectively represent the index sets of uplink AP and downlink AP; set up k For the set [1, K ] is any integer in the set [1,M], and m is any integer in the set [1,M]. m The channel vector of the access node AP is recorded as N A vector with 1 row and 1 column ;in, Indicates k User and m The large-scale fading coefficient between access nodes AP, express The small-scale fast fading vector has a mean of 0 and a correlation matrix of I N The multivariate cyclically symmetric complex Gaussian distribution of N Indicates that the rows and columns are N The identity matrix of k The channel vector from each user to all access nodes AP is recorded as MN A vector with 1 row and 1 column , Indicates k The channel vector from the user to the first access node AP, Indicates k Users to M The channel vector of the access node AP, with superscript (·) T Indicates the transposition operation of the vector; set up j For the set [1, K ul ], l For the set [1, K dl ], from j Uplink users to l The channel of a downlink user is denoted as a complex number ;set up i For the set [1, M ] is any integer in the range of , the i-th access node AP and the m The direct channel between access nodes AP without target reflection is recorded as N OK N Column complex matrix ;in, represents the large-scale fading coefficient, express The small-scale fast fading matrix of i = m hour, The value of is 0; combine the direct channel matrices between all access nodes AP to obtain all direct channels between access nodes AP , is a MN OK MN A complex matrix of columns; From i Access node AP to the m The access node AP passes through the target reflection channel and is represented as a N OK N Column complex matrix ;in, It is i Access node AP to the m The complex amplitude of the target reflection channel of the access node AP, is a N A vector with 1 row and 1 column, indicating that m The steering vector from the access node AP to the sensing target, Indicates that from i The steering vector from the access node AP to the sensing target, with superscript (·) H Indicates the conjugate transpose operation of the vector; combine the target reflection channel matrices between all access nodes AP to obtain the total target reflection channel matrix between access nodes AP ; Step 102, each coherence time symbols are used for uplink pilot training; the minimum mean square error channel estimation method is used to obtain the k Channel vectors from users to all access nodes AP Estimated value of ,in, It is k Users to m The estimated channel of the access node AP, yes The equivalent large-scale fading coefficient of express The small-scale fast fading vector of The channel estimation error is recorded as ,in It is k Users to m The estimated channel error of the access node AP; Step 103: During the remaining symbol of each coherence time, the non-cellular cooperative synaesthesia integrated system performs full-duplex communication and target perception simultaneously; for downlink transmission, select M dl Downlink access node AP K dl Each downlink user sends a communication data signal and sends a radar signal to sense the speed and direction of the target; l For the set [1, K dl ] is any integer in l The received downlink user signal is expressed as: （1） in, represents the node downlink mode selection matrix, It means to build a matrix with the vector in brackets as the diagonal. represents the Kronecker product; and Indicates l and The estimated channels from downlink users to all access nodes AP, Indicates l The actual channel vector from downlink users to all access nodes AP, Indicates l The estimated channel error vector from downlink users to all access nodes AP; , and Respectively indicate l , and k Communication precoding for downlink users; , and Respectively indicate l , and k Communication signals sent by downlink users; It is u The transmission power of uplink users is Indicates u Uplink users to l Downlink user channels, Indicates u Uplink data sent by uplink users; is additive Gaussian white noise; yes MN Row 1 and column 1 are beamforming sensing signals, which are defined as follows: （2） Formula (2) indicates: i For the set [1, M ] is any integer in i The access node AP is in downlink mode. No. Go to The value of the row is equal to ,in Indicates i The perceptual beamforming of the access nodes AP, Indicates i The sensing signal sent by the access node AP; if the i The access node AP is in uplink mode. No. Go to The values ​​of the rows are all 0; No. l The signal-to-interference-to-noise ratio of a downlink user is: （3） And the l The achievable rate for a downlink user is ;in, , , , and They correspond to the five fractions on the right side of formula (1), indicating the l The desired signal of each downlink user, as well as the interference signal caused by the downlink users, channel estimation errors, sensing signals and uplink users' signals; Step 104, for uplink communication, j The signal-to-interference-to-noise ratio of an uplink user is: （4） And the j The achievable rate for an uplink user is ; Indicates j The expected signal of the uplink user is It is for j The receiving vector of the uplink user is represents the node uplink mode selection matrix, It is j The estimated channels from uplink users to all access nodes AP, It is j The transmission power of uplink users is Indicates j Uplink data sent by uplink users; Indicates the interference caused by other uplink users. It is The estimated channels from uplink users to all access nodes AP, It is The transmission power of uplink users is Indicates Uplink data sent by uplink users; represents the interference caused by channel estimation error, It is u The transmission power of uplink users is Indicates u The estimated channel error vector from uplink users to all access nodes AP, Indicates u Uplink data sent by uplink users; Indicates the interference caused by the interfering signal. is the CPU’s estimation error matrix of the inter-AP channel, and are the inaccurate estimation coefficients of the direct channel between APs and the target reflection channel, respectively; Indicates the cross-link interference caused by the signal sent by the downlink access node AP. Expressing the k Communication precoding for downlink users, Expressing the k Communication signals sent by downlink users; Indicates noise interference; Step 105, for target perception, m The received signal of an uplink access node AP is expressed as follows after partial interference elimination: （5） in, is the perception expectation signal containing target detection information, Indicates i Access node AP to the m The access node AP passes through the target reflection channel, Indicates i The perceptual beamforming of the access nodes AP, Indicates i The sensing signal sent by the access node AP; Represents the residual uplink communication interference, is the transmission power of the u-th uplink user, It is u The uplink user and m Uplink access node AP channel The estimated error vector is It is the first u Uplink data sent by uplink users The decoding error of represents the residual cross-link interference, Indicates i Access node AP to the m The direct channel of the access node AP, Expressing the i Communication signals sent by downlink users; represents the residual interference caused by the downlink communication signal; is additive noise; No. m The maximum value of the perceived signal-to-interference-noise ratio output by an uplink AP is: （6） in, ; Define the criteria used to evaluate the perceived performance of the system.

3. The method according to claim 2, characterized in that Step 2 specifically includes: Step 201: The central processor obtains a maximum ratio transmission MRT beamforming vector and a zero forcing ZF beamforming vector for the communication signal according to the estimated channel between the user and the access node AP; when the maximum ratio transmission MRT beamforming scheme is adopted, the first k The MRT communication beamforming vector of a downlink user is expressed as MN Complex vector with rows and columns ,in, It is for k Estimate the channels between downlink users and all access nodes AP, represents the Euclidean norm of the vector; when the maximum ratio transmission zero-forcing ZF beamforming scheme is adopted, the k The ZF communication beamforming vector of a downlink user is expressed as ,in, is a matrix No. k Column, and , and Represent the first downlink user and the K dl Estimate the channels between downlink users and all access nodes AP; Step 202: The central processor obtains a conjugate beamforming vector for the sensing signal according to the estimated channel between the access nodes AP. When the conjugate beamforming scheme is adopted, i The sensing beamforming vector of an access node AP is expressed as ,in, Indicates that from i The steering vector from the access node AP to the sensing target.

4. The method according to claim 2, characterized in that: Step 3 specifically includes: Step 301, the joint optimization problem of node mode selection and beamforming design in the non-cellular cooperative synaesthesia integrated system is modeled as: （7a） （7b） （7c） （7d） （7e） （7f） In the optimization goal, is the sum of the communication rates achievable by all users, R sen is the sum of the perceived signal-to-interference-and-noise ratios of all uplink APs, and μ represents the perceived performance weight; among the optimization variables, z dl represents the downstream node mode selection vector, Indicates i The ratio of the power of the communication signal sent by the downlink AP, φ i Indicates i The ratio of MRT beamforming in the communication beamforming vector of the downlink AP; and They represent the communication requirements of uplink users and downlink users respectively, It is i The total power consumed by the downlink access nodes AP; Step 302, adopt a multi-agent hierarchical reinforcement learning method to solve the above optimization problem; at a high level, the central processor is regarded as a meta-agent; the meta-agent is in the first t The observations in a semi-Markov step are defined as a vector ; The meta-agent is in the t The action in the semi-Markov step is the node selected. The working mode is defined as a Dimensional array ,in z m ( t ) has a value of 0 or 1, corresponding to the m The AP is in t semi-Markov steps are scheduled to work in uplink mode or downlink mode; when the meta-agent performs actions After that, the lower-level agents will quickly take small steps to design the beamforming matrix of the corresponding downlink access node, and the reward obtained by the meta-agent is the average of the lower-level rewards obtained by each small step of the lower-level agents; At the low level, each access node AP is considered as an agent, and in each low-level small step, only the agent corresponding to the access node AP selected as the downlink mode will be selected The beamforming matrix is ​​designed based on the value of t The first of the semi-Markov steps t mstep The observation of a small step is defined as , This is the vector composed of the communication rate of each user and the system perceived performance at this time; m The lower-level agent is t The first of the semi-Markov steps t mstep Small steps of local action Defined as: （8） Formula (8) shows that t In the semi-Markov steps, if m The access node AP is set to uplink mode. In all the small steps of this semi-Markov step, the first m The actions of all lower-level agents are 0. m The access node AP is set to downlink mode. In each small step of this semi-Markov step, the m The low-level agents need to choose ,in Indicates m The ratio of the power of the communication signal sent by the access node AP, φ m Indicates m The ratio of MRT beamforming in the communication beamforming vector of the access node AP; the global action of the lower-level agent is the combination of the beamforming matrices of all downlink APs expressed as , is the combination of all lower-level agent actions; the reward of the lower-level agent in a small step is defined as , when the communication and perception performance of the system satisfies equations (7a), (7b) and (7c) ,otherwise .

5. The method according to claim 4, characterized in that Step 4 specifically includes: Step 401, in the offline training phase of the joint optimization design of node mode selection and beamforming design based on multi-agent hierarchical reinforcement learning: t In the semi-Markov step, the meta-agent will obtain the observation of this step , and use a high-level action deep neural network to select the optimal AP mode action ; After executing the action, the meta-agent will receive the reward for this step and get the next moment's observation ; Meta-agent’s experience tuple will be stored in the high-level experience buffer; the meta-agent periodically takes out some tuples from the high-level experience buffer, uses the high-level comment deep neural network output to evaluate the action selection strategy, and updates the parameters of the high-level action deep neural network and the high-level comment deep neural network until convergence; Similarly, for the m A low-level agent, in t The first semi-Markov step Small steps to gain observation And use its low-level action deep neural network to select the best local action ; Performing low-level global actions After that, each agent receives t The first semi-Markov step t mstep Rewards for small steps And the next small step of observation ; Experience tuples of low-level agents will be stored in the low-level experience buffer; each low-level agent periodically takes out some tuples from the low-level experience buffer, uses the low-level comment deep neural network output to evaluate the action selection strategy, and updates the parameters of the low-level action deep neural network and the low-level comment deep neural network until convergence; finally, the optimal high-level and low-level strategies after training are obtained; Step 402, in the online execution stage of the joint optimization design of node mode selection and beamforming design based on multi-agent hierarchical reinforcement learning: the estimated channel and prior channel state information within each coherence time are input into the trained multi-agent hierarchical reinforcement learning architecture, and the optimal node mode selection and beamforming matrix are output.

Images