Node selection and beamforming design method in dynamic synaesthesia integrated network

Abstract

The present invention discloses a node selection and beamforming design method for a dynamic synaesthesia integrated network, comprising: step S1, constructing a communication and perception model of the dynamic synaesthesia integrated network system; step 2, constructing a communication performance, perception performance, and delay performance representation of the dynamic synaesthesia integrated network based on the communication and perception model of the dynamic synaesthesia integrated network system; step S3, constructing an optimization model for node selection and beamforming of the dynamic synaesthesia integrated network based on the communication performance, perception performance, and delay performance representation of the dynamic synaesthesia integrated network; and step S4, expressing the optimization model as a Markov decision process and solving it using deep reinforcement learning to obtain a node selection scheme and a beamforming design scheme. The technical solution of the present invention achieves a trade-off between communication and perception performance.

Description

Technical Field

[0001] The present invention belongs to the technical field of wireless communications, and in particular relates to a node selection and beamforming design method in a dynamic synaesthesia integrated network. Background Art

[0002] With the rapid development of intelligent applications such as smart cities, intelligent transportation, and unmanned factories, the demand for wireless infrastructure with environmental awareness capabilities is growing. Driven by this trend, breakthroughs in 5G-A and 6G technologies have provided new opportunities for intelligent upgrades in communication base stations. Currently, communication base stations are evolving from a single signal transmission function to integrated communication and awareness, leveraging the communication signals they transmit to achieve environmental awareness. This technological approach not only fully leverages the potential of existing infrastructure but also significantly reduces the cost of deploying additional dedicated sensing equipment. In complex application scenarios such as smart cities and intelligent transportation, communication base stations must achieve high-precision and high-reliability environmental awareness, placing stringent demands on the robustness of the perception system. However, compared to dedicated radar systems, perception based on communication signals faces significant challenges: First, due to the resource allocation mechanism that prioritizes communication services, the time-frequency resources available for perception tasks are relatively limited. Second, communication signals generally operate in lower frequency bands and in complex spectral environments, resulting in low signal-to-interference-noise ratios. These factors severely limit the accuracy and reliability of the system in critical perception tasks such as target parameter estimation. In summary, single-base station perception solutions are no longer able to meet practical application requirements. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a node selection and beamforming design method in a dynamic synaesthesia integrated network.

[0004] To achieve the above object, the present invention adopts the following technical solutions:

[0005] A node selection and beamforming design method in a dynamic synaesthesia integrated network includes:

[0006] Step S1, constructing a communication and perception model of a dynamic synaesthesia integrated network system;

[0007] Step 2: Based on the communication and perception model of the dynamic synaesthesia integrated network system, construct the communication performance, perception performance and delay performance representation of the dynamic synaesthesia integrated network;

[0008] Step S3: constructing an optimization model for node selection and beamforming of the dynamic synaesthesia integrated network based on the communication performance, perception performance, and delay performance of the dynamic synaesthesia integrated network;

[0009] Step S4: Express the optimization model as a Markov decision process and solve it using deep reinforcement learning to obtain the node selection scheme and beamforming design scheme.

[0010] The optimization model of node selection and beamforming for the optimal dynamic synaesthesia integrated network is:

[0011] ;

[0012] in, is the weight of perceived performance, is the weight of the delay, is the maximum transmit power threshold of each node, is the threshold of the minimum communication speed of each communication user, is the maximum delay threshold for each sensing request; Is the system in time slot Actions taken when It is in the time slot Time Base Station The beamforming matrix of is its conjugate transposed matrix, is the average parameter estimate of the sensing request; is the average perceived delay, 、 and Represent the indexes of base station, target and time slot respectively, Indicates base station Users The communication rate, Indicates the target In the time slot The latency of the perception request generated when

[0013] The present invention first establishes a communication and perception model for a dynamic synaesthesia-integrated network, including dynamically arriving perception requests. It then defines representations for communication, perception, and latency performance, and establishes corresponding optimization problems. Finally, the optimization problem is solved using a Markov decision process and deep reinforcement learning methods, achieving a trade-off between communication and perception performance. The present invention has the following technical effects:

[0014] 1. Through dynamic node selection and dynamic beamforming design, the present invention can flexibly allocate resources according to real-time perception requests and channel conditions, thereby achieving an efficient trade-off between communication and perception tasks.

[0015] 2. The present invention adopts the model-free Q-learning reinforcement learning method, which can autonomously learn the optimal strategy in a dynamic environment, adapt to unknown transition probabilities and time-varying channel conditions, and achieve long-term performance optimization.

[0016] 3. The present invention takes into account communication performance, perception performance and latency performance through a unified optimization framework. This multi-objective collaborative optimization capability enables the simultaneous satisfaction of multiple demands of communication, perception and latency in complex dynamic environments, and is suitable for scenarios with high requirements for real-time performance, perception accuracy and communication quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0018] Figure 1 This is a flow chart of a node selection and beamforming design method in a dynamic synaesthesia integrated network according to an embodiment of the present invention;

[0019] Figure 2 This is a structural diagram of the dynamic synaesthesia integrated network system. DETAILED DESCRIPTION

[0020] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0021] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] Example 1:

[0023] like Figure 1 As shown, an embodiment of the present invention provides a node selection and beamforming design method, including:

[0024] Step S1, constructing a communication and perception model of a dynamic synaesthesia integrated network system;

[0025] Step 2: Based on the communication and perception model of the dynamic synaesthesia integrated network system, construct the communication performance, perception performance and delay performance representation of the dynamic synaesthesia integrated network;

[0026] Step S3: constructing an optimization model for node selection and beamforming of the dynamic synaesthesia integrated network based on the communication performance, perception performance, and delay performance of the dynamic synaesthesia integrated network;

[0027] Step S4: Express the optimization model as a Markov decision process and solve it using deep reinforcement learning to obtain the node selection scheme and beamforming design scheme.

[0028] As an implementation method of the present invention, in step S1, Figure 2 As shown, a dynamic synaesthesia integrated network system controlled by a central processor is constructed; A synaesthesia node equipped with MIMO antennas, each node is Communication services are provided to communication users. Multiple nodes can sense these targets while providing communication services to their own communication users through beamforming. Each node can only sense one target at a time, but can use the signals it sends and receives as well as the signals sent and received by others for sensing. Discretize time into time slots, each of which consists of Indicates that the node and its users In the time slot The communication channel is represented as , the node With the goal In the time slot The sensing channel is expressed as The perception targets in the scene do not always need to be served by perception services. The perception request of each perception target For random arrival, when When indicating the target In the time slot Need to be sensed, the sensing request is responded to by the central processor as soon as possible in the next time slot and the corresponding node selection and beamforming design are performed. Define the node selection binary variable set , Is the system in time slot The actions taken when Indicates that the time slot Node n is responsible for sensing target m. Define the multi-node beamforming matrix set ,in, It is in the time slot Time Base Station The beamforming matrix.

[0029] As an implementation method of the embodiment of the present invention, in step S2, define , represents the number of all sensing requests. Since more than the number of nodes may arrive in the same time slot Therefore, there will be perception requests that cannot be responded to immediately in the next time slot, defining the target In the time slot The latency of the perception request generated when ,in is the CPU's response time slot for the perception request, then the average delay of all perception requests can be expressed as . Communication performance is determined by the base station Users Communication rate Denotes that the perceived performance is estimated by the average parameter of the perceived request The node selection design is responsible for associating nodes with sensing targets, and the multi-node beamforming design distributes power at different angles. The two work together to meet communication and perception performance requirements and achieve a trade-off between communication and perception performance.

[0030] As an implementation method of an embodiment of the present invention, in step S3, an optimization model is used to optimize the perception performance and latency of the dynamic synaesthesia integrated network system, while satisfying the rate constraints of the communication users, the transmission power constraints of the dynamic synaesthesia integrated network system, and the latency constraints of each perception request. The optimization model is written as:

[0031] ;

[0032] in, is the weight of perceived performance, is the weight of the delay, is the maximum transmit power threshold of each node, is the threshold of the minimum communication speed of each communication user, is the maximum delay threshold for each sensing request; Is the system in time slot Actions taken when It is in the time slot Time Base Station The beamforming matrix of is its conjugate transposed matrix, is the average parameter estimate of the sensing request; is the average perceived delay, 、 and Represent the indexes of base station, target and time slot respectively, Indicates base station Users The communication rate, Indicates the target In the time slot The latency of the perception request generated when

[0033] As an implementation method of the present invention, in step S4, a Markov decision process is used for modeling and strategy learning. The Markov decision process can be composed of a four-tuple Definition, where: S represents the state space, which describes the information such as the sensing request, channel state, and delay constraint; A represents the action space, which reflects the current node selection and beamforming decision of the system; P is the state transition probability; R is the immediate reward function, which is used to quantify the impact of each decision on the overall optimization goal. , the processor observes the current system status , and select actions based on the current strategy After executing the action, the system will receive an instant reward. , and transfer to the next state according to the state transition probability , constantly updating the strategy. In order to meet the final state constraints and achieve the optimal cumulative return, the system continuously iterates to learn the optimal strategy , thus completing the approximation of the original optimization problem, specifically including:

[0034] Step S41: In each time slot, the dynamic interawareness integrated network system state includes the sensing request, the time-varying channel conditions, the node selection scheme and beamforming scheme adopted by multiple nodes in the previous time slot, that is,

[0035] ;

[0036] Step S42: The actions of the dynamic synaesthesia integrated network system include a node selection scheme and a beamforming scheme, namely

[0037] ;

[0038] Step S43: The optimization goal is to minimize the average CRB and average delay of the sensing request while satisfying the rate constraint of the communication user, the transmission power constraint of the system, and the delay constraint of each sensing request. It is used to measure the decision-making effect of the system in each time slot and is a key evaluation indicator in the reinforcement learning process. The optimization goal is transformed from the original constraint model to maximize the expected value of the reward function. This function comprehensively considers the perception task delay, transmission power overhead, and rate constraint violations, and is the main basis for guiding policy learning. Reinforcement learning gradually approaches the optimal policy by continuously interacting with the environment and adjusting the policy based on immediate rewards. Instant rewards The calculation formula is:

[0039] ;

[0040] in, is the optimization goal, is the normalized communication rate threshold constraint, is the normalized response delay threshold constraint, is the normalized transmit power constraint.

[0041] Step S44: Because the state changes in the perception network are complex and difficult to model, traditional Markov decision process methods that require state transition probabilities are no longer applicable. Therefore, a model-free Q-learning reinforcement learning method is employed to learn resource allocation and scheduling strategies in a dynamic environment with unknown state transition probabilities. This method iteratively updates the action-value function (Q-value) through interaction with the environment and uses an immediate reward function as the basis for evaluating the optimization objective, achieving a model-free solution to the original optimization problem. The resulting optimal policy effectively minimizes the perception task latency and system transmit power while satisfying rate and response constraints.

[0042] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A node selection and beamforming design method in a dynamic synaesthesia integrated network, characterized by: include: Step S1, constructing a communication and perception model of a dynamic synaesthesia integrated network system; Step 2: Based on the communication and perception model of the dynamic synaesthesia integrated network system, construct the communication performance, perception performance and delay performance representation of the dynamic synaesthesia integrated network; Step S3: constructing an optimization model for node selection and beamforming of the dynamic synaesthesia integrated network based on the communication performance, perception performance, and delay performance of the dynamic synaesthesia integrated network; Step S4: Express the optimization model as a Markov decision process and solve it using deep reinforcement learning to obtain a node selection scheme and a beamforming design scheme; The optimization model of node selection and beamforming in the dynamic synaesthesia integrated network is: Where ρ is the weight of perceived performance, (1-ρ) is the weight of delay, and P t is the maximum transmit power threshold of each node, R th is the threshold of the minimum communication speed of each communication user, τ th is the maximum delay threshold of each sensing request; a(l) is the action taken by the system at time slot l; W(l) is the set of multi-node beamforming matrices; W n (l) is the beamforming matrix of base station n at time slot l; is its conjugate transposed matrix, is the average parameter estimate of the sensing request; is the average perceived delay, n, m, and l represent the indices of the base station, target, and time slot, respectively. represents the communication rate of user c at base station n, represents the delay incurred by the sensing request generated by target m at time slot l; In step S4, the Markov decision process is used for modeling and strategy learning. The Markov decision process is defined by a four-tuple (S, A, P, R), where: S represents the state space, which describes information such as perception requests, channel states, and delay constraints; A represents the action space, which reflects the system's current node selection and beamforming decisions; P is the state transition probability; R is the immediate reward function, which is used to quantify the impact of each decision on the overall optimization goal; in each time slot l, the processor observes the current system state s(l) and selects an action a(l) according to the current strategy; after executing the action, the system obtains an immediate reward r(l) and transfers to the next state s(l+1) according to the state transition probability, and continuously updates the strategy; in order to meet the final state constraints and achieve the optimal cumulative return, the optimal strategy π is continuously learned iteratively. * (s), thus completing the approximation of the original optimization problem, specifically including: Step S41: In each time slot, the dynamic interawareness integrated network system state includes the sensing request, the time-varying channel conditions, the node selection scheme and beamforming scheme adopted by multiple nodes in the previous time slot, that is, Among them, λ m (l) represents the sensing request of target m in time slot l, Indicates the base station n and its communicating user c in time slot l n Communication channel, g n,m (l) represents the perceived channel between base station n and target m in time slot l, is the perception target selection variable at time slot l-1, when When , it means that base station n senses target m. On the contrary, when When W n (l-1) represents the beamforming matrix of base station n at time slot l-1; Step S42: The actions of the dynamic synaesthesia integrated network system include a node selection scheme and a beamforming scheme, namely Step S43: The optimization goal is to minimize the average CRB and average delay of the sensing request while satisfying the rate constraint of the communication user, the transmission power constraint of the system, and the delay constraint of each sensing request. The immediate reward function r(l) is used to measure the decision effect of the system in each time slot. The optimization goal is transformed from the original constraint model to maximize the expected value of the reward function. The calculation formula of the immediate reward r(l) in time slot l is: in, is the optimization goal, is the normalized communication rate threshold constraint, is the normalized response delay threshold constraint, is the normalized transmit power constraint; the dynamic synaesthesia integrated network system has N synaesthesia integrated nodes equipped with MIMO antennas, each node provides communication services for C communication users; B represents the number of all perception requests; Step S44: adopt a model-free Q-learning reinforcement learning method to learn resource allocation and scheduling strategies in a dynamic environment with unknown state transition probabilities; iteratively update the action value function Q-value by interacting with the environment, and use the immediate reward function as the evaluation basis for the optimization target to achieve a model-free solution to the original optimization problem; the optimal strategy finally converged can effectively minimize the perception task delay and system transmission power, while satisfying the rate and response constraints.

Images