High-dynamic communication network spectrum resource allocation method based on multi-domain information fusion

Abstract

The invention relates to a high-dynamic communication network spectrum resource allocation method based on multi-domain information fusion, and belongs to the technical field of wireless communication. Comprising the following steps: acquiring communication environment data and a historical traffic demand data set; decoupling the correlation between the communication environment data and the electromagnetic wave propagation loss by using a residual neural network to obtain an electromagnetic wave propagation loss predicted value of the next stage; predicting the historical traffic demand data set by using a long-short-term memory network to obtain a traffic demand prediction value of a next stage; fusing the electromagnetic wave propagation loss prediction value and the flow demand prediction value so as to formulate a spectrum resource allocation strategy; applying the spectrum resource allocation strategy to the railway LTE network to obtain a dynamic spectrum resource allocation strategy; and fusing the dynamic spectrum resource allocation strategy with the output information of the network state monitor, and optimizing the dynamic spectrum resource allocation strategy. According to the invention, the resource utilization rate of the wireless communication system is improved by enhancing the multi-domain information analysis capability.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and in particular to a method for allocating spectrum resources in a highly dynamic communication network based on multi-domain information fusion. Background Art

[0002] In the wireless communication system of rail transit, trains will inevitably pass through complex environments such as tunnels, bridges and high-rise buildings during operation. These environments often cause serious multipath interference, thus affecting the transmission quality of wireless signals. For high-speed railway wireless networks, mobility management is crucial, among which location management plays a core role. Location management is responsible for tracking and updating the location of mobile users on the train to ensure that the train can seamlessly maintain data transmission services during high-speed travel. In addition, spectrum resource allocation, as a key technology to support the efficient operation and high reliability of wireless communication systems, is also indispensable. Dynamic spectrum allocation dynamically allocates spectrum resources according to actual communication needs. This method can make more efficient use of spectrum resources, avoid spectrum fragmentation problems, and can be dynamically adjusted according to the load of the communication system. As a key component of spectrum resource management, it plays a vital role in improving the data transmission efficiency and service quality (QoS) of railway communications. For the reasonable allocation of spectrum resources, it is necessary to comprehensively consider a variety of heterogeneous information such as equipment operation status, network topology, environmental factors, etc., and there is a temporal correlation between these information, such as the location change of the device and the congestion of the network.

[0003] In the high-dynamic communication scenarios of railways, spectrum resources are even more scarce, which puts higher requirements on spectrum resource allocation technology in high-dynamic scenarios of railways. Most existing technologies consider modeling the spectrum resource allocation problem in high-dynamic and complex scenarios such as railways as a nonlinear programming problem to achieve the optimization goal in the corresponding specific scenarios, and solve the problem through optimization algorithms such as heuristic methods, algebraic methods, and approximate iteration methods to achieve the optimal allocation of spectrum resources. However, these technologies often lack the ability to parse multi-domain information, which limits the improvement of resource utilization of wireless communication systems. Summary of the invention

[0004] Therefore, the technical problem to be solved by the present invention is to overcome the restriction on improving the resource utilization of the wireless communication system caused by the insufficient multi-domain information parsing capability in the prior art.

[0005] In a first aspect, in order to solve the above technical problems, the present invention provides a highly dynamic communication network spectrum resource allocation method based on multi-domain information fusion, comprising:

[0006] Acquire communication environment data and historical traffic demand data sets; use a residual neural network to decouple the correlation between the communication environment data and the electromagnetic wave propagation loss to obtain a predicted value of the electromagnetic wave propagation loss in the next stage; use a long short-term memory network to predict the historical traffic demand data set to obtain a predicted value of the traffic demand in the next stage;

[0007] The electromagnetic wave propagation loss prediction value is integrated with the traffic demand prediction value to form an input information set; a spectrum resource allocation strategy is formulated according to the input information set; the spectrum resource allocation strategy is applied to the railway LTE network, and the spectrum resource allocation strategy is dynamically adjusted to obtain a dynamic spectrum resource allocation strategy;

[0008] The dynamic spectrum resource allocation strategy is integrated with the output information of the network status monitor to optimize the dynamic spectrum resource allocation strategy; wherein the output information includes scenario domain information and service domain information; in the process of optimizing the dynamic spectrum resource allocation strategy, a deep deterministic policy gradient algorithm is used to select a certain continuous action in a continuous action space, and the continuous action is mapped into a discrete action that can perform resource scheduling in the communication environment.

[0009] In one embodiment of the present invention, the process of applying the spectrum resource allocation strategy to the railway LTE network includes designing a multi-domain information fusion strategy based on a weighted fusion model. The method of designing a multi-domain information fusion strategy based on a weighted fusion model is:

[0010] Determine the information weight of each domain according to the influence of each domain information on spectrum resource allocation;

[0011] The information weights of the various domains are integrated to obtain a real-time network benefit function.

[0012] In one embodiment of the present invention, the calculation formula of the network real-time revenue function is:

[0013]

[0014] in, is the network real-time benefit, α and β are the network's attention to transmission efficiency and real-time performance, respectively. is the transmission efficiency benefit function, is the real-time performance benefit function, UE g (t) is all users, g is the base station, G is the base station set, u is the user, p and q are different coordination factors.

[0015] In one embodiment of the present invention, the calculation formulas of the transmission efficiency benefit function and the real-time performance benefit function are respectively:

[0016]

[0017] in, represents the real-time transmission efficiency benefit of the service of user u belonging to base station g, represents the historical queue transmission efficiency benefit of user u belonging to base station g, represents the historical average transmission efficiency benefit of the service of user u belonging to base station g, is the service delay budget of user u belonging to base station g, D g,u (t) is the queue delay of user u belonging to base station g at time t.

[0018] In one embodiment of the present invention, the action space needs to satisfy a constraint, and the constraint is:

[0019]

[0020] Among them, g is the base station, G is the base station set, N is the total number of base stations in the network, u is the user, V is the total number of resource blocks, BW g is the bandwidth limit of the base station cell in the railway LTE network. To meet the remaining resource block set of the base station cell in the railway LTE network after the private network service, UE g (t) for all users, a g,u,r (t)∈{0,1}, 0 means no allocation, 1 means allocation, r is the time-frequency resource block, b RB is the standard bandwidth, and j is any resource block number in the time-frequency resource block.

[0021] In one embodiment of the present invention, the process of optimizing the dynamic spectrum resource allocation strategy includes updating the main network and the target network; the main network includes a policy network and an evaluation network, and the policy network searches for an action that can maximize the value of the evaluation network; the target network includes a target policy network and a target evaluation network.

[0022] In one embodiment of the present invention, the calculation formula for the number of input nodes of the policy network is:

[0023] N input =N·M·(2V+4);

[0024] Among them, N input is the number of input nodes, N is the total number of base stations in the network, M is the number of users in the corresponding cell of each base station, and V is the number of resource blocks in the corresponding cell of each base station;

[0025] The calculation formula for the number of output nodes of the evaluation network is:

[0026] N output =N·M·V;

[0027] Among them, N output is the number of output nodes.

[0028] In a second aspect, in order to solve the above technical problems, the present invention provides a highly dynamic communication network spectrum resource allocation system based on multi-domain information fusion, comprising:

[0029] The scenario domain cognition module and the business domain cognition module are used to respectively receive the scenario domain information and the business domain information output by the network status monitor, and obtain the predicted scenario domain information and the predicted business domain information of the next stage;

[0030] A multi-domain information fusion module, used to fuse the predicted scene domain information and the predicted business domain information through a multi-domain information fusion mechanism to obtain fused information, and calculate the real-time network revenue value through the fused information;

[0031] An intelligent agent is used to receive the real-time benefit value of the network and optimize the dynamic spectrum resource allocation according to the real-time benefit value of the network; wherein the intelligent agent includes at least two main networks and a target network.

[0032] In one embodiment of the present invention, the agent's reward function is obtained by calculating the real-time benefit value of the network.

[0033] In a second aspect, in order to solve the above technical problem, the present invention provides a computer program product, which includes a computer program. When the computer program is run, the above method is executed.

[0034] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0035] (1) The method for allocating spectrum resources in a highly dynamic communication network based on multi-domain information fusion described in the present invention integrates the predicted values ​​of electromagnetic wave propagation loss and traffic demand to construct a comprehensive input information set, providing rich contextual information for the decision-making process. By deeply analyzing this contextual information, the present invention can integrate data from different fields, accurately understand and predict the behavior of the wireless communication system, and achieve efficient allocation of spectrum resources. By using a long short-term memory network to conduct in-depth analysis and prediction of historical traffic demand data, the method not only improves the accuracy of the prediction, but also optimizes resource allocation, thereby enhancing network performance. In addition, the present invention converts continuous actions into discrete actions, ensuring the feasibility of resource scheduling in an actual communication environment. The present invention solves the problem that the traditional spectrum resource allocation method results in low resource utilization of the wireless communication system due to incomplete information consideration, and effectively avoids the limitation on improving the resource utilization of the wireless communication system due to insufficient multi-domain information analysis capabilities.

[0036] (2) The present invention uses intelligent agents to continuously interact with the communication environment to obtain and predict multi-domain network information; combined with the multi-domain information fusion mechanism, it continuously updates the long-term cumulative reward return for optimizing network performance and obtains the optimal allocation matrix of time-frequency resource blocks in the network. The wireless communication network achieves optimal allocation of spectrum resources through the optimal allocation matrix, effectively improving its own resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein

[0038] Figure 1 It is a flow chart of a method for allocating spectrum resources in a highly dynamic communication network based on multi-domain information fusion in a preferred embodiment of the present invention;

[0039] Figure 2 A schematic diagram of the residual neural network structure in a preferred embodiment of the present invention;

[0040] Figure 3 A schematic diagram of a convolution process in a preferred embodiment of the present invention;

[0041] Figure 4 A schematic diagram of the residual block unit structure in a preferred embodiment of the present invention;

[0042] Figure 5 This is a schematic diagram of the internal structure of a unit of an LSTM network in a preferred embodiment of the present invention;

[0043] Figure 6 A schematic diagram of a highly dynamic communication network spectrum resource allocation method based on multi-domain information fusion in a preferred embodiment of the present invention;

[0044] Figure 7 A schematic diagram of a strategy network structure in a preferred embodiment of the present invention;

[0045] Figure 8 This is a schematic diagram of the evaluation network structure in a preferred embodiment of the present invention;

[0046] Fig. 9 A schematic diagram of a high dynamic wireless communication network scenario for simulation in a preferred embodiment of the present invention;

[0047] Fig.10 A curve diagram of the reward value of the method for different numbers of users in a rural line scenario in a preferred embodiment of the present invention;

[0048] Fig.11 A reward value curve diagram of the method for different numbers of users in a high-rise building dense area scenario in a preferred embodiment of the present invention;

[0049] Fig.12The statistical results of average throughput for different numbers of users in a rural line scenario in a preferred embodiment of the present invention;

[0050] Fig.13 The statistical results of average throughput for different numbers of users in a high-rise building dense area scenario in a preferred embodiment of the present invention;

[0051] Fig.14 The statistical results of average delays for different numbers of users in the rural line scenario in the preferred embodiment of the present invention;

[0052] Fig.15 The statistical results of average delays for different numbers of users in a high-rise building dense area scenario in a preferred embodiment of the present invention;

[0053] Fig.16 The statistical results of average packet loss rate for different numbers of users in the rural line scenario in the preferred embodiment of the present invention;

[0054] Fig.17 This is the statistical result of average packet loss rate for different numbers of users in a high-rise building dense area scenario in the preferred embodiment of the present invention. DETAILED DESCRIPTION

[0055] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0056] Embodiment 1

[0057] Reference Figure 1 As shown, an embodiment of the present invention provides a highly dynamic communication network spectrum resource allocation method based on multi-domain information fusion, including:

[0058] Acquire communication environment data and historical traffic demand data sets; use residual neural network to decouple the correlation between communication environment data and electromagnetic wave propagation loss, and obtain the electromagnetic wave propagation loss prediction value for the next stage; use long short-term memory network to predict the historical traffic demand data set and obtain the traffic demand prediction value for the next stage;

[0059] The predicted value of electromagnetic wave propagation loss is integrated with the predicted value of traffic demand to form an input information set; a spectrum resource allocation strategy is formulated based on the input information set; the spectrum resource allocation strategy is applied to the railway LTE network, and the spectrum resource allocation strategy is dynamically adjusted to obtain a dynamic spectrum resource allocation strategy;

[0060] The dynamic spectrum resource allocation strategy is integrated with the output information of the network status monitor to optimize the dynamic spectrum resource allocation strategy; the output information includes scenario domain information and service domain information; in the process of optimizing the dynamic spectrum resource allocation strategy, a deep deterministic policy gradient algorithm is used to select a certain continuous action in the continuous action space, and the continuous action is mapped into a discrete action that can be used for resource scheduling in the communication environment.

[0061] The embodiment of the present invention provides a spectrum resource allocation method for a highly dynamic communication network based on multi-domain information fusion, which integrates the electromagnetic wave propagation loss prediction value with the traffic demand prediction value to form a more comprehensive input information set, providing richer context information for decision-making. By analyzing the multi-domain information of the wireless communication system by analyzing the context information, the embodiment of the present invention can combine information from different domains to accurately understand and predict the behavior of the wireless communication system, thereby achieving more efficient allocation of spectrum resources. The use of a long short-term memory network to predict the historical traffic demand data set can not only improve the prediction accuracy, but also optimize resource allocation and enhance network performance. In addition, the embodiment of the present invention maps continuous actions to discrete actions, ensuring the feasibility of resource scheduling in an actual communication environment. At the same time, the combination of scenario domain information and service domain information provides a broader perspective for network management and optimization. This method effectively overcomes the limitation of insufficient multi-domain information parsing capabilities on improving the resource utilization of wireless communication systems, and provides an efficient and economical solution for spectrum resource allocation in highly dynamic communication networks.

[0062] Specifically, a residual neural network is used to decouple the correlation between these parameters and the electromagnetic wave propagation loss in the communication scenario, so as to predict the future electromagnetic wave propagation loss value. Figure 2 , the residual neural network structure includes an input layer, a convolutional layer, a residual block, a fully connected layer and an output layer. The embodiment of the present invention assumes that there are q parameters related to electromagnetic wave propagation loss in the communication scenario, which are X = x1, x2, ..., x q , the true value of electromagnetic wave propagation loss is expressed as PL real .

[0063] In terms of the input layer, the input layer will input data X = x1, x2, ..., x q Mapping to high dimensions facilitates the extraction of data features by subsequent network modules. The calculation formula is:

[0064]

[0065] Among them, H j is the input layer node output, w ij is the input layer node weight, is the input layer node bias, f(·) is the activation function, and the calculation formula is:

[0066] f(δ)=max(0,δ); (2)

[0067] Among them, δ is the input of the activation function f(·).

[0068] In terms of convolutional layers, convolutional layers are the core components of convolutional neural networks. The operation of the convolution kernel on the feature map is equivalent to a sliding window, which slides back and forth on the entire feature map with a certain step size to achieve feature extraction. The convolution process can be referred to Figure 3 The operation process of the convolutional layer is expressed mathematically as follows:

[0069]

[0070] in, is the result of the convolution kernel θ at position [i, j] of the input matrix H, is the output feature map.

[0071] After the convolution kernel completes the convolution of the entire feature map, the size of the output feature map is calculated as follows:

[0072]

[0073] Among them, h in × in is the input data size, h out × out is the output data size, h kernel × kernel is the convolution kernel size, h stride × stride It is the horizontal and vertical step size of the convolution kernel movement, and padding is the edge padding unit.

[0074] In terms of residual blocks, residual blocks solve the problem of gradient vanishing and gradient exploding by using cross-layer skip connections, which can improve the convergence speed and performance of the network. The unit structure diagram can be found in Figure 4 . Figure 4 where φ is the input matrix of the residual block unit and F(·) is the residual function.

[0075] In terms of the fully connected layer, the fully connected layer maps the features learned by the convolutional layer, pooling layer, and residual block to the sample space. The calculation formula is:

[0076]

[0077] Among them, s j is the output layer input, O k is the output layer node output, μ jk is the node weight of the output layer, is the output layer node bias.

[0078] In terms of the output layer, the output layer activates the output of the fully connected layer to increase the linear capacity of the network. The calculation formula of the activation function is the same as formula (2).

[0079] Furthermore, a long short-term memory network (LSTM) is used to predict the future real-time demand for user service traffic in the network through the user service historical traffic data set. In the embodiment of the present invention, it is assumed that the normalized historical traffic data set in the wireless communication network is R = (r1, r2, r3, ..., r T ,). Among them, r t is the network user traffic demand in time slot t. The prediction model predicts the network user traffic demand in the next time slot based on the user traffic demand in the previous 20 time slots. The internal structure of a typical LSTM unit can be referred to Figure 5 Compared with the traditional residual neural network (RNN) structure, it adds a forget gate, input gate and output gate to control the flow of historical information. The specific calculation formula is:

[0080] f t =σ(W f [H t-1 ,x t ]+b f ); (7)

[0081] i t =σ(W i [H t-1 ,x t ]+b i ); (8)

[0082] o t =σ(W o [H t-1 ,x t ]+b o ); (9)

[0083]

[0084] H t =o t *tanh C t ; (11)

[0085]

[0086] Among them, f t ,i t , o t are the outputs of the forget gate, input gate, and output gate respectively, σ(x)=(1+e -x ) -1is the activation function of the control gate output. According to the characteristics of the activation function σ, these three gates control the influence of historical information at time t. t is the state of the LSTM unit at time t, is the candidate memory cell state, H t is the output of the LSTM unit at time t, tanh(x)=(e x -e -x )·(e x +e -x ) -1 is the activation function that controls the input and output. f 、b i 、b o 、b C Represent different bias terms. f , W i , W o , W C They represent different weight matrices. t Represents the input item. Through the interaction between the three gates, LSTM can better learn the timing information.

[0087] Specifically, the resource block rate descriptor and resource block allocation descriptor are used to describe the detailed allocation relationship of wireless resources in the time and frequency domain. The resource block rate descriptor is a g×u×r dimensional matrix, which describes the rate relationship between different users, services and resource blocks at a certain point in time. This descriptor can help the embodiments of the present invention better understand the resource quality in the wireless communication network. Its specific mathematical form is:

[0088]

[0089] in, represents the resource block rate descriptor at time point t, R g,u,r (t) represents the maximum transmission rate that can be achieved when the time-frequency resource block r belonging to the base station g is allocated to the user u at the time point t.

[0090] The resource block allocation descriptor describes the allocation relationship between different users, services and resource blocks at a certain point in time. This descriptor can help the embodiments of the present invention better understand the resource allocation situation in the wireless communication network and how to optimize the resource allocation according to the actual situation. Similarly, a g×u×r dimensional matrix is ​​used to represent the mathematical form of the resource block allocation descriptor, and its specific mathematical form is:

[0091]

[0092] in, represents the resource block allocation descriptor at time point t, a g,u,r(t) indicates whether the time-frequency resource block r belonging to base station g is allocated to user u at time point t, a g,u,r (t)∈{0,1}, 0 means no allocation, 1 means allocation.

[0093] Furthermore, in order to achieve dynamic management and optimization of spectrum resources, avoid waste and idleness of spectrum resources, and improve the utilization efficiency of spectrum resources, the embodiment of the present invention analyzes the intelligent adaptation of spectrum resources in the railway LTE (LTE is Long Term Evolution, railway LTE can be abbreviated as LTE-R) homogeneous network.

[0094] Specifically, consider a discrete time slot LTE-R network in a specific communication scenario, where T is the total number of time slots. The set of base stations in the network is represented as:

[0095] g∈G={1,2,…,N}; (15)

[0096] Among them, G is the base station set, and N is the total number of base stations in the network.

[0097] At time t, all users in the cell under base station g are expressed as:

[0098] UE g (t) = {1, 2, ..., M}; (16)

[0099] Wherein, M is the number of users in the corresponding cell of each base station.

[0100] At time t, the resource block set of the cell under base station g is expressed as:

[0101]

[0102] in, It is the set of remaining resource blocks of the base station g cell in the LTE-R network after satisfying the private network service, and V is the number of resource blocks of the corresponding cell under each base station.

[0103] Further, the uniqueness of resource block allocation is analyzed, and the resource block allocation descriptor The following formula should be satisfied:

[0104]

[0105] Furthermore, the bandwidth upper limit BW of the cell within the LTE-R network base station g is analyzed. g , The following formula should be satisfied:

[0106]

[0107] Furthermore, analyzing the technical limitations of LTE-R network such as modulation and demodulation, carrier aggregation, etc., resource blocks can only be allocated continuously in the frequency domain. The following formula should be satisfied:

[0108]

[0109] Wherein, j is any resource block number in the time-frequency resource block. The specific meaning of formula (20) is that when user u is not allocated resources in resource block j of the cell within base station g, then no resources can be allocated in resource blocks with higher numbers (r≥j+1).

[0110] For LTE-R homogeneous networks, the time-frequency resource blocks have a standard bandwidth b RB , and can adaptively adjust its own transmission capacity according to the channel quality. In each TTI, the LTE-R resource scheduler allocates all available resource blocks in the network that are not allocated to private network services to users accessing each cell base station. The embodiment of the present invention assumes that the minimum scheduling period TTI is 1ms, and it is necessary to solve the problem of solving the optimal allocation descriptor of resource domain resource blocks in the entire LTE-R network.

[0111] Multi-Domain Information Fusion (MDIF) is a unique technology for comprehensive diagnosis and evaluation of networks or systems. It integrates information from various fields and comprehensively evaluates the object under test according to specific standards, thereby obtaining a unified prediction and evaluation. Compared with single-domain information processing technology, multi-domain information fusion can analyze the experimental object from multiple levels, multiple aspects, and multiple angles. The embodiment of the present invention adopts a weight-based fusion model to design a multi-domain information fusion strategy, and fuses the information according to the weight of each domain.

[0112] Specifically, the information weight is determined according to the degree of influence of the information in each domain on the allocation of spectrum resources. In this way, a comprehensive information can be obtained, namely the network real-time benefit function Used to guide the allocation of spectrum resources. The specific calculation formula is:

[0113]

[0114] Among them, α and β are two weight parameters introduced, which represent the network's attention to transmission efficiency and real-time performance respectively; is the transmission efficiency benefit function, which represents the transmission efficiency benefit generated by the base station g performing a resource block scheduling at time t; is a real-time performance benefit function, which represents the real-time performance benefit generated by base station g performing a resource block scheduling at time t; p and q are different coordination factors. In the embodiment of the present invention, the values ​​of the two coordination factors are preferably: p=3, q=1.

[0115] Specifically, the transmission efficiency benefit function The specific calculation formula is:

[0116]

[0117] in, represents the real-time transmission efficiency benefit of the service of user u belonging to base station g, represents the historical queue transmission efficiency benefit of user u belonging to base station g, Represents the historical average transmission efficiency benefit of the service of user u belonging to base station g.

[0118] Further, It has the greatest impact on the user's online transmission experience. It comprehensively considers the scene domain, business domain, and resource domain information. The calculation formula is:

[0119]

[0120] in, is the upper limit of the real-time transmission rate of the service of user u belonging to base station g, is the lower bound of the real-time transmission rate of the service of user u belonging to base station g, R g,u (t) is the real-time transmission rate, R g,u The calculation formula of (t) is:

[0121]

[0122] Further, The reliability of user transmission has the greatest impact, taking into account the scenario domain, business domain and resource domain information. The calculation formula is:

[0123]

[0124] Among them, Q g,i (t) is the queue length of user u belonging to base station g at time t, and TTI is the transmission time interval.

[0125] Further, It has the greatest impact on fairness among users, mainly considering the business domain information, and its calculation formula is:

[0126]

[0127] in, is the expected average historical transmission rate of user u belonging to base station g, is the historical transmission rate of user u belonging to base station g at time t, and The calculation formula is:

[0128]

[0129] Among them, L g,u is the service data packet size of user u belonging to base station g, For L g,u The probability density distribution function of g,u is the service data packet grouping interval of user u belonging to base station g, T g,u The probability density distribution function of g,u (τ) is the transmission data size of user u belonging to base station g at time τ; is an adjustable constant, and its calculation formula is:

[0130]

[0131] Among them, E(T g,u ) is the average data packet grouping interval of the service of user u belonging to base station g.

[0132] Specifically, the real-time performance benefit function The business domain information is mainly considered, and the calculation formula is as follows:

[0133]

[0134] in, is the service delay budget of user u belonging to base station g, D g,u (t) is the queue delay of user u belonging to base station g at time t.

[0135] The embodiment of the present invention proposes a method for intelligently allocating spectrum resources in a high-speed railway mobile communication network based on multi-domain information reinforcement learning. The method framework diagram can be referred to Figure 6 . Figure 6 It consists of two parts: a highly dynamic communication environment based on multi-domain information fusion and an intelligent agent. In a highly dynamic communication environment based on multi-domain information fusion, the scene domain cognitive model and the business domain cognitive model accept the scene domain information and business domain information output by the network status monitor to predict future information. The multi-domain information fusion mechanism calculates the real-time network benefit value r by fusing information. t , and feed it back to the agent to improve performance. The agent has two main networks and two target networks. The main network contains a policy network μ and an evaluation network Q, and the network parameters are θ μ and θ QThe goal of the strategy network is to find the action that can maximize the Q value of the evaluation network, and guide the high-dynamic communication network to achieve intelligent allocation of time-frequency resource blocks through this action. The target network consists of the target strategy network μ′ and the target evaluation network Q′, and the network parameters are θ μ′ and θ Q′ Among them, the structures of the main network and the target network are exactly the same, but the frequency of network parameter update is different. The main network is updated once per step, and the target network is updated once after the main network is updated multiple times.

[0136] The considered state includes the current user's transmission capacity information on each time-frequency resource block in the base station, the user queue length information, the user's historical transmission rate information, the user queue delay information, the user's traffic demand information at the next moment, and the user's transmission capacity information on each time-frequency resource block in the base station. t The mathematical form is:

[0137]

[0138] Among them, R max The time-frequency resource block transmission capability of the user under the optimal channel conditions, is the historical maximum queue length of user M belonging to base station N, is the service flow prediction value of user M belonging to base station N.

[0139] Specifically, the embodiment of the present invention utilizes the characteristics of the deep deterministic policy gradient algorithm to select a certain action in the continuous action space, rather than calculating the specific probabilities of all possible actions. Then, this continuous action is mapped to a discrete action that can be used for resource scheduling in the communication environment. This method utilizes the continuous decision-making characteristics of the deep deterministic policy gradient algorithm, thereby effectively solving the problem of too large an action space. At time t, action a t The mathematical form is:

[0140]

[0141] Among them, δ N,M,V The value of (t) indicates whether user M belonging to base station N occupies its Vth time-frequency resource block, and its value range is {0,1}; 1 indicates occupation, and 0 indicates non-occupancy. It should be noted that the action space should satisfy the constraints (18)(19)(20) mentioned above.

[0142] Furthermore, the network real-time revenue function calculated by formula (21) is As a reward function, the reward function is defined as:

[0143]

[0144] Furthermore, the design of the strategy (also referred to as actor) network and the evaluation (also referred to as critic) network is based on deep neural network technology, aiming to achieve the optimization of the strategy and the evaluation of the decision-making process. The structure of the actor network and the critic network can be referred to Figure 7 and Figure 8 .

[0145] For the actor network, a fully connected network architecture is adopted, which takes the environment state s as input and outputs the corresponding action a. According to the definition of the state space in formula (31), the number of nodes in the input layer of the actor network can be determined, and the mathematical expression is:

[0146] N input =N·M·(2V+4); (34)

[0147] According to the definition of action space in formula (32), the number of output nodes of the actor network can be determined as:

[0148] N output =N·M·V; (35)

[0149] Reference Figure 7 The actor network has four hidden layers: h1, h2, h3, and h4. The number of nodes is 500, 1000, 1000, and 500, respectively. The ReLu activation function is preferred between all layers except the output layer. The calculation formula of the ReLu activation function is the same as that of formula (2). The Sigmoid function is preferred for the output layer, and its calculation formula is:

[0150]

[0151] For the critic network, the critic network is also a fully connected network. The input is a combination of the environment state s and the action a. The output is the Q value of the state-action pair. The number of nodes in the input layer of the critic network is equal to

[0152] N input =N·M·(3V+4); (37)

[0153] Furthermore, the number of output nodes of the critic network is

[0154] N output =1; (38)

[0155] Reference Figure 8 ,The critic network has two hidden layers, h1 and h2, with 128 and 64 nodes respectively. ReLu activation function is used between each layer except the output layer, and Sigmoid function is used in the output layer.

[0156] In the process of finding the optimal spectrum resource intelligent allocation strategy, the agent selects action a from the strategy network μ. t =μ(s t )+n. Where n is the action exploration noise to avoid local optimality, and n~N(0,σ). After the communication network executes this action, a new state s will be generated. t+1 and rewards t , and the four-tuple sample (s t ,a t ,r t ,s t+1 ) into the experience replay pool. When the experience replay pool is full, Z sample data are taken out from it and input into the policy network and evaluation network for training and updating of network parameters.

[0157] Specifically, the network parameters θ of the critic network Q are Q The gradient formula is:

[0158]

[0159] Among them, the target value y t The calculation formula is:

[0160] y t =r t +γQ′(s t+1 ,a t+1 ); (40)

[0161] Among them, γ is the discount factor.

[0162] Specifically, the network parameters θ of the actor network μ are μ The gradient formula is as follows:

[0163]

[0164] in, is the action gradient of the evaluation network, is the policy gradient of the policy network.

[0165] Furthermore, the target evaluation network and the target strategy network use a soft update method to update their own network parameters. The specific update formula is as follows:

[0166] θ Q′ =δθ Q +(1-δ)θ Q′ ; (42)

[0167] θ μ′ =δθ μ +(1-δ)θ μ′ ; (43)Wherein, δ is the soft update coefficient.

[0168] At this point, the iterative gradient descent method can be used to obtain the target evaluation network θ that meets the conditions. Q′ and the target policy network θ μ′ .

[0169] In view of the problem that the existing spectrum resource allocation method has low resource utilization of wireless communication systems due to incomplete information consideration, the embodiment of the present invention proposes a highly dynamic communication network spectrum resource allocation method based on multi-domain information fusion. The method integrates multi-domain information technology and combines it with a reinforcement learning algorithm to realize the intelligent allocation of spectrum resources. The intelligent agent collects and predicts the multi-domain information of the network through continuous interaction with the communication environment; the multi-domain information fusion mechanism is used to dynamically update the long-term cumulative reward for optimizing network performance, thereby obtaining the optimal allocation matrix of time-frequency resource blocks in the network. Based on this optimal allocation matrix, the wireless communication network can realize efficient allocation of spectrum resources, significantly improve resource utilization, and optimize network service quality. In addition, the embodiment of the present invention accurately captures information based on contextual connections, which has a significant effect on the optimization of rail transit communication network performance. In-depth analysis of contextual information and fusion analysis of multi-source data are particularly critical, which provides strategic guidance for the continuous optimization of rail transit network performance.

[0170] In order to more clearly explain the specific content of the embodiments of the present invention, an example description is given below.

[0171] Example: A high-dynamic wireless communication network simulation platform is built based on the NS3 simulation platform. Users are randomly distributed in the base station coverage area, and each base station has a certain overlapping coverage area. The user's communication service is generated according to the relevant service feature information in Table 1. Considering the network load, this example gradually increases the number of services from 6 to 72, and sets two different service ratios, namely service ratio 1 (Number 2) and service ratio 2 (Number 3). Volte :Num Video :Num URLLC =2:3:1) and business ratio 2 (Num Volte :Num Video :Num URLLC =1:1:1)(where Num Volte 、Num Video 、Num URLLC are the number of Volte services, Video services, and URLLC services, respectively, to simulate different service demand distributions. Considering the scenario domain information, this example conducts experiments in two scenarios: a high-rise building-dense area and a rural area. The specific scenarios are as follows: Fig. 9 shown.

[0172] Table 1 Wireless communication service characteristics

[0173]

[0174] In the method described in this example, the design of the reward function is crucial. The reward function defines the goal of the agent, that is, the agent learns how to take actions to maximize its cumulative reward. In highly dynamic scenarios, the reward function is designed to reflect network performance, including indicators such as throughput and latency.

[0175] On the network simulation platform set up in the example of the present invention, simulation experiments are conducted to compare the changes in reward values ​​of intelligent agents under different network load conditions and different scenarios. Fig.10 and Fig.11 The reward value change curves when training the proposed method are respectively in the following four cases: business ratio 1 and business ratio 2 in the rural line scenario, and business ratio 1 and business ratio 2 in the high-rise building dense area scenario. In the figure, when the number of users is 6, 12, 18, 24, 48 and 72, the reward values ​​obtained by the proposed method can eventually stabilize, indicating that the method can reach a convergence state. It can be seen from the training curve that the reward value decreases with the increase in the number of users. This is because when the network load is low, the agent can obtain higher rewards through simple strategies, such as uniform resource allocation. However, as the network load increases, this strategy becomes ineffective. The agent needs to learn more complex strategies to obtain higher rewards. In addition, the reward obtained by the agent in the case of business ratio 1 is slightly higher than that in the case of business ratio 2. This is because the amount of data generated by the video streaming service is much smaller than that of the URLLC service. When the network load is too high, even if the agent adopts the optimal resource allocation strategy, it cannot meet the needs of all users due to the physical limitations of resources, resulting in a decrease in network performance. In addition, the reward obtained by the agent in the high-rise building dense area scenario is slightly higher than that in the rural line scenario. This is because the communication environment in rural areas is more complex than in areas with densely populated high-rise buildings, and user mobility is higher. The intelligent agent needs to make resource allocation decisions more frequently, which places higher demands on the learning ability of the intelligent agent.

[0176] Combined with the above analysis, it can be seen that the agent can find effective resource allocation strategies through learning in different scenarios and network load conditions. Although the performance of the agent may slightly decrease in some cases, such as rural scenarios or when the network load is too high, the agent can still maintain a high performance overall.

[0177] The trained proposed method is compared with the maximum carrier-to-interference ratio method, proportional fairness algorithm, polling algorithm and channel sensing algorithm. This example respectively counts the average throughput, average delay and average packet loss rate of each algorithm under different user conditions, such as Figures 12 to 17 shown.

[0178] Fig.12 and Fig.13The following are the statistical results of the average throughput of Volte services, Video services and URLLC services of each algorithm in the rural line scenario and the high-rise building dense area scenario.

[0179] from Fig.12 and Fig.13 It can be seen that compared with the four algorithms of maximum carrier-to-interference ratio, proportional fairness, polling and channel perception, the proposed method has brought certain improvements to the average throughput of each service. It can be seen that compared with the other four algorithms, the proposed method has better guarantee for the average throughput performance of URLLC services while maintaining the expected average throughput of Volte services and Video services. This shows that the proposed method can effectively improve the average throughput of the system compared with the traditional classical method. With the increase in the number of users, the ability of the proposed method to improve the average throughput compared with the traditional classical method is increasingly reduced. This is because under high load conditions, the demand relationship of each user is more complex, and the approximate optimal allocation method given by the intelligent agent can only meet the communication needs of each user as much as possible, and it is difficult to find an accurate numerical allocation matrix. Similarly, compared with the case of service ratio 2, the improvement of the average throughput of the network by the intelligent agent in the case of service ratio 1 is also more obvious. In addition, although the reward value obtained by the intelligent agent in the rural line scenario is slightly lower than that in the high-rise dense area scenario, the spectrum allocation strategy given by it improves the average throughput more than the high-rise dense area scenario. This also shows that the intelligent agent has strong adaptability to the scene.

[0180] Fig.14 and Fig.15 The following are the statistical results of the average delay of Volte services, Video services and URLLC services of each algorithm in the rural line scenario and the high-rise building dense area scenario.

[0181] from Fig.14 and Fig.15It can be seen that compared with the four algorithms of maximum carrier-to-interference ratio, proportional fairness, polling and channel sensing, the proposed method has brought certain improvements to the average delay of each service. It can be seen that the proposed method is better than the other four algorithms in dealing with average delay, regardless of the number of users. This shows that the proposed method has significant advantages over traditional classical algorithms in reducing the average delay of the system. Moreover, the degree of optimization of the average delay in the rural line scenario by the proposed method is higher than that in the high-rise building dense area scenario. However, it is worth noting that with the increase in the number of users, the advantage of the proposed method in reducing the average delay gradually weakens. This may be attributed to the increased complexity of user requirements in a high-load network environment, which leads to the fact that the intelligent agent can only provide an approximately optimal resource allocation solution, but cannot find an accurate numerical allocation matrix. In addition, in the case of service ratio one, compared with service ratio two, the intelligent agent performs more prominently in reducing the average delay of the network. This result further confirms that when service demand is more concentrated, the intelligent agent can more effectively optimize network performance and reduce average delay.

[0182] Fig.16 and Fig.17 The following are the statistical results of the average packet loss rate of Volte service, Video service and URLLC service of each algorithm in the rural line scenario and the high-rise building dense area scenario.

[0183] from Fig.16 and Fig.17 It can be seen that compared with the four algorithms of maximum carrier-to-interference ratio, proportional fairness, polling and channel perception, the proposed method has brought certain improvements to the average packet loss rate of each service. Obviously, compared with the other four algorithms, the proposed method has good performance in the average packet loss rate of the three types of services under the condition of each number of users. This shows that the proposed method has more advantages than the traditional classical algorithm in reducing the average packet loss rate of the system. Similarly, the proposed method optimizes the average delay in the rural line scenario more than the high-rise building dense area scenario. However, with the increase in the number of users, the advantage of the proposed method in reducing the average packet loss rate gradually weakens. In addition, compared with the service ratio 2, the service ratio 1 agent performs more prominently in reducing the average packet loss rate of the network. This further proves that when the service demand is more concentrated, the agent can more effectively optimize the network performance and reduce the average packet loss rate.

[0184] comprehensive Figures 12 to 17It can be seen that the time-frequency resource block allocation strategy obtained by the method proposed in the present invention will significantly improve the system performance of the high-dynamic wireless communication system compared with the other four types of algorithms. This is mainly because the method proposed in the embodiment of the present invention takes into account the joint information of the scene domain, service domain and resource domain, which not only takes into account the current scene and number of services, but also takes into account the available resources. In this way, the method can realize the intelligent optimization allocation of spectrum resources under various scenes and numbers of services. In addition, the method is also adaptive. The intelligent agent can dynamically adjust its behavior according to different scenes, different numbers of users and different business distribution states to meet the needs of each user, that is, the intelligent agent can adjust its own behavior according to the real-time state of the network to optimize network performance.

[0185] Embodiment 2

[0186] Based on the same inventive concept, this embodiment provides a high-dynamic communication network spectrum resource allocation system based on multi-domain information fusion. The principle of solving the problem is similar to the high-dynamic communication network spectrum resource allocation method based on multi-domain information fusion provided in Example 1, and the repeated parts will not be repeated.

[0187] This embodiment provides a highly dynamic communication network spectrum resource allocation system based on multi-domain information fusion, including:

[0188] The scenario domain cognition module and the business domain cognition module are used to respectively receive the scenario domain information and the business domain information output by the network status monitor, and obtain the predicted scenario domain information and the predicted business domain information of the next stage;

[0189] A multi-domain information fusion module is used to fuse the predicted scene domain information and the predicted business domain information through a multi-domain information fusion mechanism to obtain fused information, and calculate the real-time network revenue value through the fused information;

[0190] The intelligent agent is used to receive the real-time network benefit value and optimize the dynamic spectrum resource allocation according to the real-time network benefit value; wherein the intelligent agent includes at least two main networks and a target network.

[0191] Specifically, the reward function of the intelligent agent is obtained by calculating the real-time benefit value of the network. The calculation formula of the real-time benefit value of the network can refer to formula (21) in Example 1.

[0192] In practical applications, the highly dynamic communication network spectrum resource allocation system based on multi-domain information fusion provided by the embodiment of the present invention is applicable to high-speed railway wireless communication networks. When a train is running on the track, the system can collect key parameters of the communication environment in real time through sensors. These parameters include the precise location of the train, the installation height of the antenna, the specific scene type, and the communication frequency band currently in use. Through the real-time collection and analysis of these data, the system can more effectively manage and optimize the spectrum resources in the high-speed railway wireless communication network, thereby improving communication efficiency and quality.

[0193] Embodiment 3

[0194] This embodiment provides a computer program product, which includes a computer program. When the computer program is run, the method described in the first embodiment is executed.

[0195] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0196] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0197] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0198] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0199] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims

1. A highly dynamic communication network spectrum resource allocation method based on multi-domain information fusion, characterized in that ,include: Acquire communication environment data and historical traffic demand data sets; use a residual neural network to decouple the correlation between the communication environment data and the electromagnetic wave propagation loss to obtain a predicted value of the electromagnetic wave propagation loss in the next stage; use a long short-term memory network to predict the historical traffic demand data set to obtain a predicted value of the traffic demand in the next stage; The electromagnetic wave propagation loss prediction value is integrated with the traffic demand prediction value to form an input information set; a spectrum resource allocation strategy is formulated according to the input information set; the spectrum resource allocation strategy is applied to the railway LTE network, and the spectrum resource allocation strategy is dynamically adjusted to obtain a dynamic spectrum resource allocation strategy; The dynamic spectrum resource allocation strategy is integrated with the output information of the network status monitor to optimize the dynamic spectrum resource allocation strategy; wherein the output information includes scenario domain information and service domain information; in the process of optimizing the dynamic spectrum resource allocation strategy, a deep deterministic policy gradient algorithm is used to select a certain continuous action in a continuous action space, and the continuous action is mapped into a discrete action that can perform resource scheduling in the communication environment.

2. The high-dynamic communication network spectrum resource allocation method based on multi-domain information fusion according to claim 1 is characterized in that: The process of applying the spectrum resource allocation strategy to the railway LTE network includes designing a multi-domain information fusion strategy based on a weighted fusion model. The method of designing a multi-domain information fusion strategy based on a weighted fusion model is: Determine the information weight of each domain according to the influence of each domain information on spectrum resource allocation; The information weights of the various domains are integrated to obtain a real-time network benefit function.

3. The high-dynamic communication network spectrum resource allocation method based on multi-domain information fusion according to claim 2 is characterized in that: The calculation formula of the network real-time revenue function is: in, is the network real-time benefit, α and β are the network's attention to transmission efficiency and real-time performance, respectively. is the transmission efficiency benefit function, is the real-time performance benefit function, UE g (t) represents all users, g represents base stations, G represents the base station set, u represents users, and p and q represent different coordination factors.

4. The high-dynamic communication network spectrum resource allocation method based on multi-domain information fusion according to claim 3 is characterized in that: The calculation formulas of the transmission efficiency benefit function and the real-time performance benefit function are respectively: in, represents the real-time transmission efficiency benefit of the service of user u belonging to base station g, represents the historical queue transmission efficiency benefit of user u belonging to base station g, represents the historical average transmission efficiency benefit of the service of user u belonging to base station g, is the service delay budget of user u belonging to base station g, D g,u (t) is the queue delay of user u belonging to base station g at time t.

5. The high-dynamic communication network spectrum resource allocation method based on multi-domain information fusion according to claim 1 is characterized in that: The action space needs to satisfy the constraint formula, which is: Among them, g is the base station, G is the base station set, N is the total number of base stations in the network, u is the user, V is the total number of resource blocks, BW g is the bandwidth limit of the base station cell in the railway LTE network. To meet the remaining resource block set of the base station cell in the railway LTE network after the private network service, UE g (t) for all users, a g,u,r (t)∈{0,1}, 0 means no allocation, 1 means allocation, r is the time-frequency resource block, b RB is the standard bandwidth, and j is any resource block number in the time-frequency resource block.

6. The method for allocating spectrum resources of a highly dynamic communication network based on multi-domain information fusion according to claim 1 is characterized in that: The process of optimizing the dynamic spectrum resource allocation strategy includes updating the main network and the target network; the main network includes a strategy network and an evaluation network, and the strategy network searches for an action that can maximize the value of the evaluation network; the target network includes a target strategy network and a target evaluation network.

7. The method for allocating spectrum resources in a highly dynamic communication network based on multi-domain information fusion according to claim 6 is characterized in that: The calculation formula for the number of input nodes of the strategy network is: N input =N·M·(2V+4); Among them, N input is the number of input nodes, N is the total number of base stations in the network, M is the number of users in the corresponding cell of each base station, and V is the number of resource blocks in the corresponding cell of each base station; The calculation formula for the number of output nodes of the evaluation network is: N output =N·M·V; Among them, N output is the number of output nodes.

8. A highly dynamic communication network spectrum resource allocation system based on multi-domain information fusion, characterized in that: include: The scenario domain cognition module and the business domain cognition module are used to respectively receive the scenario domain information and the business domain information output by the network status monitor, and obtain the predicted scenario domain information and the predicted business domain information of the next stage; A multi-domain information fusion module, used to fuse the predicted scene domain information and the predicted business domain information through a multi-domain information fusion mechanism to obtain fused information, and calculate the real-time network revenue value through the fused information; An intelligent agent, configured to receive the network real-time revenue value, and optimize dynamic spectrum resource allocation according to the network real-time revenue value; The intelligent agent includes at least two main networks and a target network.

9. The highly dynamic communication network spectrum resource allocation system based on multi-domain information fusion according to claim 8 is characterized in that: The reward function of the agent is obtained by calculating the real-time benefit value of the network.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is run, the method as claimed in any one of claims 1 to 7 is executed.

Images