Railway wireless communication network switching selection method and system for space-air-ground integration

Abstract

The invention discloses a railway wireless communication network switching selection method and system for space-air-ground integration, and belongs to the field of railway mobile communication, and the method comprises the following steps: constructing a railway wireless communication network switching selection model for space-air-ground integration; on the basis of a railway wireless communication network switching selection model, a train position prediction means is adopted to activate a switching trigger mechanism; and after the switching trigger mechanism is activated, obtaining a candidate network list, and based on the candidate network list, solving by adopting a network selection algorithm of deep reinforcement learning to obtain an optimal network selection strategy. According to the method, unnecessary switching is reduced while the communication service quality of the user is ensured, the switching failure probability is reduced, and the reliability of a railway wireless communication system and the travel experience of the user are effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of railway mobile communications, and in particular relates to a railway wireless communication network switching selection method and system oriented to air-ground-ground integration. Background Art

[0002] In recent years, with the rapid development of railway technology and the iteration of mobile communication networks, users' expectations for the service quality of railway wireless communication networks have increased exponentially. The complex application conditions of wireless communication systems that carry unique and diverse services such as new generation train control, multi-mode dispatching communication, infrastructure and disaster monitoring, and unmanned and less-manned operation and maintenance have put forward higher requirements for railway wireless communication systems. Therefore, in order to overcome the limitations of existing railway wireless communication systems, the combination of Space-Air-Ground Integrated Networks (SAGIN) and existing railway wireless communication systems has become increasingly important. The various networks in railway SAGIN complement each other and provide a new solution for low-latency and high-reliability railway communication systems. However, since railway SAGIN is an ultra-dense heterogeneous network with dual mobility of network nodes and user terminals, when the terminal is far away from the current service network node or the service type changes, the current service network node may not meet the terminal service requirements, which requires the terminal to perform network switching selection to ensure the quality of communication service. The application scenarios of railway wireless communication services are complex. In the railway SAGIN scenario, high-speed mobile terminals will face more frequent and complex network switching selection problems. When multiple communication modes coexist, facing different communication service requirements, how mobile users holding multi-mode interface terminals can make effective network switching choices is the key to improving user QoS.

[0003] At present, deep reinforcement learning algorithms are widely used in the field of network switching selection. Using traditional reinforcement learning algorithms for network switching selection will face the problem of too large state space and action space, resulting in a huge amount of algorithm calculation, affecting the efficiency and convergence speed of the algorithm, and thus unable to adapt to the high dynamics and high complexity of railway SAGIN. Therefore, the DRL method is used to optimize the railway SAGIN switching selection problem. The intelligent agent continuously interacts with the environment to learn and select the optimal network access for each mobile terminal in a dynamic and complex network environment. While ensuring the quality of user communication services, it reduces unnecessary switching and effectively improves the reliability of the railway communication system and the user's travel experience. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a railway wireless communication network switching selection method and system for air-ground integration to solve the problems existing in the above-mentioned prior art.

[0005] To achieve the above object, the present invention provides a railway wireless communication network switching selection method for air-ground integration, comprising:

[0006] Construct a railway wireless communication network switching selection model for air-ground integration;

[0007] Based on the railway wireless communication network switching selection model, a switching trigger mechanism is activated by using train position prediction means;

[0008] After activating the switching trigger mechanism, a candidate network list is obtained, and a network selection algorithm based on deep reinforcement learning is used to solve the optimal network selection strategy based on the candidate network list.

[0009] Optionally, the process of constructing the railway wireless communication network switching selection model includes establishing an objective function based on utility function theory; wherein the expression of the objective function is:

[0010] max x n,m (t)(U n,k (t)-C n,m (t)+L n (t))

[0011]

[0012] In the formula, x n,m (t) is the decision variable; U n,k (t) represents the QoS attribute parameter utility function; C n,m (t) represents the switching cost utility function; L n (t) represents the load utility function; C1 represents MT m The connection relationship between the candidate network node n; C2 represents MT m At time t, at most one network can be selected as the switching target network; C3 means that the average switching delay cannot be higher than the given maximum delay threshold; C4 means that the number of users on each network node cannot exceed the maximum threshold of the network load.

[0013] Optionally, the expression of the QoS attribute parameter utility function is:

[0014]

[0015] Where n represents the network node in the candidate network list, n∈{N1,N2,…,N n ,…,N N}; k represents the network QoS attribute parameter, k∈{B,D,J,BER}; K represents the number of QoS attribute parameters; represents the weight factor of QoS attribute parameter k; u n,k(t) represents the utility function of the QoS attribute parameter k of the candidate network n at time t.

[0016] Optionally, the switching cost utility function is expressed as:

[0017]

[0018] Where C(t) = τ h,n +P h,n ; τ h,n represents the switching delay; P h,n Represents the energy consumption during the switching process; P h,n =P n +P circuit,n ;P n represents the downlink transmission power at network node n; P circuit,n Represents the circuit power of network node n.

[0019] Optionally, the load utility function is expressed as:

[0020]

[0021] Where M n (t) represents the user terminal accessing the network node n at time t, M n is the maximum number of terminals that network node n can carry.

[0022] Optionally, the process of activating the trigger mechanism using the train position prediction means includes:

[0023] Acquiring track data, wherein the track data includes train position, train speed and train direction;

[0024] Preprocessing the trajectory data to obtain preprocessed data;

[0025] Processing the preprocessed data based on a long short-term memory network model predicts the position of the train at the next moment;

[0026] estimating link quality based on the position of the train at a next moment;

[0027] When the link quality meets the switching trigger condition, the switching trigger mechanism is activated in advance.

[0028] Optionally, satisfying the handover triggering condition includes satisfying a handover triggering policy and the candidate network list being not empty at the same time;

[0029] The expression of the switching trigger strategy is:

[0030]

[0031] In the formula, RSS S,t+1 Indicates the received signal strength of the current service network at the next moment; CNR S,t+1 Indicates the carrier-to-noise ratio of the current service network at the next moment; RSS T,t+1 Indicates the received signal strength of the candidate network at the next moment; CNR T,t+1 Indicates the carrier-to-noise ratio of the candidate network at the next moment; RSS Hys Indicates the given received signal strength hysteresis margin; CNR Hys Represents a given carrier-to-noise ratio hysteresis margin; Γ RSS,th represents a given received signal strength threshold; Γ CNR,th Indicates a given carrier-to-noise ratio threshold.

[0032] Optionally, the system of the railway wireless communication network switching selection method for air-ground integration includes: a data acquisition module, a data processing module, a position prediction module, a network estimation module, a trigger decision module, a parameter determination module, a weight calculation module, a network selection module and an output module connected in sequence;

[0033] Wherein, the data acquisition module is used to collect trajectory data;

[0034] The data processing module is used to preprocess the trajectory data to obtain preprocessed data;

[0035] The position prediction module is used to predict the pre-processed data based on the long short-term memory network model to obtain the position of the train at the next moment;

[0036] The network estimation module is used to estimate the link quality of the current serving network and the candidate network based on the predicted position of the train at the next moment;

[0037] The trigger decision module is used to determine whether the switching trigger condition is met based on the link quality between the current serving network and the candidate network and the user terminal;

[0038] The parameter determination module is used to determine the QoS attribute parameters that affect the switching process after the trigger mechanism is activated in advance;

[0039] The weight calculation module is used to determine the weight of the QoS attribute parameter based on the best-worst method and the entropy method;

[0040] The network selection module is used to use a deep reinforcement learning network selection algorithm to solve and obtain an optimal network selection strategy;

[0041] The output module is used to output the optimal network switching node.

[0042] Compared with the prior art, the present invention has the following advantages and technical effects:

[0043] In a railway wireless communication network switching selection method and system for air-ground integration provided by the present invention, according to the established railway SAGIN switching selection model, the network switching trigger condition is set based on the train position, speed, terminal service type and network characteristics; the train position at the next moment is predicted according to the long short-term memory network model, and the link quality is estimated; according to the link quality of the current service network and the candidate network at the next moment, it is judged whether the switching trigger condition is met; when the network switching trigger condition is met, a list of candidate network nodes is obtained; according to the differentiated requirements of different terminal service types for QoS attribute parameters, the QoS attribute parameters that affect the switching process are determined, and based on the network scenario and optimization target, the switching trigger condition is set. Define an intelligent agent, a state space, an action space, and a reward function; use the best-worst method and the entropy method to determine the weights of the QoS attribute parameters; use a deep reinforcement learning algorithm to solve the optimization problem and output an optimal network selection strategy; the present invention provides a railway wireless communication network switching selection method and system for air-ground integration to solve the problems of frequent terminal switching and unbalanced network load caused by the dual movement of terminals and network nodes and changes in business types in the railway SAGIN scenario, which affect the quality of user communication services. While ensuring the quality of user communication services, it reduces unnecessary switching, reduces the probability of switching failure, and effectively improves the reliability of the railway communication system and the travel experience of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0045] Figure 1 A diagram showing the steps of a railway wireless communication network switching selection method for air-ground integration according to an embodiment of the present invention;

[0046] Figure 2 A schematic diagram of a system model of an embodiment of the present invention;

[0047] Figure 3 A schematic flow chart of a method for switching and selecting a railway wireless communication network for air-ground integration according to an embodiment of the present invention;

[0048] Figure 4 A schematic diagram of the principle of a long short-term memory network model according to an embodiment of the present invention;

[0049] Figure 5 A schematic diagram of the principle of a network selection algorithm based on deep reinforcement learning according to an embodiment of the present invention;

[0050] Figure 6The present invention is a schematic diagram of the structure of a railway wireless communication network switching selection system for air-ground integration according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0052] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0053] Embodiment 1

[0054] like Figure 1 As shown, in this embodiment, a railway wireless communication network switching selection method and system for air-ground integration is provided, including: determining optimization targets, data collection and processing, location prediction, link quality estimation, switching trigger judgment, switching target network selection, etc. The following steps are included: constructing a railway wireless communication network switching selection model for air-ground integration; based on the railway wireless communication network switching selection model, using train location prediction means to activate the switching trigger mechanism; after activating the switching trigger mechanism, a candidate network list is obtained, and a network selection algorithm based on the candidate network list is used to solve and obtain the optimal network selection strategy.

[0055] S1. Construct a railway wireless communication network switching selection model for air-ground integration. According to the characteristics of different types of services, resource configuration of each network node and constraints stipulated by terminal requirements, based on the utility function theory, the user quality of service (QoS) requirements, switching costs, and network load balancing are integrated to form a comprehensive utility function as the optimization goal; the optimization problem is further decomposed into the switching trigger problem and the switching target network selection problem. Specifically including steps S101-S102:

[0056] S101: According to the characteristics of different types of services, resource configuration of each network node and terminal demand, constraints are set, and based on the utility function theory, user QoS requirements, switching costs, and network load balancing are integrated to form a comprehensive utility function as an optimization target;

[0057] S102: further decomposing the optimization problem into a handover triggering problem and a handover target network selection problem.

[0058] S2: Set network switching trigger conditions based on train location, speed, terminal service type and network characteristics, and propose a switching trigger algorithm based on location prediction; prepare for switching in advance by predicting changes in network conditions, and reduce switching delay and switching failure probability. Specifically, it includes steps S103-S105:

[0059] S103: Propose a handover triggering algorithm based on location prediction and set handover triggering conditions;

[0060] S104: predicting the train position at the next moment according to the long short-term memory network model and estimating the link quality;

[0061] S105: Determine whether a switching trigger condition is met based on the link quality of the current serving network and the candidate network at the next moment.

[0062] The process of activating the switching trigger mechanism by means of train position prediction includes:

[0063] Acquiring track data, wherein the track data includes train position, train speed and train direction;

[0064] Preprocessing the trajectory data to obtain preprocessed data;

[0065] The pre-processed data is processed based on the long short-term memory network model to predict the train's position at the next moment;

[0066] Estimate link quality based on the train's next position;

[0067] When the link quality meets the switching trigger condition, the switching trigger mechanism is activated in advance.

[0068] S3. When the switching trigger condition is met, the mobile terminal selects the optimal network according to a certain strategy; the network selection problem in the railway SAGIN is modeled as a reinforcement learning problem; according to the differentiated requirements of different terminal service types for QoS attribute parameters, the QoS attribute parameters that affect the switching process are determined, and the intelligent agent, state space, action space, and reward function are set based on the network scenario and optimization goal, so as to iteratively solve the established optimization problem according to the deep reinforcement learning algorithm. Specifically, steps S106-S109 are included:

[0069] S106: When the network switching triggering condition is met, obtain a candidate network list;

[0070] S107: Propose a network selection algorithm based on deep reinforcement learning and output the optimal network selection strategy;

[0071] S108: Determine the weight of the QoS attribute parameter by using the best-worst method and the entropy method;

[0072] S109: Repeat the solution algorithm until the algorithm converges and outputs the optimal switching network node.

[0073] As a preferred solution of this embodiment, the specific implementation of step S1 includes:

[0074] like Figure 2 As shown in FIG. 1 , a railway SAGIN switching selection system model composed of satellites, drones, ground cellular network nodes, and trains is shown in FIG. 1 , which includes N network nodes and M mobile terminals. As N={N1,N2,…,N BS ,…,N k ,N k+1 ,…,N UAV ,…,N u ,N u+1 ,…,N Sat ,…,N N} represents N network nodes, where {N1,N2,...,N BS ,...,N k} represents the ground network node, {N k+1 ,...,N UAV ,...,N u} represents a UAV network node, {N u+1 ,…,N Sat ,…,N N} represents a LEO satellite network node. The LEO satellite moves along a fixed ephemeris trajectory. When it moves to the user's visible range, that is, when the elevation angle between the satellite and the user is greater than the minimum communication elevation angle, it can provide communication services to the user. M} represents M mobile terminals randomly distributed on the train. In order to indicate service continuity, it is assumed that network services are provided in a certain time period [0, T]. The decision time interval is τ, and the sequence T = {T0, T1, …, T k}={[t0,t1),[t1,t2),…,[t k-2 ,t k-1 ),[t k-1 ,t k ]} represents the decision time series, where T k Indicates the time when the connection is terminated. Each mobile terminal may be covered by multiple networks, but the communication terminal can only be provided with communication services by one network node at each moment.

[0075] (1) Service QoS requirements

[0076] In order to adapt to the diverse characteristics of user terminal services in actual communication scenarios and the differentiated requirements of different services for QoS attribute parameters, personalized utility functions are designed for various services. Referring to the QoS classification of mobile communication networks defined by 3GPP, four types of services are considered: conversational, interactive, streaming media, and background.

[0077] Bandwidth (B), delay (D), jitter (J), and bit error rate (BER) are selected as decision parameters for evaluating service QoS and network status. The QoS attribute parameter utility function is constructed by weighted sum to reflect the importance of different QoS attribute parameters in switching decisions. Then the QoS attribute parameter utility function U n,k (t) can be expressed as:

[0078]

[0079] Where n represents the network node in the candidate network list, n∈{N1,N2,…,N n ,…,N N}; k represents the network QoS attribute parameter, k∈{B,D,J,BER}; K represents the number of QoS attribute parameters; represents the weight factor of QoS attribute parameter k; u n,k (t) represents the utility function of the QoS attribute parameter k of the candidate network n at time t.

[0080] ①Network attribute parameter weight

[0081] When constructing the QoS attribute parameter utility function, a method combining subjective preferences and objective performance is used to determine the QoS attribute parameter weights. Using subjective or objective weights alone may not fully reflect all the information of network attribute parameters. Therefore, a linear weighting method is used to combine the subjective weights calculated by the Best Worst Method (BWM) and the objective weights calculated by the Entropy Method (EM). This method takes into account both the user's personal preferences and the actual performance of the network, thereby providing more comprehensive decision support for network selection.

[0082] ② Utility function design:

[0083] In network selection, the utility function is used to quantify the user terminal's preference for network attribute parameters. The larger the utility value, the more the network attribute parameters meet the user's business needs. Various commonly used mathematical functions, such as piecewise function, logarithmic function, exponential function, Sigmoid function (S-type function), etc., can be used to construct the utility model of network attribute parameters. By designing a reasonable utility function, the requirements of different business types for network attribute parameters can be effectively mapped. This helps to make network selection more accurate and ensure that the selected network meets the user's business needs to the greatest extent.

[0084] For benefit-type network attribute parameters such as bandwidth and signal-to-noise ratio, the utility function u(x) is modeled using a monotonically increasing function. For cost-type parameters such as latency and jitter, a decreasing function such as 1-u(x) is used. For different types of services, the utility function of network attribute parameters should reflect the unique requirements of each service for network attributes.

[0085] (2) Switching Cost

[0086] The network switching cost mainly involves various expenses incurred during the switching process, including time and resource consumption. Time consumption refers to the time required for a user to complete a network switching process. Resource consumption mainly considers power resource consumption and energy consumption. Define x n,m (t) is a decision variable, which is a binary variable (0 or 1) representing the MT at time t. m The connection relationship with network node n can be expressed as:

[0087]

[0088] In order to avoid frequent switching, a switching cost utility function is introduced; when the target network to be switched meets the QoS required by the service, C(t) = 0, otherwise C(t) = τ h,n +P h,n . Then the switching cost utility function can be expressed as:

[0089]

[0090] Among them, τ h,n represents the switching delay; P h,n Indicates the energy consumption during the switching process; C min and C max They represent the minimum and maximum switching costs respectively.

[0091] P h,n =P n +P circuit,n (4)

[0092] Among them, P nrepresents the downlink transmission power at network node n; P circuit,n Represents the circuit power of network node n.

[0093] (3) Load Balancing

[0094] A network can provide network services to multiple terminals at the same time, but the network resources owned by each network node are limited. When the number of connected terminals exceeds the maximum carrying capacity of the network, the network will be blocked and the user access request will fail. The load condition at network node n at time t can be quantified by the load utility function.

[0095]

[0096] Among them, M n (t) represents the user terminal accessing the network node n at time t, M n is the maximum number of terminals that network node n can carry.

[0097] (4) Objective function

[0098] The optimization goal considered in this embodiment is to reduce unnecessary switching while ensuring user QoS; in summary, the final optimization problem is modeled as maximizing the comprehensive utility function. The objective function can be specifically expressed as:

[0099]

[0100] Among them, C1 represents MT m The connection relationship between MT and candidate network node n. m At time t, at most one network can be selected as the switching target network; C3 means that the average switching delay cannot be higher than the given maximum delay threshold. C4 means that the number of users on each network node cannot exceed the maximum threshold of the network load.

[0101] As a preferred solution of this embodiment, the specific implementation of step S2 includes:

[0102] According to the constructed railway wireless communication network switching selection model for air-ground integration, a switching triggering algorithm based on position prediction is proposed. Figure 3 As shown, first, the position, speed and other data of the train are obtained through sensors installed on the train and the track, and the direction data of the train is obtained in combination with the train operation diagram and timetable; the collected trajectory data is preprocessed, including data cleaning, data conversion, data analysis, etc.; the position of the train at the next moment is predicted based on the long short-term memory network model; the link quality between the current service network and the candidate network and the user terminal is estimated based on the predicted position information; by predicting changes in network conditions, the switching is prepared in advance to reduce the switching delay and the probability of switching failure;

[0103]

[0104] Among them, l t+1,m Indicates the user's predicted location coordinates at the next moment; l t+1,n Represents the location coordinates of network node n at the next moment.

[0105] According to the link quality of the current service network and the target network at the next moment, it is judged whether the handover trigger condition is met; if formula (8) is met and the candidate network list is not empty, that is, a candidate network with better link quality is found to be available, the handover trigger mechanism will be activated in advance to reserve more time for the subsequent handover target network selection. In order to improve the accuracy of handover trigger decision and reduce unnecessary handover, referring to the A3 handover trigger event in the ground network, the hysteresis margin is introduced as an additional trigger condition, and the handover trigger strategy can be expressed as:

[0106]

[0107] Among them, RSS S,t+1 and CNR S,t+1 Indicates the received signal strength and carrier-to-noise ratio of the current service network at the next moment; RSS T,t+1 and CNR T,t+1 Indicates the received signal strength and carrier-to-noise ratio of the candidate network at the next moment; RSS Hys and CNR Hys Indicates the given carrier-to-noise ratio and received signal strength hysteresis margin, that is, the difference between the current serving cell and the target cell, and sets the RSS according to its own position and moving speed Hys and CNR Hys Parameters; Γ RSS,th and Γ CNR,th Indicates the given received signal strength and carrier-to-noise ratio thresholds.

[0108] like Figure 4 As shown, it shows a schematic diagram of the principle of the long short-term memory network model of the present invention; Long Short-Term Memory Networks (LSTM) is a special recurrent neural network (RNN). It solves the "gradient disappearance" and "gradient explosion" problems encountered by traditional RNN when processing long sequences by introducing memory cells (Cell State), input gates (Input Gate), output gates (Output Gate) and forget gates (Forget Gate), effectively solving the problem that RNN cannot handle long-distance dependencies.

[0109] The performance of LSTM model is affected by many parameters, including the number of hidden layers, the number of nodes in each layer, the learning rate, the discard rate and other parameters, which jointly determine the predictive ability of the model. However, in the model training process, it is mainly based on historical experience to manually adjust the parameters, which is not only inefficient but also difficult to ensure that the model reaches the optimal state. Therefore, this embodiment uses a relatively novel sparrow search algorithm (Sparrow Search Algorithm, SSA) to optimize the key hyperparameters in the LSTM model, including the number of LSTM layers, the number of Dense layers, the number of neurons in each layer, etc. SSA is a swarm intelligence optimization algorithm based on simulating the foraging behavior of sparrows. Using the powerful search ability of SSA, the best parameter combination of the neural network model can be efficiently determined, thereby improving the performance of the model and the accuracy of prediction.

[0110] As a preferred solution of this embodiment, the specific implementation of step S3 includes:

[0111] When the switching trigger condition is met, the mobile terminal will select the optimal network according to a certain strategy; the network selection problem in the railway SAGIN is modeled as a reinforcement learning problem and solved; according to the differentiated requirements of different terminal service types for QoS attribute parameters, the QoS attribute parameters that affect the switching process are determined, and the intelligent agent, state space, action space, and reward function are set based on the network scenario and optimization goal, so as to iteratively solve the established optimization problem according to the Deep Reinforcement Learning (DRL) algorithm; Figure 3 As shown in Figure 1, the process of solving the network selection problem through deep reinforcement learning algorithm.

[0112] First, we construct the four key elements of the reinforcement learning model: intelligent agent, state space, action space, and reward function, transform the railway SAGIN network selection problem into a reinforcement learning problem, and achieve multi-objective switching performance optimization under complex system constraints. The specific description is as follows:

[0113] Agent: In this embodiment, the agent refers to the user terminal in the railway SAGIN system.

[0114] State space: The current service type S of the user terminal type 、Currently selected network N n and QoS parameter P representing the network status n (Bandwidth B, delay D, jitter J, bit error rate BER) as the network selection decision factors, the state space S can be expressed as:

[0115] S={S type ,N n ,Pn |n=1,2,...,N} (9)

[0116] Among them, P n Represents the QoS parameters of candidate network n, including {B n ,D n ,J n ,BER n}; N n Indicates the network to which the current user terminal is connected; S type Indicates the four service types of user terminals, S type ∈{0,1,2,3}.

[0117] Action space (Action): The action of the user terminal is to select a network to switch. Due to the heterogeneity of railway SAGIN, there may be multiple networks for users to switch to at the same time, but at any time t, the terminal can only select one network to access. Then the action space A can be expressed as:

[0118] A={a(t)|a(t)∈(N1,N2,…,N i ,…,N n )} (10)

[0119] Among them, a t Indicates the action taken by the user terminal at the time of network selection.

[0120] Reward function: r(t) represents the reward after the user terminal performs action a(t) at time t. The reward function can be expressed as:

[0121] r(t)=U n,k (t)-C n,m (t)+L n (t) (11)

[0122] Among them, U n,k C (t) represents the utility function of the QoS attribute parameter k of the candidate network n at time t; n,m (t) represents the switching cost utility function; L n (t) represents the load utility function.

[0123] like Figure 5 As shown, it shows a schematic diagram of the principle of a network selection algorithm based on deep reinforcement learning of the present invention; the method for solving the network selection problem by using a deep reinforcement learning algorithm may include the following steps:

[0124] Step S31, initialize the main Q network weight parameter θ main 、Target Q network weight parameter θ target =θ main, experience replay pool D, learning rate, discount factor, state space, action space;

[0125] Step S32, setting the number of iterations Z;

[0126] Step S33, initialization state s(1);

[0127] Step S34, setting decision time;

[0128] Step S35: Input the current state s(t) into the main Q network to obtain the Q of all state-action pairs. Main Value output, use the ε-greedy strategy to select action a(t), that is, select the target network to switch to;

[0129] Step S36: After executing action a(t), obtain the next moment environment state s(t+1) and calculate the reward value r(t);

[0130] Step S37, store (s(t), a(t), r(t), s(t+1)) into the experience replay pool D;

[0131] Step S38: Randomly extract a batch of experience samples (size is batch size) from the experience replay pool D, and calculate the target Q value, i.e., Q Target (s(t+1),a(t+1);θ target );

[0132] Step S39: Calculate the loss function and loss gradient, and update the main Q network weight parameter θ by gradient descent method. main , that is, update the network selection strategy π * (s) = argmaxQ(s(t),a(t);θ main );

[0133] Step S40: copy the weight parameters of the main Q network to the target Q network, θ target =θ main ;

[0134] Step S41, repeat steps S31 to S40 until the optimization problem converges and the optimal switching network node is output.

[0135] like Figure 6 The figure shows a schematic diagram of the structure of a railway wireless communication network switching selection system for air-ground integration of the present invention. The system can be implemented as all or part of a terminal device through software, hardware or a combination of both. The system includes a data acquisition module, a data processing module, a location prediction module, a network estimation module, a trigger judgment module, a parameter determination module, a weight calculation module, a network selection module and an output module connected in sequence.

[0136] Among them, the data acquisition module obtains the train's position, speed and other data through sensors installed on the train and track, and obtains the train's direction data in combination with the train operation diagram and timetable;

[0137] The data processing module pre-processes the collected trajectory data, including data cleaning, data conversion, data analysis, etc.

[0138] The location prediction module predicts the next location of the train based on the long short-term memory network model;

[0139] A network estimation module estimates the link quality between the current serving network and the candidate network and the user terminal based on the predicted location information;

[0140] The trigger decision module determines whether the switching trigger condition is met according to the link quality;

[0141] The parameter determination module determines the QoS attribute parameters that affect the switching process according to the differentiated requirements of different service types of the terminal for QoS attribute parameters, and sets the intelligent agent, state space, action space, and reward function based on QoS attributes and network scenarios;

[0142] The weight calculation module determines the weights of QoS attribute parameters according to the best-worst method and entropy method;

[0143] The network selection module solves the optimization problem according to the network selection algorithm based on deep reinforcement learning and selects the network node with the largest cumulative expected reward value;

[0144] The output module outputs the optimal switching network nodes according to the algorithm solution results.

[0145] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A railway wireless communication network switching selection method for air-ground integration, characterized in that: include: Construct a railway wireless communication network switching selection model for air-ground integration; Based on the railway wireless communication network switching selection model, a switching trigger mechanism is activated by using train position prediction means; After activating the switching trigger mechanism, a candidate network list is obtained, and a network selection algorithm based on deep reinforcement learning is used to solve the optimal network selection strategy based on the candidate network list.

2. The method for switching and selecting a railway wireless communication network for air-ground integration according to claim 1, characterized in that: The process of constructing the railway wireless communication network switching selection model includes establishing an objective function based on the utility function theory; wherein the expression of the objective function is: max x n,m (t)(U n,k (t)-C n,m (t)+L n (t)) In the formula, x n,m (t) is the decision variable; U n,k (t) represents the QoS attribute parameter utility function; C n,m (t) represents the switching cost utility function; L n (t) represents the load utility function; C1 represents MT m The connection relationship between the candidate network node n; C2 represents MT m At time t, at most one network can be selected as the switching target network; C3 means that the average switching delay cannot be higher than the given maximum delay threshold; C4 means that the number of users on each network node cannot exceed the maximum threshold of the network load.

3. The method for switching and selecting a railway wireless communication network for air-ground integration according to claim 2, characterized in that: The expression of the QoS attribute parameter utility function is: Where n represents the network node in the candidate network list, n∈{N1,N2,...,N n ,…,N N }; k represents the network QoS attribute parameter, k∈{B,D,J,BER}; K represents the number of QoS attribute parameters; represents the weight factor of QoS attribute parameter k; u n,k (t) represents the utility value of the QoS attribute parameter k of the candidate network n at time t.

4. The method for switching and selecting a railway wireless communication network for air-ground integration according to claim 2, characterized in that: The expression of the switching cost utility function is: Where C(t) = τ h,n +P h,n ; τ h,n represents the switching delay; P h,n Represents the energy consumption during the switching process; P h,n =P n +P circuit,n ;P n represents the downlink transmission power at network node n; P circuit,n Represents the circuit power of network node n.

5. The method for switching and selecting a railway wireless communication network for air-ground integration according to claim 2, characterized in that: The expression of the load utility function is: Where M n (t) represents the user terminal accessing the network node n at time t, M n is the maximum number of terminals that network node n can carry.

6. The method for switching and selecting a railway wireless communication network for air-ground integration according to claim 1, characterized in that: The process of activating the trigger mechanism using train position prediction means includes: Acquiring track data, wherein the track data includes train position, train speed and train direction; Preprocessing the trajectory data to obtain preprocessed data; Processing the preprocessed data based on a long short-term memory network model predicts the position of the train at the next moment; estimating link quality based on the position of the train at a next moment; When the link quality meets the switching trigger condition, the switching trigger mechanism is activated in advance.

7. The method for switching and selecting a railway wireless communication network for air-ground integration according to claim 6, characterized in that: Meeting the handover triggering condition includes simultaneously meeting the handover triggering policy and the candidate network list being not empty; The expression of the switching trigger strategy is: In the formula, RSS S,t+1 Indicates the received signal strength of the current service network at the next moment; CNR S,t+1 Indicates the carrier-to-noise ratio of the current service network at the next moment; RSS T,t+1 Indicates the received signal strength of the candidate network at the next moment; CNR T,t+1 Indicates the carrier-to-noise ratio of the candidate network at the next moment; RSS Hys Indicates the given received signal strength hysteresis margin; CNR Hys Represents a given carrier-to-noise ratio hysteresis margin; Γ RSS,th represents a given received signal strength threshold; Γ CNR,th Indicates a given carrier-to-noise ratio threshold.

8. The system of the railway wireless communication network switching selection method for air-ground integration according to claim 1 is characterized in that: include: A data acquisition module, a data processing module, a location prediction module, a network estimation module, a trigger decision module, a parameter determination module, a weight calculation module, a network selection module and an output module connected in sequence; Wherein, the data acquisition module is used to collect trajectory data; The data processing module is used to preprocess the trajectory data to obtain preprocessed data; The position prediction module is used to predict the pre-processed data based on the long short-term memory network model to obtain the position of the train at the next moment; The network estimation module is used to estimate the link quality between the current serving network and the candidate network and the user terminal based on the predicted position of the train at the next moment; The trigger decision module is used to determine whether the switching trigger condition is met based on the link quality of the current serving network and the candidate network at the next moment; The parameter determination module is used to determine the QoS attribute parameters that affect the switching process after the trigger mechanism is activated in advance; The weight calculation module is used to determine the weight of the QoS attribute parameter based on the best-worst method and the entropy method; The network selection module is used to use a deep reinforcement learning network selection algorithm to solve and obtain an optimal network selection strategy; The output module is used to output the optimal network switching node.

Images