A high-speed vehicle intelligent network switching method based on rate and cost

Abstract

The present invention belongs to the technical field of heterogeneous network switching, and specifically relates to a method for intelligent network switching of high-speed target vehicles based on rate and cost in a heterogeneous car-city network. The implementation steps are as follows: First, based on heterogeneous network units, high-speed mobile target vehicle scenarios under different types of business requirements are constructed and network parameters are obtained. Secondly, the optimization target is determined by combining the transmission rate and the switching cost. Finally, a heterogeneous network switching algorithm based on DQN is designed. The simulation results show that the method can eventually achieve convergence, and the number of network switching caused by the three types of services set by the present invention is no more than 2 times, which is reduced by 1 time compared to the switching algorithm that only uses the rate requirement as the switching condition. It has been verified by experiments that this method can effectively solve the network switching problem affected by the near-far effect in high-speed mobile scenarios while reducing the network switching cost.

Description

Technical Field

[0001] The present invention belongs to the technical field of heterogeneous network switching, and mainly relates to the design of a high-speed moving target vehicle network switching method, and specifically to a high-speed target vehicle intelligent network switching method based on rate and cost in a heterogeneous vehicle-city network. Background Art

[0002] As the scale of Checheng Network continues to expand, the demand for vehicle services has surged, and new requirements have been put forward for data transmission under the prominent network heterogeneity characteristics. Vehicles switch to different standard networks according to different business needs. In the high-speed moving scenario of the target vehicle of Checheng Network, on the one hand, due to the influence of the near-far effect, the data transmission rate when the vehicle travels to the edge of the access point coverage area cannot meet the business rate requirements; on the other hand, high-speed movement causes a surge in network switching costs. Therefore, how to meet the different business transmission rate requirements of high-speed moving target vehicles while performing network switching at a low cost has become a problem worthy of attention.

[0003] In the document with application number "202010014344.3", the "5G heterogeneous network switching decision method based on adaptive multi-criteria fuzzy logic" is disclosed. This method uses eight decision parameters to make intelligent and comprehensive switching decisions when multiple available networks are covered at the same time. The current input decision parameters form fuzzy criteria and then the output scores are calculated by mathematical functions to obtain the final decision value, which improves the accuracy of network switching. However, affected by the multi-parameter criteria, considering multiple parameters will increase the signaling overhead required for network switching exponentially, and the switching cost will increase significantly. In the document with application number "201810198466.5", "A vehicle network switching method in a heterogeneous vehicle network environment" is disclosed. This method comprehensively considers the throughput factors of the heterogeneous network environment and the service quality requirements of vehicle applications on different vehicle mobile terminals, and proposes a bio-inspired switching decision model to quickly reach the global optimum, but does not consider the switching cost and transmission rate impact brought by high-speed moving targets during network switching. From the above analysis, it can be seen that the network switching algorithm of the heterogeneous car-city network, which takes into account the network switching cost and the service data transmission rate, is a direct way to ensure the user experience. Summary of the invention

[0004] The present invention provides a high-speed vehicle intelligent network switching method based on rate and cost, so as to overcome the problem that the existing network switching technology is difficult to balance the task transmission rate and switching overhead.

[0005] To achieve the above object, the technical solution provided by the present invention is: a high-speed vehicle intelligent network switching method based on rate and cost, comprising the following steps:

[0006] Step 1: Based on heterogeneous network units, high-speed moving target vehicle scenarios under different types of business requirements are constructed and relevant network parameters are obtained;

[0007] Step 2: Calculate the data transmission rate and network switching cost, and determine the optimization target;

[0008] (1) Calculate the data transmission rate and network switching cost of high-speed moving target vehicles;

[0009] (2) Combine the transmission rate and switching cost to construct an optimization objective function;

[0010] Step 3: Design a network switching algorithm based on DQN in heterogeneous network units.

[0011] Furthermore, in the above step 1, the heterogeneous network unit is composed of k 4G BSs, l 5G BSs and m RSUs, where k = {0, 1}, 0≤l≤L, 0≤m≤M, l, m∈N*.

[0012] Furthermore, in the above step 1, the service types are transmission rate sensitive services, switching cost sensitive services and simple services.

[0013] Furthermore, for the above-mentioned high-definition video transmission or large-scale data upload, this type of business is a transmission rate-sensitive business; for low-latency emergency business, this type of business is a switching cost-sensitive business; and the business in between is a simple business.

[0014] Furthermore, in the above step 1, the network parameters are the vehicle's location information and network switching signaling overhead.

[0015] Furthermore, in step 2 (1) above, in the high-speed moving target vehicle scenario, the total switching cost is expressed as C tot :

[0016]

[0017] Among them C tau and C lp are the related message overheads of TAU and LP, ap′ indicates the current access network of the target vehicle, ap indicates the target network to be switched, λ is the rate of triggering the TAU process, and α is the rate of triggering the LP process.

[0018] Furthermore, in the above step 2 (2), the network unit is used as the selection range for the vehicle to switch the network, the network with the maximum transmission rate is selected as the accessible target network, and the network switching cost is combined to construct the utility function as follows:

[0019]

[0020] Where ap is the target network to be switched, i represents the i-th switch, i∈I, I represents the total number of switches in a network unit, and w1,w2 are the weights corresponding to different service types;

[0021] The optimization goal is:

[0022]

[0023]

[0024]

[0025] C3:0<P ap <max{P ap}(7d)

[0026] Among them, R(z) ap represents the transmission rate between the vehicle and the AP, C tot represents the total switching cost, Indicates the lower limit of the transmission rate of different service types. Indicates the upper limit of switching cost. type indicates the service type, and type1, type2, and type3 respectively represent transmission rate-sensitive services, switching cost-sensitive services, and simple services. In general, P1 is the optimization goal, which means that the total utility function is maximized after the vehicle passes through a network unit; C1 and C2 respectively constrain the minimum task transmission rate and maximum network switching cost of the target vehicle; C3 constrains the transmission power of the AP.

[0027] Furthermore, in the above step 3, in the network switching algorithm designed based on DQN, the reward function is expressed as:

[0028]

[0029] Compared with the existing method, the beneficial effects of the present invention are:

[0030] (1) The present invention uses heterogeneous network units as basic scenario elements, which enhances the scalability and adaptability of the system. Units in the heterogeneous network can be added or replaced at will to adapt to new tasks, data types or performance requirements without making large-scale changes to the entire system. This provides higher system flexibility, reduces system complexity, and makes the overall system easier to understand and maintain.

[0031] (2) The present invention uses the combined transmission rate and switching cost as the basis for determining the optimization target, which can improve system performance. Frequent switching may introduce unnecessary unstable factors. By reducing the number of switching times, the risk of network interruption can be reduced, switching delay can be reduced, and communication reliability can be improved. While reducing the network switching cost, it effectively solves the network switching problem affected by the near-far effect in high-speed mobile scenarios, reduces system energy consumption and maintenance costs, and promotes the development of a sustainable and green economy.

[0032] (3) The present invention implements algorithm optimization based on the DQN algorithm, reshapes the reward function and the experience replay mechanism, so that high-speed vehicles can use the network status information of the current time slot to search for the optimal network online according to business needs and decide whether to switch networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a flow chart of the present invention;

[0034] Figure 2 It is a schematic diagram of applicable scenarios of examples of the present invention;

[0035] Figure 3 Schematic diagram for calculating the communication distance between the vehicle and the base station;

[0036] Figure 4 Design a network switching algorithm flow chart for DQN;

[0037] Figure 5 is the average reward during the training phase;

[0038] Figure 6 The average number of switching times for different service types. DETAILED DESCRIPTION

[0039] The technical scheme in the embodiment of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] The present invention is further described in detail through a specific implementation method and accompanying drawings. The specific steps are as follows:

[0041] Step 1: Based on heterogeneous network units, high-speed moving target vehicle scenarios under different types of business requirements are constructed and relevant network parameters are obtained. The specific process is:

[0042] (1) Establishment of heterogeneous network units:

[0043] In the present invention, the scenario of high-speed moving target vehicles under different types of service requirements in heterogeneous network units will be considered. Figure 2 As shown in Figure 1, a network unit is constructed based on the coverage of 4G BS. Each network unit consists of k 4G BSs, l 5G BSs, and m RSUs, where k = {0, 1}, 0≤l≤L, 0≤m≤M, l, m∈N*. Assuming that each BS and RSU can provide corresponding resources for vehicles within its radio coverage, and there is an overlapping area of ​​network coverage between each selectable access point (AP), that is, the vehicle is covered by multiple networks and there are multiple APs to choose from, then the set of selectable access points is expressed as

[0044] (2) Classification of different service types, and determination of minimum transmission rate and switching cost requirements for different service types:

[0045] For each vehicle, a new task will be randomly generated in each time period, corresponding to the audio, data and video applications of the communication service. Different types of vehicle tasks have different task requirements. For example, for high-definition video transmission or large-scale data upload, a higher transmission rate is required. Therefore, the present invention divides different service types based on transmission rate and switching cost as the main basis, and divides them into transmission rate sensitive services, switching cost sensitive services and simple services. For different service types, there are different minimum transmission rate and switching cost requirements, which are respectively expressed as

[0046] (2) Determine relevant network parameters, where the network parameters are the location information of the vehicle and the signaling overhead of Tracking Area Update (TAU) and Location Paging (LP) involved in the network switching process.

[0047] Regarding the acquisition of vehicle location information, the sub-area Z = {1, ..., z} is divided according to the distance traveled by the target vehicle in each time slot t, t = 1ms. The vehicle obtains the starting and ending location information of the corresponding sub-area through the global positioning system and displays it in the form of coordinates (l z ,0) and (m z ,0) reflects, where l z and m z They represent the horizontal coordinates of the starting and ending positions of the z region respectively.

[0048] Regarding the acquisition of signaling overhead involved in the network switching process, during the period when the vehicle establishes communication with the AP, the two processes that mainly generate signaling overhead are TAU and LP. Through these two processes, the number, type and timing information of related signaling events are captured and analyzed. For example, signaling events such as switching request, switching command and switching confirmation. Therefore, Table 1 is introduced to obtain the signaling overhead of TAU and LP involved in each network switching process, where TAU and LP are measured according to the number of messages M or the number of bits required for each network unit.

[0049] Table 1 Signaling overhead of TAU and LP process

[0050]

[0051]

[0052] Step 2: Calculate the data transmission rate and network switching cost and determine the optimization target, including the following two steps:

[0053] (1) Calculate the data transmission rate and network switching cost of high-speed moving target vehicles.

[0054] Because the vehicle always maintains a high speed v, the change in channel gain caused by this cannot be ignored. The present invention discretizes the channel model and calculates it in different sub-areas. It is assumed that the channel gain when the vehicle is in sub-area z is the channel gain when the vehicle is in the middle of sub-area z, that is, the channel state information of each time slot t remains unchanged. Figure 3 As shown, when the vehicle moves to sub-area z, the channel gain in the area is given by The channel gain of the position is expressed as: The communication distance between the vehicle and the base station is expressed as:

[0055]

[0056] Among them, a xap ,a yap They are respectively represented as the horizontal and vertical coordinates of the accessible network point AP.

[0057] Then the channel gain g(z) when the vehicle is in sub-area z is expressed as:

[0058] g(z)=d(z) -σ , z∈Z (2)

[0059] Where σ is the path loss exponent. d(z) is the communication distance between the vehicle and the AP in sub-area z. Therefore, the transmission rate R(z) between the vehicle and the AP in the zth sub-area is ap :

[0060]

[0061] where w ap is the bandwidth between the target vehicle and the AP; p ap is the transmit power of the AP; N0 is the Gaussian white noise power.

[0062] During the network switching process, TAU and LP record and analyze the corresponding signaling quantity, type and timing information such as switching request, switching command and switching confirmation to evaluate the signaling overhead of the entire network, which is also the main source of switching cost. In the above high-speed moving target vehicle scenario, the total switching cost is expressed as C tot :

[0063]

[0064] Among them C tau and C lp are the related message overheads of TAU and LP, ap′ indicates the current access network of the target vehicle, and ap indicates the target network to be switched. λ is the rate of triggering the TAU process. α is the rate of triggering the LP process. After triggering the paging process, the MME waits to receive the expected paging response within a predefined time limit. If the vehicle fails to respond to the first incoming paging message in time, the MME starts sending multiple paging attempts, which are captured by α. When there is an incoming data packet for a vehicle, the MME always triggers the required paging process, which is captured by σ. a(t) is the indicator function:

[0065]

[0066] (2) Combine the transmission rate and switching cost to construct the optimization objective function.

[0067] The present invention uses the network unit as the selection range for vehicle network switching, selects the target network AP with the largest transmission rate within this range, and combines the network switching cost to meet the business needs. Therefore, the utility function is:

[0068]

[0069] Wherein, ap is the target network to be switched, i represents the i-th switching, i∈I, I represents the total number of switching times in a network unit, w1, w2 are the weights corresponding to different service types, and the specific values ​​are shown in Table 2.

[0070] Table 2 Weights corresponding to different business types

[0071] Business Type <![CDATA[w1]]> <![CDATA[w2]]> Transmission rate sensitive services 1 -0.1 Switching cost-sensitive business 0.1 -1 Simple business 0.5 -0.5

[0072] The network access decision for each time slot will affect the service transmission rate and the size of the switching cost. The optimization goal of the present invention is to maximize the utility function, the optimization factor is the selection set of the access network, and the optimization goal can be expressed as formula P1:

[0073]

[0074]

[0075]

[0076] C3:0<P ap <max{P ap}(7d)

[0077] Among them, R(z) ap represents the transmission rate between the vehicle and the AP, C tot represents the total switching cost, Indicates the lower limit of the transmission rate of different service types. Indicates the upper limit of switching cost. type indicates the service type, and type1, type2, and type3 respectively represent transmission rate-sensitive services, switching cost-sensitive services, and simple services. In general, P1 is the optimization goal, which means that the total utility function is maximized after the vehicle passes through a network unit; C1 and C2 respectively constrain the minimum task transmission rate and maximum network switching cost of the target vehicle; C3 constrains the transmission power of the AP.

[0078] Step 3: Design a deep learning network switching algorithm based on DQN (Deep Q-Network) in heterogeneous network units:

[0079] The purpose of this step is to reshape the reward function by combining the transmission rate and the switching cost to maximize the optimization target to update the experience value and realize dynamic network switching decisions.

[0080] According to the proposed utility function and optimization goal, this paper designs a deep learning network switching algorithm based on DQN (Deep Q-Network), and rephrases the optimization goal in Markov decision process, reshapes the reward function r(t) based on the original decision process, which is specifically expressed as follows:

[0081] (1) System state space s(t): The state space of the heterogeneous network at time slot t, including vehicle speed, location, transmission service type, and task transmission rate of the current vehicle connected to the network;

[0082] (2) Action space a(t): The action space of the heterogeneous network in time slot t, which refers to the selection of accessible networks, including 4GBS, 5G BS, and RSU, i.e.

[0083] (3) Reward function r(t): Reshape the reward function and the data transmission rate R(z) of the target vehicle ap and switching cost C tot Related, expressed as:

[0084]

[0085] The specific use process of the present invention is: see Figure 4 , in the process of sample collection, first randomly select an initial state s(t), then select the execution action based on this state, set the parameter ε=0.01, ε∈[0,1], when s(t)∈[0,1-ε], randomly select an action in the action space a(t); otherwise, select a corresponding action with the largest Q value in the action space through Q-Network. Since the relevant parameters in the initial Q-Network are random, ε is usually set very small before the experience pool is full, that is, the initial action is basically randomly selected. After the action selection is completed, the agent will execute this action in the environment, and then the environment will return the next state s(t)_ and reward r(t). At this time, (s(t), a(t), r(t), s(t)_) is stored in the experience pool. Next, the next state s(t)_ is regarded as the current state s(t), and the above steps are repeated until the experience pool is full.

[0086] When the experience pool is full, the network in DQN starts to update. It starts to randomly sample from the experience pool, i.e. the experience replay mechanism, and sends the sampled reward r(t) and the next state s(t)_ to the target network to calculate the Q_value, and then to the Q-Network to calculate the loss value, and starts to update the Q-Network, and repeats until the Q-Network completes convergence, i.e. the reward function gradually converges and tends to be stable, indicating that the optimal network switching strategy has been selected.

[0087] A specific simulation example will be provided below to illustrate the present invention in detail:

[0088] In the experiment, a straight road with a length of 1000m is considered, which is the length of the road covered by a network unit. The network unit consists of 1 4G BS, 2 5G BS, and 4 RSUs. The target vehicle travels at a constant speed of 90km / h. The specific parameter settings of the base station, such as coverage range and bandwidth, are shown in Table 3.

[0089] Table 3DQN network switching algorithm simulation parameters

[0090]

[0091] For different service types, corresponding transmission rate and switching cost thresholds are set. The specific requirements are shown in Table 4.

[0092] Table 4 Transmission rate lower limit and switching cost upper limit requirements for different service types

[0093] Business Type Transfer rate Switching costs Transmission rate sensitive services 20Gbit / s 292.50M / s Switching cost-sensitive business 1Mbit / s 87.75M / s Simple business 20Mbit / s 107.25M / s

[0094] In the experiment, the vehicle access network selection process uses the reshaped reward function and experience replay mechanism to improve the learning method, and finally obtains the operation method of high-speed vehicle intelligent network switching based on rate and cost required by specific application scenarios. Figure 5 As shown in line 1, the average reward during the training phase eventually converges and becomes stable. Figure 5 In the figure, line 1 is the method proposed by the present invention, which reshapes the reward function by combining the transmission rate and the switching cost, and line 2 is the convergence of the reward function that only considers the transmission rate. Figure 5 It can be seen that at the beginning of line1 training, with the continuous optimization of network parameters, the reward function shows an upward trend. Before 2000 rounds of training, the reward function fluctuates greatly, because the relevant parameters in the initial Q-Network are random, so before the experience pool is full, the actions are basically randomly selected, so the fluctuation is large. In the later stage of training, the experience pool is full, and the network parameters in DQN begin to be updated, that is, random sampling from the experience pool begins. When the training round reaches 4000, the reward gradually stabilizes and converges. In contrast, line2 gradually stabilizes after the training round reaches 8000, and the reward function value of line1 is also higher than that of line2 when it finally converges. Therefore, the method provided by the present invention can select the optimal switching strategy faster and better.

[0095] Figure 6 It shows that when the vehicle travels 1 km, services one, two, and three are randomly accessed with equal probability, and the corresponding average switching frequencies of heterogeneous networks are: 0.0653 times / second, 0.0246 times / second, and 0.0299 times / second. This is because service one is sensitive to transmission rate, and high-frequency network switching is required to meet the transmission rate requirements. Service two is sensitive to switching cost, and the switching cost is reduced by reducing the frequency, which meets the actual application requirements. The number of switching times and switching frequency of service three are in the middle. It can be obtained from the switching frequency that the number of network switching caused by the three services in a 1-kilometer driving distance is no more than 2 times. Compared with the switching algorithm that only uses the rate requirement as the switching condition, the number of switching is reduced by 1 time.

[0096] The above is an explanation of the specific implementation of the present invention, rather than a limitation of the present invention. Those skilled in the relevant technical field can also make various equivalent technical solutions without departing from the scope of the present invention, so all equivalent technical solutions should be included in the patent protection scope of the present invention.

Claims

1. A high-speed vehicle intelligent network switching method based on rate and cost, comprising the following steps: Step 1: Based on heterogeneous network units, high-speed moving target vehicle scenarios under different types of business requirements are constructed and relevant network parameters are obtained; Step 2: Calculate the data transmission rate and network switching cost, and determine the optimization target; (1) Calculate the data transmission rate and network switching cost of high-speed moving target vehicles; (2) Combine the transmission rate and switching cost to construct an optimization objective function; Step 3: Design a network switching algorithm based on DQN in heterogeneous network units; In the step 1, the heterogeneous network unit is composed of k 4G BSs, l 5G BSs and m RSUs, where k = {0, 1}, 0≤l≤L, 0≤m≤M, l, m∈N*; In the step 1, the service type is a transmission rate sensitive service, a switching cost sensitive service, and a simple service; For high-definition video transmission or large-scale data upload, this type of business is transmission rate-sensitive; for low-latency emergency business, this type of business is switching cost-sensitive; and the business in between is simple business; In the step 1, the network parameters are the vehicle's location information and network switching signaling overhead; In step 2 (1), in the high-speed moving target vehicle scenario, the total switching cost is expressed as C tot : Among them C tau and C lp are the related message overheads of TAU and LP, ap′ represents the current access network of the target vehicle, ap represents the target network to be switched, λ is the rate of triggering the TAU process, α is the rate of triggering the LP process, and a(t) is the indicator function, which can be expressed as σ refers to the rate at which the paging process is triggered.

2. A high-speed vehicle intelligent network switching method based on rate and cost according to claim 1, characterized in that: In step 2 (2), the network with the maximum transmission rate is selected as the accessible target network, and the network switching cost is combined to construct the utility function: Where ap is the target network to be switched, i represents the i-th switch, i∈I, I represents the total number of switches in a network unit, and w1,w2 are the weights corresponding to different service types; The optimization goal is: Among them, R(z) ap represents the transmission rate between the vehicle and the AP, C tot represents the total switching cost, Indicates the lower limit of the transmission rate of different service types. Represents the upper limit of switching cost, type represents the service type, type1, type2, and type3 represent transmission rate sensitive services, switching cost sensitive services, and simple services, respectively. Generally speaking, P1 is the optimization goal, which means that the total utility function is maximized after the vehicle passes through a network unit; C1 and C2 respectively constrain the minimum task transmission rate and maximum network switching cost of the target vehicle; C3 constrains the transmission power of the AP.

3. A high-speed vehicle intelligent network switching method based on rate and cost according to claim 1 or 2, characterized in that: In step 3, in the network switching algorithm designed based on DQN, the reward function is expressed as: Wherein, I is the total number of switching in a network unit, i is the i-th switching, and i≤I.

Images