A base station handover method and device for a dual connectivity mobile communication system

Abstract

The application relates to a base station switching method and device for a dual-connection mobile communication system, and belongs to the technical field of wireless communication. The method solves the problem that the existing base station switching method is not sensitive to environmental changes in complex application scenarios, and the base station switching is not timely. The method comprises the following steps: modeling a dual-connection heterogeneous access network, a base station switching process and a transmission rate of the dual-connection mobile communication system to establish a base station switching problem; establishing a noisy double deep Q network, and training the noisy double deep Q network by using base station switching experiences stored in a replay memory to obtain a base station switching decision model; and configuring the base station switching decision model in a base station controller of the access network, so that the base station controller solves the base station switching problem by using the base station switching decision model according to an actual environment, makes a base station switching decision, and continuously updates a base station switching strategy according to a feedback result. The NDDQN is used for sensing a communication environment and a mobile user state to make a base station switching decision.

Description

Technical Field

[0001] This invention relates to the field of wireless communication technology, and in particular to a base station handover method and apparatus for dual-connectivity mobile communication systems. Background Technology

[0002] Future mobile communication systems will face more complex application scenarios such as telemedicine, autonomous driving, and smart cities, requiring the transmission of 4K high-definition video, VR / AR data, or control data to remote locations. This necessitates wireless links that can provide ultra-high-speed and highly reliable wireless communication services. While 5G millimeter-wave communication offers massive bandwidth and capacity to meet ultra-high-speed communication demands, severe path attenuation, the mobility of mobile communication systems, and numerous obstructions in urban environments make 5G millimeter-wave communication prone to interruptions. To address this issue, this invention proposes using a dual-connectivity 5G link to provide services to the mobile communication system. This involves the mobile communication system simultaneously accessing both 5G millimeter-wave and microwave links, forming a heterogeneous wireless access network. Simultaneously, two base stations (one millimeter-wave base station and one microwave base station) provide wireless communication services to the mobile communication system.

[0003] However, current base station handover methods used in cellular networks are all designed for single-connectivity mode, where users are connected to only one base station. There is still no mature base station handover method for dual-connectivity mode. Furthermore, current base station handover methods rely on the user's received reference signal strength. Due to delays in signal transmission and processing, these methods are insensitive to environmental changes in complex future application scenarios, leading to untimely base station handovers and a decline in wireless communication performance. Summary of the Invention

[0004] Based on the above analysis, the embodiments of the present invention aim to provide a base station handover method and apparatus for dual-connectivity mobile communication systems, in order to solve the problem that existing base station handover methods are not sensitive to environmental changes in complex application scenarios, which leads to untimely base station handover and a decline in wireless communication performance.

[0005] On one hand, embodiments of the present invention provide a base station handover method for a dual-connectivity mobile communication system, comprising: modeling the dual-connectivity heterogeneous access network, base station handover process, and transmission rate of the dual-connectivity mobile communication system to establish the base station handover problem of the dual-connectivity mobile communication system; establishing a noisy dual-deep Q network and training the noisy dual-deep Q network using base station handover experiences stored in a playback memory to obtain a base station handover decision model; and configuring the base station handover decision model in the base station controller of the access network, so that the base station controller solves the base station handover problem and makes a base station handover decision based on the actual environment using the base station handover decision model, and continuously updates the base station handover strategy based on feedback results.

[0006] The beneficial effects of the above technical solution are as follows: By using artificial intelligence technology to add noise to dual deep Q network (NDDQN), the communication environment and mobile user status are perceived, and base station handover decisions are made accordingly to obtain performance benefits. The base station handover strategy is further optimized, and the access network base station handover method is self-updated, ensuring that continuous high-quality communication services are provided to mobile users.

[0007] Further improvements to the above method include modeling the dual-connectivity heterogeneous access network, base station handover process, and transmission rate of the dual-connectivity mobile communication system. This includes: modeling the dual-connectivity heterogeneous access network, where microwave and millimeter-wave base stations simultaneously provide wireless access services to the mobile communication system. The millimeter-wave base station can only communicate with the microwave base station within its macrocell. The microwave base station, acting as the primary base station, provides control signal and data packet transmission services to the mobile communication system through the control plane and user plane, respectively. The millimeter-wave base station, acting as the secondary base station, provides data packet transmission services to the mobile communication system through the user plane. Modeling the base station handover process in the dual-connectivity mode, which includes intra-macrocell handover or inter-macrocell handover. Modeling the transmission rate in the dual-connectivity mode, where the transmission rate of the mobile communication system at time t in dual-connectivity mode is expressed by the following formula:

[0008] Case 1 indicates that the secondary base station is a millimeter-wave base station, which is the dominant case; Case 2 indicates that the secondary base station is a target microwave base station, which occurs during the intermediate state of intercellular handover. and These represent the distances between the primary and secondary base stations and the mobile user, respectively. and These are the locations of the primary and secondary base stations currently in service, respectively. q(t) represents the user's location at time t, and r... M (d) represents the microwave communication transmission rate of the mobile communication system, r S (d) represents the millimeter-wave transmission rate of the mobile communication system.

[0009] Based on a further improvement of the above method, the microwave communication transmission rate r of the mobile communication system is calculated using the following formula. M (d):

[0010] r M (d)=B M log2(1+Γ M (d));

[0011]

[0012] hM (d)=α M +10β M log 10 (d)+η M +ξ;

[0013] Where d is the distance between the mobile user and the base station, α M and β M Representing the microwave communication path attenuation factor and exponent, respectively, η M Representing random small-scale fading in microwave communication, ξ is the random shadowing effect, and p M Γ represents the transmit power allocated by the microwave base station to the mobile user. M (d) represents the signal-to-noise ratio of the microwave link, B M N is the bandwidth allocated by the microwave base station to mobile users. M The power spectral density of Gaussian white noise in microwave communication, h M (d) represents the channel attenuation of the microwave link; the millimeter-wave transmission rate r of the mobile communication system is calculated using the following formula. S (d):

[0014] r S (d)=B S log2(1+Γ S (d));

[0015]

[0016] h S (d)=α S +10β S log 10 (d)+η S +ζ;

[0017] Where, α S and β S η represents the path attenuation factor and exponent for millimeter-wave communication, respectively. S p represents random small-scale fading in millimeter-wave communication S Γs(d) represents the transmit power allocated by the millimeter-wave base station to the mobile user, and Γs(d) represents the signal-to-noise ratio of the millimeter-wave link; B S N is the bandwidth allocated to mobile users by millimeter-wave base stations. S The power spectral density of Gaussian white noise, h, represents millimeter-wave communication. S (d) represents the channel attenuation of the millimeter-wave link.

[0018] Based on a further improvement of the above method, the intra-macrocell handover includes: handover between two millimeter-wave base stations within the same macrocell under the control of the microwave base station corresponding to the macrocell, wherein, during the intra-macrocell handover process, a handover protocol is used to first establish a connection between the mobile user and the target millimeter-wave base station and then disconnect the connection between the mobile user and the serving millimeter-wave base station.

[0019] Based on a further improvement of the above method, the cross-macrocell handover includes: handover between two millimeter-wave base stations in different cells, including handover between a millimeter-wave base station and a microwave base station, wherein the handover between a millimeter-wave base station and a microwave base station includes changing the secondary base station from the serving millimeter-wave base station to the first target microwave base station to enter a first intermediate state; and the handover between a microwave base station and a millimeter-wave base station includes: when the target millimeter-wave base station is in the original serving cell, switching the secondary base station from the first target microwave base station to the target millimeter-wave base station; when the target millimeter-wave base station is in the original target cell, changing the primary base station from the serving microwave base station to the first target microwave base station and sending a control command to the target millimeter-wave base station, and then switching the secondary base station from the first microwave base station to the target millimeter-wave base station to complete the user plane handover; when neither the primary base station nor the first target microwave base station can control the target millimeter-wave base station, switching the secondary base station to the target microwave base station to enter a second intermediate state, wherein the second intermediate state is different from the first intermediate state.

[0020] Based on further improvements to the above method, the base station handover problem of the dual-connectivity mobile communication system includes: the purpose of the base station handover problem is to maximize the transmission rate.

[0021]

[0022]

[0023] in, b represents the average transmission rate of a mobile user over a period of time. T (t) represents the target millimeter-wave base station. For macrocells k The set of millimeter-wave base stations within the region, where t is a discrete time point.

[0024] Further improvements to the above method lead to the establishment of a noisy dual-depth Q-network, including:

[0025] A multi-step bootstrapping method is used to obtain the target Q value, so that the loss function is expressed as:

[0026]

[0027]

[0028] S(t+n) and S(t) are the system states at times t+n and t, respectively, A(t) is the action at time t, {R(t+l)|l=0,1,…,n-1} is the set of reward values ​​from time t to time t+n-1, γ∈[0,1] is the discount factor, and θ and These represent the parameters of the current Q-network and the target Q-network, respectively. and These are the target Q value and the current Q value, respectively.

[0029] By employing dual DQN technology to decouple the Q-network used for target evaluation and action selection when calculating the target Q-value, the loss function of multi-step dual DQN becomes:

[0030]

[0031] Among them, R (n) (t), S(t+n), S(t), and A(t) are replaced with R respectively. (n) S (n) S and A;

[0032] The standard linear layer y = Wx + b in a DNN is transformed into a noisy linear layer using a noisy network technique:

[0033] y = (μ W +σ W ⊙∈ W )x+μ b +σ b ⊙∈ b ;

[0034] Where μ W σ W μ b and σ b All are learnable parameters, μ W and σ W μ represents the weights of the neural network after adding noise. b and σ b For the bias of the neural network after adding noise, ∈ W and ∈ b is a Gaussian noise variable, and ⊙ represents the multiplication operation of matrix factors.

[0035] On the other hand, embodiments of the present invention provide a base station handover device for a dual-connectivity mobile communication system, comprising: a base station handover problem acquisition module, used to model the dual-connectivity heterogeneous access network, base station handover process, and transmission rate of the dual-connectivity mobile communication system, and establish the base station handover problem of the dual-connectivity mobile communication system, wherein the base station handover process includes intra-macrocell handover and inter-macrocell handover; a base station handover problem solving model, used to establish a noisy dual-deep Q network and train the noisy dual-deep Q network using base station handover experiences stored in a playback memory to obtain a base station handover decision model, wherein the base station handover problem is solved by the base station handover decision model; and a base station handover module, used to configure the base station handover decision model in the base station controller of the access network, so that the base station controller makes base station handover decisions according to the actual environment and continuously updates the base station handover strategy according to the feedback results.

[0036] Based on further improvements to the above-mentioned device, the base station handover problem acquisition module includes an access network model establishment module, a base station handover model establishment module, and a transmission rate model establishment module. The access network model establishment module is used to model the dual-connectivity heterogeneous access network, providing wireless access services to the mobile communication system simultaneously through microwave base stations and millimeter-wave base stations. The millimeter-wave base station can only communicate with the microwave base station within its macrocell. The microwave base station, acting as the primary base station, provides control signal and data packet transmission services to the mobile communication system through the control plane and user plane, respectively. The millimeter-wave base station, acting as the secondary base station, provides data packet transmission services to the mobile communication system through the user plane. The base station handover model establishment module is used to model the base station handover process in the dual-connectivity mode, including intra-macrocell handover and inter-macrocell handover. The transmission rate model establishment module is used to model the transmission rate of the dual-connectivity mode, where the transmission rate of the mobile communication system at time t in dual-connectivity mode is expressed by the following formula:

[0037]

[0038] Case 1 indicates that the secondary base station is a millimeter-wave base station, which is the dominant case; Case 2 indicates that the secondary base station is a target microwave base station, which occurs during the intermediate state of intercellular handover. and These represent the distances between the primary and secondary base stations and the mobile user, respectively. and These are the locations of the primary and secondary base stations currently in service, respectively. q(t) represents the user's location at time t, and r... M (d) represents the microwave communication transmission rate of the mobile communication system, r S(d) represents the millimeter-wave transmission rate of the mobile communication system.

[0039] Based on further improvements to the above-mentioned device, the transmission rate model establishment module is also used for:

[0040] The microwave communication transmission rate r of the mobile communication system is calculated using the following formula. M (d):

[0041] r M (d)=B M log2(1+Γ M (d));

[0042]

[0043] h M (d)=α M +10β M log 10 (d)+η M +ξ;

[0044] Where d is the distance between the mobile user and the base station, α M and β M Representing the microwave communication path attenuation factor and exponent, respectively, η M Representing random small-scale fading in microwave communication, ξ is the random shadowing effect, and p M Γ represents the transmit power allocated by the microwave base station to the mobile user. M (d) represents the signal-to-noise ratio of the microwave link, B M N is the bandwidth allocated by the microwave base station to mobile users. M The power spectral density of Gaussian white noise in microwave communication, h M (d) represents the channel attenuation of the microwave link;

[0045] The millimeter-wave transmission rate r of the mobile communication system is calculated using the following formula. S (d):

[0046] r S (d)=B S log2(1+Γ S (d));

[0047]

[0048] h S (d)=α S +10β S log 10 (d)+η S +ζ;

[0049] Where, αS and β S η represents the path attenuation factor and exponent for millimeter-wave communication, respectively. S p represents random small-scale fading in millimeter-wave communication S Γs(d) represents the transmit power allocated by the millimeter-wave base station to the mobile user, and Γs(d) represents the signal-to-noise ratio of the millimeter-wave link; B S N is the bandwidth allocated to mobile users by millimeter-wave base stations. S The power spectral density of Gaussian white noise, h, represents millimeter-wave communication. S (d) represents the channel attenuation of the millimeter-wave link.

[0050] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0051] 1. Using artificial intelligence technology, a noisy dual deep Q network (NDDQN) is used to perceive the communication environment and the status of mobile users, and make base station handover decisions accordingly to obtain performance benefits. This further optimizes the base station handover strategy, realizes the self-updating of the access network base station handover method, and ensures that continuous high-quality communication services are provided to mobile users.

[0052] 2. In order to obtain a better-performing dual-connectivity base station handover strategy, this invention uses multi-step learning technology and dual DQN technology to change the loss function to improve the learning ability of the original DQN and uses noisy network technology to convert the standard linear layer into a noisy linear layer to effectively improve the exploration performance of the original DQN and proposes a corresponding NDDQN base station handover algorithm.

[0053] 3. Dual-connectivity base station handover will primarily utilize millimeter-wave base station handover, with the main consideration being the communication status of each millimeter-wave base station. Microwave base station handover will only occur when millimeter-wave base station handover occurs between different macrocells, ensuring the correspondence between millimeter-wave and microwave base stations and the orderly handover of heterogeneous dual-connectivity base stations.

[0054] 4. To avoid interruptions during the handover process, this embodiment of the invention uses the establish-before-disconnect (MBB) handover protocol, that is, during handover, the mobile user does not disconnect from the previous serving base station before establishing a connection with the new base station.

[0055] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0056] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0057] Figure 1 This is a flowchart of a base station handover method for a dual-connectivity mobile communication system according to an embodiment of the present invention;

[0058] Figure 2 This is a schematic diagram of the base station handover process within a cell according to an embodiment of the present invention;

[0059] Figure 3 This is a schematic diagram of cross-cellular base station handover according to an embodiment of the present invention;

[0060] Figure 4 This is a schematic diagram illustrating multi-step learning according to an embodiment of the present invention;

[0061] Figure 5 This is a schematic diagram illustrating the calculation of the target Q value using dual DQN according to an embodiment of the present invention;

[0062] Figure 6 This is a diagram illustrating the input-output relationship of a noisy neural network layer according to an embodiment of the present invention.

[0063] Figure 7 This is an NDDQN base station handover strategy training method according to an embodiment of the present invention;

[0064] Figure 8 This is a comparison of the training processes of the NDDQN base station handover method and the DQN method;

[0065] Figure 9 This is a comparison of the transmission rates of the NDDQN base station handover method with two other methods;

[0066] Figure 10 This is a block diagram of a base station handover device for a dual-connectivity mobile communication system according to an embodiment of the present invention. Detailed Implementation

[0067] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0068] refer to Figure 1A specific embodiment of the present invention discloses a base station handover method for a dual-connectivity mobile communication system, comprising: in step S101, modeling the dual-connectivity heterogeneous access network, base station handover process, and transmission rate of the dual-connectivity mobile communication system to establish the base station handover problem of the dual-connectivity mobile communication system; in step S102, establishing a noisy dual-deep Q network and training the noisy dual-deep Q network using base station handover experiences stored in a playback memory to obtain a base station handover decision model; and in step S103, configuring the base station handover decision model in the base station controller of the access network, so that the base station controller solves the base station handover problem and makes a base station handover decision based on the actual environment through the base station handover decision model, and continuously updates the base station handover strategy based on the feedback results.

[0069] Compared with existing technologies, the base station handover method for dual-connectivity mobile communication systems provided in this embodiment uses artificial intelligence technology, namely, a noisy dual-deep Q network (NDDQN), to perceive the communication environment and the status of mobile users, and make base station handover decisions accordingly to obtain performance benefits, further optimize the base station handover strategy, realize the self-updating of the access network base station handover method, and ensure that continuous high-quality communication services are provided to mobile users.

[0070] The following text will refer to Figure 1 The various steps of the base station handover method for a dual-connectivity mobile communication system according to embodiments of the present invention will be described in detail.

[0071] In step S101, the dual-connectivity heterogeneous access network, base station handover process, and transmission rate of the dual-connectivity mobile communication system are modeled to establish the base station handover problem of the dual-connectivity mobile communication system.

[0072] Specifically, modeling the dual-connectivity heterogeneous access network, base station handover process, and transmission rate of the dual-connectivity mobile communication system includes: modeling the dual-connectivity heterogeneous access network, where microwave base stations and millimeter-wave base stations simultaneously provide wireless access services to the mobile communication system. The millimeter-wave base station can only communicate with the microwave base station within its macrocell. The microwave base station, acting as the primary base station, provides control signal and data packet transmission services to the mobile communication system through the control plane and user plane, respectively. The millimeter-wave base station, acting as the secondary base station, provides data packet transmission services to the mobile communication system through the user plane. Modeling the base station handover process in dual-connectivity mode, which includes intra-macrocell handover or inter-macrocell handover. Modeling the transmission rate in dual-connectivity mode, where the transmission rate of the mobile communication system at time t in dual-connectivity mode is expressed by the following formula:

[0073]

[0074] Case 1 indicates that the secondary base station is a millimeter-wave base station, which is the dominant case; Case 2 indicates that the secondary base station is a target microwave base station, which occurs during the intermediate state of intercellular handover. and These represent the distances between the primary and secondary base stations and the mobile user, respectively. and These are the locations of the primary and secondary base stations currently in service, respectively. q(t) represents the user's location at time t, and r... M (d) represents the microwave communication transmission rate of the mobile communication system, r S (d) represents the millimeter-wave transmission rate of the mobile communication system.

[0075] The microwave communication transmission rate r of a mobile communication system is calculated using the following formula. M (d):

[0076] r M (d)=B M log2(1+Γ M (d));

[0077]

[0078] h M (d)=α M +10β M log 10 (d)+η M +ξ;

[0079] Where d is the distance between the mobile user and the base station, and α M and β M Representing the microwave communication path attenuation factor and exponent, respectively, η M Representing random small-scale fading in microwave communication, ξ is the random shadowing effect, and p M Γ represents the transmit power allocated by the microwave base station to mobile users. M (d) represents the signal-to-noise ratio of the microwave link, B M N is the bandwidth allocated by the microwave base station to mobile users. M The power spectral density of Gaussian white noise in microwave communication, h M (d) represents the channel attenuation of the microwave link;

[0080] The millimeter-wave transmission rate r of a mobile communication system is calculated using the following formula. S (d):

[0081] r S (d)=B S log2(1+Γ S (d));

[0082]

[0083] h S (d)=α S +10β S log 10 (d)+η S +ζ;

[0084] Where, α S and β S η represents the path attenuation factor and exponent for millimeter-wave communication, respectively. S p represents random small-scale fading in millimeter-wave communication S Γs(d) represents the transmit power allocated to mobile users by the millimeter-wave base station, and Γs(d) represents the signal-to-noise ratio of the millimeter-wave link; B S N is the bandwidth allocated to mobile users by millimeter-wave base stations. S The power spectral density of Gaussian white noise, h, represents millimeter-wave communication. S (d) represents the channel attenuation of the millimeter-wave link.

[0085] Macrocell intra-cell handover includes: handover between two millimeter-wave base stations within the same macrocell under the control of the microwave base station corresponding to the macrocell. During the macrocell intra-cell handover process, a handover protocol is used to first establish a connection between the mobile user and the target millimeter-wave base station, and then disconnect the connection between the mobile user and the serving millimeter-wave base station.

[0086] Cross-macrocell handover includes: handover between two millimeter-wave base stations in different cells, including handover between a millimeter-wave base station and a microwave base station, and handover between two millimeter-wave base stations. The handover between a millimeter-wave base station and a microwave base station includes: switching the secondary base station from the serving millimeter-wave base station to the first target microwave base station to enter a first intermediate state; and the handover between a microwave base station and a millimeter-wave base station includes: when the target millimeter-wave base station is within the original serving cell, switching the secondary base station from the first target microwave base station to the target millimeter-wave base station; when the target millimeter-wave base station is within the original target cell, switching the primary base station from the serving microwave base station to the first target microwave base station and sending control commands to the target millimeter-wave base station, and then switching the secondary base station from the first microwave base station to the target millimeter-wave base station to complete the user plane handover; when neither the primary base station nor the first target microwave base station can control the target millimeter-wave base station, switching the secondary base station to the target microwave base station to enter a second intermediate state, wherein the second intermediate state is different from the first intermediate state.

[0087] The base station handover issue in establishing a dual-connectivity mobile communication system aims to maximize the transmission rate.

[0088]

[0089]

[0090] in, b represents the average transmission rate of a mobile user over a period of time. T (t) represents the target millimeter-wave base station. For macrocells k The set of millimeter-wave base stations within the region, where t is a discrete time point.

[0091] In step S102, a noisy dual-deep Q network is established and trained using base station handover experiences stored in the playback memory to obtain a base station handover decision model. Specifically, a base station handover experience refers to a set of states, actions, and rewards obtained by performing different actions (i.e., handover to different base stations) under different environmental conditions and receiving corresponding rewards (i.e., transmission rates).

[0092] Building a noisy dual-deep Q-network includes:

[0093] (1) Multi-step learning technique: Using a multi-step bootstrapping method to obtain the target Q value, the loss function is expressed as:

[0094]

[0095]

[0096] S(t+n) and S(t) are the system states at times t+n and t, respectively, A(t) is the action at time t, {R(t+l)|l=0,1,…,n-1} is the set of reward values ​​from time t to time t+n-1, γ∈[0,1] is the discount factor, and θ and These represent the parameters of the current Q-network and the target Q-network, respectively. and These are the target Q value and the current Q value, respectively.

[0097] (2) Dual DQN technique: The dual DQN technique is used to decouple the Q-network used for target evaluation and action selection when calculating the target Q-value, so that the loss function of multi-step dual DQN becomes:

[0098]

[0099] Among them, R (n) (t), S(t+n), S(t), and A(t) are replaced with R respectively. (n) S (n) S and A;

[0100] (3) Noisy Network Technique: The standard linear layer y = Wx + b in a DNN is transformed into a noisy linear layer using the noisy network technique.

[0101] y = (μ W +σ W ⊙∈ W )x+μ b +σ b ⊙∈ b ;

[0102] Where μ W σ W μ b and σ b All are learnable parameters, μ W and σ W μ represents the weights of the neural network after adding noise. b and σ b For the bias of the neural network after adding noise, ∈ W and ∈ b is a Gaussian noise variable, and ⊙ represents the multiplication operation of matrix factors.

[0103] Figure 8 The following table shows the performance comparison results of NDDQN and the original DQN in base station handover strategy learning according to an embodiment of the present invention, that is, the average cumulative reward obtained by the two base station handover strategy models within a specified number of learning episodes. Figure 8 The upper and lower parts correspond to the performance under different millimeter-wave obstruction conditions. (Reference) Figure 9 After improvements to multi-step learning, dual DQN, and noisy networks in the embodiments of the present invention, the learning performance of NDDQN is significantly better than that of the original DQN, proving the effectiveness of the improved technology in the embodiments of the present invention.

[0104] Figure 9 The paper presents a comparison of the communication performance of the NDDQN and UCB algorithms and the original DQN under base station handover strategies according to embodiments of the present invention. Specifically, it shows the average transmission rate of the mobile communication system under three base station handover strategies with different numbers of base stations (for simplicity, the number of millimeter-wave base stations is set to be ten times that of microwave base stations in this simulation). Figure 9 The upper and lower parts correspond to the performance under different millimeter-wave obstruction conditions. (Reference) Figure 9 The NDDQN implementation of this invention significantly outperforms UCB and the original DQN, demonstrating the technical superiority of this invention.

[0105] In step S103, the base station handover decision model is configured in the base station controller of the access network, so that the base station controller can solve the base station handover problem and make a base station handover decision based on the actual environment through the base station handover decision model, and continuously update the base station handover strategy based on the feedback results.

[0106] Specifically, the base station handover decision model is a deep neural network that can be configured in the access network base station control unit with an embedded AI chip. The actual environmental conditions (in this embodiment, these conditions include the mobile user's geographical location, movement speed, millimeter-wave channel obstruction, and base station geographical location) are input into the model. The deep neural network calculates the long-term value of each action (i.e., selecting the base station to access), selects the action (base station) with the highest value, and then hands over to the corresponding base station. The actual reward corresponding to the action (e.g., setting the actual reward as the transmission rate) is then obtained, and together with the aforementioned state and action, constitutes a new set of base station handover experiences. This experience is added to the backhaul memory for further updates and training of the base station handover strategy model.

[0107] refer to Figure 10 A specific embodiment of the present invention discloses a base station handover device for a dual-connectivity mobile communication system, comprising: a base station handover problem acquisition module 1001, used to model the dual-connectivity heterogeneous access network, base station handover process, and transmission rate of the dual-connectivity mobile communication system, and to establish a base station handover problem for the dual-connectivity mobile communication system, wherein the base station handover process includes intra-macrocell handover and inter-macrocell handover; a base station handover problem solving model 1002, used to establish a noisy dual-deep Q network and train the noisy dual-deep Q network using base station handover experiences stored in a playback memory to obtain a base station handover decision model, wherein the base station handover problem is solved through the base station handover decision model; and a base station handover module 1003, used to configure the base station handover decision model in the base station controller of the access network, so that the base station controller makes base station handover decisions according to the actual environment and continuously updates the base station handover strategy according to the feedback results.

[0108] The base station handover problem acquisition module includes an access network model establishment module, a base station handover model establishment module, and a transmission rate model establishment module. The access network model establishment module models the dual-connectivity heterogeneous access network, providing wireless access services to the mobile communication system simultaneously through microwave and millimeter-wave base stations. The millimeter-wave base station can only communicate with the microwave base station within its macrocell. The microwave base station, acting as the primary base station, provides control signals and data packets to the mobile communication system through the control plane and user plane, respectively. The millimeter-wave base station, acting as the secondary base station, provides data packet transmission services to the mobile communication system through the user plane. The base station handover model establishment module models the base station handover process in dual-connectivity mode, including intra-macrocell handover and inter-macrocell handover. The transmission rate model establishment module models the transmission rate in dual-connectivity mode, where the transmission rate of the mobile communication system at time t in dual-connectivity mode is expressed by the following formula:

[0109]

[0110] Case 1 indicates that the secondary base station is a millimeter-wave base station, which is the dominant case; Case 2 indicates that the secondary base station is a target microwave base station, which occurs during the intermediate state of intercellular handover. and These represent the distances between the primary and secondary base stations and the mobile user, respectively. and These are the locations of the primary and secondary base stations currently in service, respectively. q(t) represents the user's location at time t, and r... M (d) represents the microwave communication transmission rate of the mobile communication system, r S (d) represents the millimeter-wave transmission rate of the mobile communication system.

[0111] The transmission rate model building module is also used to: calculate the microwave communication transmission rate r of the mobile communication system using the following formula. M (d):

[0112] r M (d)=B M log2(1+Γ M (d));

[0113]

[0114] h M (d)=α M +10β M log 10 (d)+η M +ξ;

[0115] Where d is the distance between the mobile user and the base station, and α M and β M Representing the microwave communication path attenuation factor and exponent, respectively, η M Representing random small-scale fading in microwave communication, ξ is the random shadowing effect, and p M Γ represents the transmit power allocated by the microwave base station to mobile users. M (d) represents the signal-to-noise ratio of the microwave link, B M N is the bandwidth allocated by the microwave base station to mobile users. M The power spectral density of Gaussian white noise in microwave communication, h M (d) represents the channel attenuation of the microwave link;

[0116] The millimeter-wave transmission rate r of a mobile communication system is calculated using the following formula. S (d):

[0117] r S (d)=BS log2(1+Γ S (d));

[0118]

[0119] h S (d)=α S +10β S log 10 (d)+η S +ζ;

[0120] Where, α S and β S η represents the path attenuation factor and exponent for millimeter-wave communication, respectively. S p represents random small-scale fading in millimeter-wave communication S Γs(d) represents the transmit power allocated to mobile users by the millimeter-wave base station, and Γs(d) represents the signal-to-noise ratio of the millimeter-wave link; B S N is the bandwidth allocated to mobile users by millimeter-wave base stations. S The power spectral density of Gaussian white noise, h, represents millimeter-wave communication. S (d) represents the channel attenuation of the millimeter-wave link.

[0121] In the following text, refer to Figures 2 to 9 The base station handover method for a dual-connectivity mobile communication system according to embodiments of the present invention will be described in detail with specific examples.

[0122] Assume all base stations are randomly distributed within a square area with sides of 1 km. A mobile user starts from a fixed position (200m, 500m) and travels along a straight line at an average speed of 20m / s for 25 seconds. The base station handover interval is 50ms. The bandwidths allocated to microwave and millimeter-wave base stations are 10MHz and 100MHz, respectively, with allocated powers of 30dBm and 46dBm, respectively. The antenna gain of the millimeter-wave base station is 10dB, the Gaussian white noise power spectral density is -174dBm / Hz, and the path loss parameter is set as α. M =38dB, β M =3dB, α S =70dB, β S =2dB, microwave link shadowing effect variance is 4, millimeter-wave link Nakagami-m integer parameter is 3, number of candidate millimeter-wave base stations is 5, obstruction attenuation is randomly selected from {20, 40, 60}dB, and the probability that there is no obstruction in the road segment is set to p. LoSThe NDDQN training had a learning rate of 0.001, an optimizer of Adam, 2 hidden layers, 64 neurons, a ReLU activation function, 64 training batches, 500 training sessions, 500 steps per session, a discount factor of γ = 0.98, and 3 multi-step learning steps.

[0123] The implementation of this method requires first building an environmental simulation platform (or in a real environment) to train and learn the base station handover strategy for a dual-connectivity mobile communication system. After the algorithm converges, the trained strategy is applied to the actual mobile radio link. The base station handover controller makes dual-connectivity base station handover decisions based on collected mobile user status information and communication environment information, thereby achieving long-term high-speed wireless transmission service for the mobile communication system. A base station handover method for a dual-connectivity mobile communication system includes the following specific steps:

[0124] Step 1: Model the dual-connectivity heterogeneous access network, base station handover process, and transmission rate of the dual-connectivity mobile communication system, and establish the base station handover problem.

[0125] This invention optimizes the base station handover method for dual-connectivity mobile communication systems to maximize the wireless transmission rate. Therefore, it first models the dual-connectivity heterogeneous access network, base station handover process, and transmission rate of the dual-connectivity mobile communication system.

[0126] 1. Modeling a dual-connectivity heterogeneous access network

[0127] Suppose that 5G millimeter-wave and microwave base stations are randomly distributed within a fixed area using independent isomorphic Poisson point processes of varying densities. Based on the distribution of microwave base stations, the Delaunay triangulation method is used to divide the area into multiple irregular polygons to represent the macrocell coverage. Let there be K microwave base stations in the area, let... Let w represent the set of microwave base stations. The location coordinates and cell of microwave base station k can be represented as w, respectively. k =[x k ,y k ] and Cell k .

[0128] This invention employs dual connectivity technology and a control / user plane separation network architecture from the 3GPP standard to improve the communication performance and robustness of the radio access link in a mobile communication system. Specifically, a microwave base station and a millimeter-wave base station simultaneously provide radio access services to the mobile communication system. The microwave base station, acting as the primary base station (MgNB), provides control signal and data packet transmission services to the mobile communication system through the control plane and user plane, respectively. The millimeter-wave base station, acting as the secondary base station (SgNB), provides data packet transmission services to the mobile communication system solely through the user plane. The transmitted data from the two base stations will be split or merged at the PDCP layer of the network, and the two base stations interact via the X2 interface. For ease of management, the millimeter-wave base station is configured to communicate only with the microwave base station within its macrocell. Therefore, the millimeter-wave base station can be represented by its macrocell location. k The millimeter-wave base station set is Among them, millimeter wave base station n k The corresponding coordinates can be written as

[0129] 2. Modeling the base station handover process in dual-connectivity mode.

[0130] Because millimeter-wave bandwidth is ten times or more than microwave bandwidth, millimeter-wave links can transmit several times more data than microwave links, playing a more crucial role and significantly impacting communication performance in dual-connectivity mode. Therefore, the dual-connectivity mode base station handover of this invention primarily utilizes millimeter-wave base station handover, with the main consideration being the communication status of each millimeter-wave base station. Microwave base station handover only occurs when millimeter-wave base station handover occurs between different macrocells, ensuring the correspondence between millimeter-wave and microwave base stations. In this case, dual-connectivity mode base station handover can be divided into two main categories: intra-macrocell handover and inter-macrocell handover, the specific processes of which are explained below.

[0131] Macrocell intra-cell handover occurs between two millimeter-wave base stations within the same macrocell. This handover only involves the secondary base station in dual connectivity, and the handover process is controlled by the microwave base station corresponding to the macrocell, i.e., the primary base station in dual connectivity. To avoid interruptions during handover, this invention uses a connect-before-disconnect (MBB) handover protocol. That is, during handover, the mobile user does not disconnect from the previous serving base station until a connection is established with the new base station. Considering that handover requires a certain amount of time due to protocol execution and random access procedures, and no other handover decision can be made during this time, the handover decision interval is set to be the same as the handover time, both being a time block δ. t Let discrete time points t∈{1,2,…,T}, and let b M (t), b S (t) and b T(t) represent the primary base station, secondary base station, and target millimeter-wave base station currently serving the mobile user, respectively. During handover within a macrocell, if... Then there is b M (t)=b M (t+1) = k and b S (t+1)=b T (t), the base station handover process within a cell is shown in the attached figure. Figure 2 As shown.

[0132] Inter-macrocell handover occurs between two millimeter-wave base stations in different cells. This handover involves two phases: millimeter-wave to microwave and microwave to millimeter-wave base station handover. Specifically, if the target millimeter-wave base station is not within the cell of the currently serving millimeter-wave base station, i.e. The currently serving microwave base station cannot transmit control signals or data to the target millimeter-wave base station via the X2 interface in dual-connectivity mode. Therefore, in addition to switching the secondary base station, it is necessary to switch the primary base station to a microwave base station capable of controlling the target millimeter-wave base station. Furthermore, according to the standard protocol, data plane conversion must follow control plane conversion. Therefore, the switching of the primary and secondary base stations cannot be performed simultaneously; the primary base station must be switched to the target microwave base station first. This process requires switching the secondary base station from the currently serving millimeter-wave base station to the target microwave base station, known as millimeter-wave to microwave base station switching. The primary base station remains unchanged. The entire process consumes one time block, resulting in b. M (t+1)=b M (t)=k、 Afterwards, the inter-macrocell handover enters an intermediate state where a new handover decision can be made, selecting the target millimeter-wave base station to be handed over to in the next moment. Depending on the selected target, the next stage of handover is divided into three cases: the target millimeter-wave base station is within the original serving cell, the target millimeter-wave base station is within the original target cell, and other cases, which are described in detail below:

[0133] (A) If the target millimeter-wave base station is within the original serving cell, it is only necessary to switch the secondary base station from the current base station to the target millimeter-wave base station to obtain b. S (t+2)=b T (t+1);

[0134] (B) If the target millimeter-wave base station is within the original target cell, the second stage of cross-macrocell handover will be triggered, namely, the microwave to millimeter-wave base station handover. First, the primary base station and the secondary base station will swap to complete the control plane handover. That is, the current microwave base station of the secondary base station becomes the primary base station and sends control commands to the target millimeter-wave base station. Then, the secondary base station will be switched from the original serving microwave base station to the target millimeter-wave base station to complete the user plane handover. The whole process also takes one time block, so b can be obtained. M (t+2)=b S(t+1) and b S (t+2)=b T (t+1);

[0135] (C) In other cases, neither of the two currently connected microwave base stations can control the target millimeter-wave base station. The secondary base station needs to be switched to the target microwave base station to enter a new intermediate state, initiating another cross-macrocell handover process. At this time, b M (t+2)=b M (t+1) and A diagram illustrating cross-cell handover is attached. Figure 3 As shown.

[0136] 3. Modeling the transmission rate of dual-connection mode.

[0137] The channel attenuation (in dB) of a microwave link can be expressed as:

[0138] h M (d)=α M +10β M log 10 (d)+η M +ξ (1)

[0139] Where d is the distance between the mobile user and the base station, α M and β M Representing the microwave communication path attenuation factor and exponent, respectively, η M Let ξ represent the random small-scale fading in microwave communication, denoted as Rayleigh fading, and ξ be the random shadowing effect. Assume all microwave base stations allocate the same transmit power p to mobile users. M Regarding the unit antenna gain, since only one user's communication is considered, inter-cell interference in microwave communication is ignored here. Even if a mobile user needs to connect to two microwave base stations during inter-cell handover, the two base stations can be assigned different frequency bands to avoid interference. Therefore, the signal-to-noise ratio of the microwave link can be expressed as...

[0140]

[0141] Among them B M N is the bandwidth allocated by the microwave base station to mobile users. M Let represent the power spectral density of Gaussian white noise in microwave communication. Then the microwave communication transmission rate can be written as...

[0142] r M (d)=B M log2(1+Γ M (d)) (3)

[0143] For 5G millimeter-wave links, in addition to free path loss, obstruction is also a significant factor affecting transmission performance. Assuming the user's path is confined to a fixed road, this invention equates various obstruction effects (including those from buildings or moving objects) to obstructed road segments. Let the random obstruction attenuation (in dB) be ξ, then the channel attenuation (in dB) of a 5G millimeter-wave link can be expressed as:

[0144] h S (d)=α S +10β S log 10 (d)+η S +ζ (4)

[0145] Where α S and β S η represents the path attenuation factor and exponent for millimeter-wave communication, respectively. S The random small-scale fading in millimeter-wave communication is denoted as Nakagami-m fading, and random shadowing effects are ignored here; it can be incorporated into the occlusion attenuation. The millimeter-wave link is equipped with a directional antenna. Since this invention only considers base station handover, it is assumed that perfect beamforming and beam tracking can be performed to obtain sufficient antenna gain, which can be expressed as g. max Because millimeter waves have poor penetration and strong directionality, this invention ignores interference between millimeter wave base stations; therefore, the signal-to-noise ratio of the millimeter wave link can be written as...

[0146]

[0147] Among them B S p S These are the bandwidth and power allocated to users by the millimeter-wave base station, N. S Let represent the power spectral density of Gaussian white noise in millimeter-wave communication. Then, the microwave communication transmission rate of a mobile communication system can be written as...

[0148] r S (d)=B S log2(1+Γ S (d)) (6)

[0149] In summary, the transmission rate of a mobile communication system in dual-connectivity mode at time t can be expressed as:

[0150]

[0151] Case 1 indicates that the secondary base station is a millimeter-wave base station, which is the primary case; Case 2 indicates that the secondary base station is a target microwave base station, which occurs during the intermediate state of inter-cellular handover. and These represent the distances between the primary and secondary base stations and the user, respectively. and These are the locations of the primary and secondary base stations currently in service.

[0152] 4. Base station handover issues in establishing dual-connectivity mobile communication systems

[0153] This invention aims to provide high-speed wireless communication services for mobile users; therefore, the goal of dual-connectivity base station handover is to maximize the transmission rate.

[0154]

[0155] in This represents the average transmission rate of a mobile user over a period of time. Given the time-varying and random environmental factors, including wireless channels, base station distribution, and mobile trajectories, this invention aims to maximize the expected value of the transmission rate.

[0156] Step 2: Design the NDDQN algorithm to solve the base station handover problem.

[0157] After clarifying the base station handover problem, in order to solve this problem, this invention first transforms the problem into a sequential decision problem under the reinforcement learning framework, and then proposes the NDDQN reinforcement learning algorithm to learn intelligent strategies to solve the sequential decision problem and realize the handover decision of dual-connectivity base stations. The following is a detailed introduction.

[0158] 1. Base station handover decision problem under reinforcement learning framework

[0159] To solve the dual-connectivity base station handover problem using reinforcement learning-based algorithms, the problem needs to be transformed into a sequential decision problem and the three fundamental elements of reinforcement learning need to be defined: state, action, and reward. Based on the problem... The optimization variables, objectives, and constraints can be defined as follows: the action at time t can be set as... The reward is Reinforcement learning algorithms learn policies to select actions for different states in order to obtain the maximum expected long-term reward, i.e., for the state at time t. accomplish in

[0160] This indicates the operation of finding the expected value. This is a discount factor. To avoid the signal overhead, latency, and errors associated with collecting channel state information, this invention chooses to use geographical information about mobile users, base stations, and obstructed road sections to describe the wireless communication environment, thereby defining the system state. That is, the system state of the dual-connectivity base station handover decision problem at time t is set as follows: Among them I HO(t) indicates whether time t is in the intermediate state of inter-cellular handover, b c (t) represents the set of candidate micro base stations for handover at time t. This includes geographical and obstruction information for candidate millimeter-wave base stations. Assuming there are I candidate millimeter-wave base stations, then... in Location of the i-th candidate millimeter-wave base station Corresponding microwave base station location Starting point of the obstructed road section and termination position and the attenuation caused by shading This invention includes I-1 millimeter-wave base stations as candidates, which are closest to the mobile user and those currently serving the mobile user. Due to the high path attenuation of millimeter waves, millimeter-wave base stations farther from the mobile user are no longer considered. This reduces the action set to...

[0161] With the settings for status, actions, and rewards, the base station handover problem... The problem is transformed into a sequential decision-making problem within a reinforcement learning framework. The approach to solving this problem is to formulate an optimal policy that continuously selects the best action based on changing environments. Reinforcement learning algorithms undergo trial and error, learning to evaluate the value function of policy π. Based on this, the optimal strategy is found, and the optimal action is selected for all states. The following section introduces the NDDQN base station handover algorithm proposed in this invention to find the optimal base station handover strategy.

[0162] 2. NDDQN base station handover algorithm

[0163] NDDQN is an improved version of the Deep Q-Network algorithm (DQN), which uses a deep neural network (DNN) to estimate the state-action value q. π (s,a) is used to train a DNN by minimizing the difference between the current Q-value and the target Q-value. Due to the correlation between state observations and the correlation between the current Q-network and the target Q-network, simple DNN estimators have proven difficult to learn stably or even converge in reinforcement learning. To address this problem, empirical replay and fixed Q-target techniques are needed to break the aforementioned correlations, i.e., each time the DQN agent acquires a state observation... Instead of immediately using this state for learning, it will first be used as experience. Store in playback memory In the middle. When DQN needs to learn, it randomly draws small batches of experience samples from the replay memory. This breaks the correlation in the state observation sequence. Furthermore, the fixed Q-target technique breaks the correlation between the current Q-network and the target Q-network by updating the target Q-network only once every C learning steps. Using these two techniques, the loss function of DQN during training can be expressed as...

[0164]

[0165] Where θ and Let represent the parameters of the current Q-network and the target Q-network, respectively. By minimizing the above loss function, the parameters of the current Q-network will be updated, resulting in a more accurate value estimation network. To obtain a superior dual-connectivity base station handover strategy, this invention uses three techniques to effectively improve the learning performance of the original DQN and proposes a corresponding NDDQN base station handover algorithm.

[0166] (A) Multi-step learning techniques: The original DQN uses a single-step bootstrapping method to solve for the target Q value, i.e., in (9). Only one actual reward value is used. To estimate the target Q value more accurately, this invention proposes to use a more forward-looking multi-step bootstrapping method to obtain the target Q value, that is, to use multiple actual reward values ​​in the future at the sampling time points. Through this method, the loss function in (9) will become

[0167]

[0168] in,

[0169] and All data comes from playback memory. A diagram illustrating the multi-step learning process is attached. Figure 4 As shown.

[0170] (B) Dual DQN Technique: To alleviate the overestimation problem caused by DQN maximization, this invention employs dual DQN technique to decouple the Q-network used for target evaluation and action selection when calculating the target Q-value. The loss function of the multi-step dual DQN then becomes...

[0171]

[0172] For convenience, in equation (10) and They were replaced with and A schematic diagram of calculating the target Q value using dual DQN is attached. Figure 5 As shown.

[0173] (C) Noisy Network Technique: To enable more efficient exploration when learning strategies, this invention employs a noisy network technique, which transforms the standard linear layer y = Wx + b in a DNN into a noisy linear layer.

[0174] y = (μ W +σ W ⊙∈ W )x+μ b +σ b ⊙∈ b (13)

[0175] Where μ W σ W μ b and σ b All are learnable parameters, μ W and σ W μ represents the weights of the neural network after adding noise. b and σ b For the bias of the neural network after adding noise, ∈ W and ∈ b Let be the Gaussian noise variable, and ⊙ denote the multiplication operation of matrix factors. The input-output relationship of the network layers in the noisy neural network is shown in the appendix. Figure 6 As shown.

[0176] The three techniques described above are used to replace the target Q-value estimation and exploration parts of the original DQN. Combined with the state, action, and reward settings mentioned above, this constitutes the NDDQN dual-connectivity base station handover method proposed in this invention. Specific training methods are detailed in the appendix. Figure 7 First, the entire training environment is established according to step one. Then, at the beginning of each training session, the environment needs to be initialized and reset so that NDDQN can learn from different experiences to improve its generalization ability. At each training step t, NDDQN observes the current state. To further explore this, the noise in the noisy network needs to be reset before selecting an action. Then... Input NDDQN and select an action for base station handover based on the output. Observe the current reward And this experience Store in playback memory bank After the replay memory has accumulated enough experience, NDDQN begins to sample from it and train the current Q-network. The training method updates the neural network parameters by minimizing the loss function (12). In addition, after every C training steps, the target Q-network is updated using the current Q-network. After training for a period of time, NDDQN will converge and learn an optimal base station handover strategy, enabling it to make the best base station handover decision for different states.

[0177] Step 3: Configure this pre-trained NDDQN base station handover method in the base station controller of the actual access network, make base station handover decisions based on the actual environment, and continuously update the base station handover strategy based on feedback results to ensure continuous optimization of mobile communication system base station access and ultra-high-speed wireless links.

[0178] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0179] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A base station handover method for a dual-connectivity mobile communication system, characterized in that, include: The dual-connectivity heterogeneous access network, base station handover process, and transmission rate of the dual-connectivity mobile communication system are modeled to establish the base station handover problem of the dual-connectivity mobile communication system. A noisy dual-deep Q network is established and trained using base station handover experiences stored in the playback memory to obtain a base station handover decision model. as well as The base station handover decision model is configured in the base station controller of the access network, enabling the base station controller to solve the base station handover problem based on the actual environment using the base station handover decision model and make base station handover decisions, and continuously update the base station handover strategy based on feedback results. Modeling the dual-connectivity heterogeneous access network, base station handover process, and transmission rate of the dual-connectivity mobile communication system includes: The dual-connectivity heterogeneous access network is modeled, providing wireless access services to the mobile communication system simultaneously through microwave base stations and millimeter-wave base stations. The millimeter-wave base station can only communicate with the microwave base station within its macrocell. The microwave base station, acting as the primary base station, provides control signal and data packet transmission services to the mobile communication system through the control plane and user plane, respectively. The millimeter-wave base station, acting as the secondary base station, provides data packet transmission services to the mobile communication system through the user plane. The base station handover process in the dual connectivity mode is modeled, wherein the base station handover process includes intra-macrocell handover or inter-macrocell handover; The transmission rate of the dual-connectivity mode is modeled, and the transmission rate of the mobile communication system in dual-connectivity mode at time t is expressed by the following formula: Case 1 indicates that the secondary base station is a millimeter-wave base station, which is the dominant case; Case 2 indicates that the secondary base station is a target microwave base station, which occurs during the intermediate state of intercellular handover. and These represent the distances between the primary and secondary base stations and the mobile user, respectively. and These are the locations of the primary and secondary base stations currently in service, q( t ) indicates at time t User location, r M ( d ) represents the microwave communication transmission rate of the mobile communication system, r S ( d ) represents the millimeter-wave transmission rate of the mobile communication system.

2. The base station handover method for a dual-connectivity mobile communication system according to claim 1, characterized in that, The microwave communication transmission rate r of the mobile communication system is calculated using the following formula. M ( d ): ； ； ； in, d It is the distance between the mobile user and the base station. α M and β M These represent the microwave communication path attenuation factor and the exponent, respectively. η M This represents random small-scale fading in microwave communication. ξ This is a random shadowing effect. p M Γ represents the transmit power allocated by the microwave base station to the mobile user. M ( d () represents the signal-to-noise ratio of the microwave link. B M It is the bandwidth allocated by the microwave base station to mobile users. N M The power spectral density represents Gaussian white noise in microwave communication. h M ( d () represents channel attenuation in the microwave link; The millimeter-wave transmission rate r of the mobile communication system is calculated using the following formula. S ( d ): ； ； ； in, α S and β S These represent the path attenuation factor and exponent for millimeter-wave communication, respectively. η S This represents random small-scale fading in millimeter-wave communication. p S Γs represents the transmit power allocated by the millimeter-wave base station to the mobile user. d () represents the signal-to-noise ratio of a millimeter-wave link; B S It is the bandwidth allocated to mobile users by millimeter-wave base stations. N S The power spectral density of Gaussian white noise, representing millimeter-wave communication. h S ( d () represents channel attenuation in millimeter-wave links. Random occlusion attenuation, This represents the antenna gain.

3. The base station handover method for a dual-connectivity mobile communication system according to claim 1, characterized in that, The intra-macrocell handover includes: handover between two millimeter-wave base stations within the same macrocell under the control of the microwave base station corresponding to the macrocell, wherein, during the intra-macrocell handover process, a handover protocol is used to first establish a connection between the mobile user and the target millimeter-wave base station and then disconnect the connection between the mobile user and the serving millimeter-wave base station.

4. The base station handover method for a dual-connectivity mobile communication system according to claim 1, characterized in that, The cross-macrocell handover includes: handover between two millimeter-wave base stations in different cells, including handover from a millimeter-wave base station to a microwave base station and handover from a microwave base station to a millimeter-wave base station, wherein... The handover from the millimeter-wave base station to the microwave base station includes changing the secondary base station from the serving millimeter-wave base station to the first target microwave base station to enter a first intermediate state; and The handover between the microwave base station and the millimeter-wave base station includes: When the target millimeter-wave base station is within the original serving cell, the secondary base station will be switched from the first target microwave base station to the target millimeter-wave base station; When the target millimeter-wave base station is within the original target cell, the primary base station is changed from the serving microwave base station to the first target microwave base station and a control command is sent to the target millimeter-wave base station. Then, the secondary base station is switched from the first target microwave base station to the target millimeter-wave base station to complete the user plane handover. When neither the primary base station nor the first target microwave base station can control the target millimeter-wave base station, the secondary base station is switched to the target microwave base station to enter a second intermediate state, wherein the second intermediate state is different from the first intermediate state.

5. The base station handover method for a dual-connectivity mobile communication system according to claim 1, characterized in that, The base station handover issues in establishing the dual-connectivity mobile communication system include: The purpose of addressing the base station handover issue is to maximize the transmission rate. in, This represents the average transmission rate of a mobile user over a period of time. b T ( t () indicates the target millimeter-wave base station. For macrocells k The collection of millimeter-wave base stations within, t For discrete time points.

6. The base station handover method for a dual-connectivity mobile communication system according to claim 1, characterized in that, Building a noisy dual-deep Q-network includes: A multi-step bootstrapping method is used to obtain the target Q value, so that the loss function is expressed as: ； ； S ( t + n )and S ( t Let A(t) represent the system states at times t+n and t, respectively, and let A(t) represent the action at time t. R ( t + l )| l =0,1,…, n {-1} is the set of reward values ​​from time t to time t+n-1. ∈[0,1] is the discount factor. and These represent the parameters of the current Q-network and the target Q-network, respectively. and These are the target Q value and the current Q value, respectively. By employing dual DQN technology to decouple the Q-network used for target evaluation and action selection when calculating the target Q-value, the loss function of multi-step dual DQN becomes: ； Among them, R (n) ( t ), S ( t + n ), S ( t )and A ( t Replace them with R (n) , S (n) , S and A ; Using noisy network techniques to modify the standard linear layers in DNN y =W x + b Convert to a noisy linear layer: ； in μ W , σ W , μ b and σ b All are learnable parameters. μ W and σ W The weights of the neural network after adding noise. μ b and σ b This is the bias of the neural network after adding noise. and is a Gaussian noise variable, and ⊙ represents the multiplication operation of matrix factors.

7. A base station handover device for a dual-connectivity mobile communication system, characterized in that, include: The base station handover problem acquisition module is used to model the dual-connectivity heterogeneous access network, base station handover process and transmission rate of the dual-connectivity mobile communication system, and establish the base station handover problem of the dual-connectivity mobile communication system, wherein the base station handover process includes intra-macrocell handover and inter-macrocell handover. A base station handover problem solving model is provided, which is used to establish a noisy dual-deep Q network and train the noisy dual-deep Q network using base station handover experiences stored in a playback memory to obtain a base station handover decision model, wherein the base station handover problem is solved through the base station handover decision model; and The base station handover module is used to configure the base station handover decision model in the base station controller of the access network, so that the base station controller makes base station handover decisions based on the actual environment and continuously updates the base station handover strategy based on feedback results. The base station handover problem acquisition module includes an access network model establishment module, a base station handover model establishment module, and a transmission rate model establishment module, wherein... The access network model building module is used to model the dual-connectivity heterogeneous access network, providing wireless access services to the mobile communication system simultaneously through microwave base stations and millimeter-wave base stations. The millimeter-wave base station can only communicate with the microwave base station within its macrocell. The microwave base station, acting as the primary base station, provides control signal and data packet transmission services to the mobile communication system through the control plane and user plane, respectively. The millimeter-wave base station, acting as the secondary base station, provides data packet transmission services to the mobile communication system through the user plane. The base station handover model establishment module is used to model the base station handover process in the dual connectivity mode, wherein the base station handover process includes intra-macrocell handover or inter-macrocell handover. The transmission rate model building module is used to model the transmission rate of the dual-connectivity mode, wherein the transmission rate of the mobile communication system in dual-connectivity mode at time t is expressed by the following formula: Case 1 indicates that the secondary base station is a millimeter-wave base station, which is the dominant case; Case 2 indicates that the secondary base station is a target microwave base station, which occurs during the intermediate state of intercellular handover. and These represent the distances between the primary and secondary base stations and the mobile user, respectively. and These are the locations of the primary and secondary base stations currently in service, q( t ) indicates at time t User location, r M ( d ) represents the microwave communication transmission rate of the mobile communication system, r S ( d ) represents the millimeter-wave transmission rate of the mobile communication system.

8. The base station handover device for a dual-connectivity mobile communication system according to claim 7, characterized in that, The transmission rate model establishment module is also used for: The microwave communication transmission rate r of the mobile communication system is calculated using the following formula. M ( d ): ； ； ； Among them, d It is the distance between the mobile user and the base station. α M and β M These represent the microwave communication path attenuation factor and the exponent, respectively. η M Represents random small-scale fading in microwave communication. ξ This is a random shadowing effect. p M Γ represents the transmit power allocated by the microwave base station to the mobile user. M ( d () represents the signal-to-noise ratio of the microwave link. B M It is the bandwidth allocated by the microwave base station to mobile users. N M The power spectral density represents Gaussian white noise in microwave communication. h M ( d () represents channel attenuation in the microwave link; The millimeter-wave transmission rate r of the mobile communication system is calculated using the following formula. S ( d ): ； ； ； in, α S and β S These represent the path attenuation factor and exponent for millimeter-wave communication, respectively. η S This represents random small-scale fading in millimeter-wave communication. p S Γs represents the transmit power allocated by the millimeter-wave base station to the mobile user. d () represents the signal-to-noise ratio of a millimeter-wave link; B S It is the bandwidth allocated to mobile users by millimeter-wave base stations. N S The power spectral density of Gaussian white noise, representing millimeter-wave communication. h S ( d () represents channel attenuation in millimeter-wave links. Random occlusion attenuation, This represents the antenna gain.

Images