Rapid spectrum allocation method and system for ultra-dense network based on transfer learning

By adopting the transfer learning method in super-dense networks, building a maximum throughput problem model and transfer optimization strategy, the delay and efficiency problems of traditional resource allocation strategies in dynamic scenarios are solved, and more efficient resource allocation and throughput optimization are achieved.

CN119095170AActive Publication Date: 2024-12-06NAT UNIV OF DEFENSE TECH
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411119868.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-15
Publication Date
2024-12-06
Estimated Expiration
2044-08-15

AI Technical Summary

Technical Problem

In super-dense networks, traditional static resource allocation strategies cannot effectively respond to dynamically changing network conditions and user behavior patterns, resulting in resource allocation delays and inefficiency, making it difficult to quickly respond to changes in dynamic scenarios.

Method used

Using a fast spectrum allocation method based on transfer learning, by building a maximum throughput problem model, setting the status, action and reward functions of the user equipment, calculating the similarity between new users and source scenario users, and the source scenario with the highest similarity User policies are migrated to new users to quickly adapt to changes in user count and environment.

Benefits of technology

It improves the efficiency of resource allocation and system throughput optimization, reduces the time to learn optimal strategies, and significantly improves performance in dynamic scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119095170A_ABST
    Figure CN119095170A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of ultra-dense network communication, and particularly discloses an ultra-dense network rapid spectrum allocation method and system based on transfer learning, and the method comprises the steps: building a maximum throughput problem model under the conditions of spectrum multiplexing and common channel interference based on an application scene of dense base station deployment in an ultra-dense network; setting a state function, an action function and a reward function of each user equipment, and training the maximum throughput problem model; after the number of the users changes dynamically, the similarity between the new users and the source scene users is calculated according to the Euclidean distance, the cosine similarity and the Pearson's correlation coefficient; and migrating the strategy of the source scene user with the highest similarity to a new user according to a transfer learning algorithm.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of ultra-dense network communications, and in particular to an ultra-dense network fast spectrum allocation method and system based on transfer learning. Background Art

[0002] In the study of ultra-dense networks (UDN), the efficiency and flexibility of resource allocation are particularly important. UDN aims to cope with the growing demand for mobile communications and improve service quality by densely deploying a large number of small base stations in a specific area. This network layout can not only improve spectrum utilization efficiency and reduce communication delays, but also significantly improve user experience. The architecture of UDN can provide higher transmission rates and more reliable service quality, thereby greatly improving user experience. However, the dense deployment of UDN also brings new technical challenges, especially in resource management and allocation.

[0003] Traditional resource allocation strategies are mostly based on static network states and environmental assumptions. Many existing research methods also focus on resource allocation in specific, fixed scenarios. For example, a study proposed an improved cluster-assisted resource allocation (CARA) scheme to reduce interference between small cells in UDN, but it considers the case where the user's static position remains unchanged. Another study, considering only static virtual cells (VC), proposed a local anchor-based VC switching scheme for meaningless mobility in UDN, where the user equipment (UE) in the VC can select multiple service access points according to different channel conditions. Similarly, other studies have also explored the energy efficiency optimization problem in dense networks with fixed user locations.

[0004] However, in real environments, network states and user behavior patterns are dynamic and uncertain. In this case, methods based on static assumptions often cannot effectively cope with changes in scenarios or environments. Especially when complex learning models are involved, retraining models to adapt to new environments usually takes a lot of time and computing resources. For example, methods using reinforcement learning models and deep learning models require continuous iterative learning to obtain the optimal strategy. Whenever a new user joins the network or the external conditions of the network change, retraining is required to find the optimal strategy. This method is not only inefficient, but also difficult to apply in dynamic scenarios that require rapid response.

[0005] The core challenge of resource allocation in UDN is not only to improve spectrum efficiency and reduce network latency, but more importantly, to be able to sensitively adapt to the ever-changing network conditions and user behavior patterns. This requires the resource management system to be not only efficient, but also able to quickly respond to changes in various scenarios. In view of this, a more flexible and adaptive strategy is needed to cope with this ever-changing and uncertain environment. Summary of the invention

[0006] To achieve the purpose of the present invention, the present application provides a method for fast spectrum allocation in ultra-dense networks based on transfer learning, comprising:

[0007] Step S1: Based on the application scenario of dense base station deployment in an ultra-dense network, a maximum throughput problem model under spectrum reuse and co-channel interference conditions is constructed;

[0008] Step S2: setting the state function, action function and reward function of each user device, and training the maximum throughput problem model;

[0009] Step S3: After the number of users changes dynamically, the similarity between the new user and the source scene user is calculated based on the Euclidean distance, cosine similarity and Pearson correlation coefficient;

[0010] Step S4: Migrate the strategy of the source scenario user with the highest similarity to the new user according to the transfer learning algorithm.

[0011] In some specific embodiments, in step S1, the maximum system total throughput is calculated under the constraints of user association and sub-channel allocation:

[0012]

[0013] In the formula, is the transmission power of base station i on subchannel n, is the signal-to-interference-noise ratio of user k and base station i on signal n at time t; C1 and C2 are binary variables, representing the association between the user, the base station and the channel respectively; C3 and C4 respectively indicate that a user can only select one base station and one channel at time t; C5 limits the transmission power of the base station to not exceed its maximum power.

[0014] In some specific embodiments, in step S2, the user's position t is represented as a two-dimensional vector (x k (t),y k (t)), each agent in the network needs to know the information of the entire network to decide the action selection, and each agent needs to determine the optimal strategy according to its own position. Therefore, the user k state is composed of the channel selection matrix, the power selection matrix and the position state:

[0015] sk (t)={C(t),B(t),d k (t)}

[0016] Among them, s k (t) is the state observed by user k in time slot t, C(t) is the set of channel selections for all users, and B(t) is the set of base station selections associated with all users.

[0017] In some specific embodiments, the best associated base station and channel of the user are determined according to the base station location and signal strength in the dynamic network environment, which is expressed as:

[0018] a k (t) = {b k (t),c k (t)}

[0019] Among them, a k (t) is the action selected by user k at time t; b k (t) is the base station selection of user k at time t; c k (t) is the channel selection of user k at time t.

[0020] In some specific embodiments, according to the system throughput, the agent is incentivized to take actions to improve the throughput, and the incentive function is expressed as:

[0021]

[0022] Among them, B is the base station index set, U is the user set, n is the selected channel, r i is the service radius of base station i, and d is the service range of the base station associated with the user connection.

[0023] In some specific embodiments, in step S3, the moving path and channel state of each user are determined according to the following formula:

[0024] U i = {Path i ,Ch i}

[0025] Among them, U i Information for each user; Path i represents the mobile path information of the i-th user; Ch i represents the channel state information of the i-th user.

[0026] In some of the specific embodiments, in step S3, a linear interpolation method is used to divide the moving path of each user from its starting position to its ending position into m sampling points, and the spatial similarity and moving direction similarity between the new user and the source scene user are calculated respectively based on the Euclidean distance and the cosine similarity.

[0027] In some specific embodiments, in step S3, the similarity of the channel states of two users and the linear correlation of the channel gains are measured according to the Pearson correlation coefficient. For each subchannel, the Pearson correlation coefficient between the channel gains of the two users is calculated, and the average value is taken as the overall channel similarity.

[0028] In some specific embodiments, two weights are designed to respectively calculate the comprehensive similarity of the spatial similarity and the moving direction similarity and the similarity determined according to the relative importance of the path and the channel. To achieve the same invention purpose, the present application also provides a system for fast spectrum allocation in ultra-dense networks based on transfer learning, including:

[0029] Model building module: used to build a maximum throughput problem model under spectrum reuse and co-channel interference conditions based on the application scenario of dense base station deployment in ultra-dense networks;

[0030] Function determination module: used to set the state function, action function and reward function of each user device and train the maximum throughput problem model;

[0031] Similarity calculation module: used to calculate the similarity between new users and source scenario users based on Euclidean distance, cosine similarity and Pearson correlation coefficient after the number of users changes dynamically;

[0032] Strategy migration module: used to migrate the strategy of the source scenario user with the highest similarity to the new user according to the migration learning algorithm.

[0033] Beneficial effects of the above technical solution:

[0034] This application aims at the problem of resource allocation delay caused by dynamic changes in the number of users in ultra-dense networks, constructs a scenario model with dynamic changes in the number of users in ultra-dense networks, and proposes a resource allocation algorithm based on transfer learning.

[0035] This application designs a comprehensive similarity calculation to analyze the correlation between new and old users. By finding the old users with the most similar characteristics to the newly added users, the optimized strategies are transferred to the new users and further trained as the initial strategies, thereby accelerating the adaptation speed of the neural network to user changes.

[0036] The simulation experiment results of this application show that in the scenario where the number of users continues to increase, the resource allocation algorithm using the transfer learning strategy shows higher efficiency and effect in terms of the speed of learning the optimal strategy and optimizing the system throughput compared with the traditional deep reinforcement learning algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 A schematic diagram of a flow chart of a method for rapid spectrum allocation in an ultra-dense network based on transfer learning provided by an embodiment of the present invention;

[0039] Figure 2 A diagram showing a scenario of a change in the number of users in an ultra-dense network provided by an embodiment of the present invention;

[0040] Figure 3 A flow chart of a transfer learning algorithm provided for one embodiment of the present invention;

[0041] Figure 4 A comparison diagram of model utilities at different learning rates provided for one embodiment of the present invention;

[0042] Figure 5 A diagram comparing the model effectiveness at different neuron scales provided for one embodiment of the present invention;

[0043] Figure 6 A utility comparison diagram of different learning models for six users provided by an embodiment of the present invention;

[0044] Figure 7 A utility comparison diagram of different learning models for seven users provided by an embodiment of the present invention;

[0045] Figure 8 A comparison chart of system throughput of different learning models in the middle of training provided by an embodiment of the present invention;

[0046] Fig. 9 A comparison chart of system throughput of different learning models in the late stage of training provided by an embodiment of the present invention;

[0047] Fig.10 A schematic diagram of the structure of a system for rapid spectrum allocation in ultra-dense networks based on transfer learning is provided for one embodiment of the present invention. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0049] Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar symbols throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0050] Embodiment 1

[0051] An embodiment of the present invention provides a method for fast spectrum allocation in ultra-dense networks based on transfer learning, referring to Figure 1 As shown, including:

[0052] Step S1: Based on the application scenario of dense base station deployment in an ultra-dense network, a maximum throughput problem model under spectrum reuse and co-channel interference conditions is constructed.

[0053] In a specific embodiment of the present invention, referring to Figure 2 As shown in, consider a scenario where the number of users in an ultra-dense heterogeneous network changes dynamically, and each user moves randomly on its mobile path, such as Figure 1 As shown, the source scenario includes a macro base station (MBS) and F femto base stations, and the base station index set is represented by B = {0, 1, 2, 3 ... F}, where 0 represents the macro base station index. The total user set of the network can be represented by U = {0, 1, 2, 3 ... K-1}.

[0054] The macro base station and each small base station share the same spectrum resources. At the same time, each user equipment (UE) can only be associated with one base station. Both macro user equipment (MUE) and small base station user equipment (SUE) can use the same spectrum resources. Considering that the base station can have different UE sets, the association vector of user k is given by It means that when the kth user chooses to associate with base station i, otherwise, Assume that each UE can select at most one BS at the same time. That is to say:

[0055]

[0056] The spectrum resources of each base station are allocated to the UE associated with it. For the kth user, the binary channel allocation vector is given by It means that if the kth UE selects channel n at time t, then otherwise Assume that each UE can select at most one channel at any time t, that is:

[0057]

[0058] Since the base station group in the scenario shares the same spectrum resources, the user will be interfered by the co-channel users. Therefore, in time slot t, the signal to interference and noise ratio of the kth UE on subchannel n can be expressed as:

[0059]

[0060] In the formula, is the transmission power of base station i on subchannel n, which is evenly distributed by the associated base stations according to the number of users associated with the base stations; σ 2 is Gaussian white noise; It represents the channel gain of base station k serving the i-th user on channel n at time t. It is close to the actual signal transmission loss and consists of two parts: path loss and Rayleigh fading. It can be expressed as:

[0061]

[0062] Where L is the corrected path loss constant; d k (t) represents the base station f at time t k The straight-line distance to its service user k; α is the path loss exponent; represents the base station f at time t k The Rayleigh fading of the user it serves on subchannel n. Therefore, through the Shannon formula, the total capacity of user k at time t can be obtained as:

[0063]

[0064] The total channel capacity of the system at time t can be expressed as:

[0065]

[0066] Specifically, considering the user association and subchannel allocation constraints to maximize the total system throughput, the problem can be modeled as:

[0067]

[0068] In the formula, is the transmission power of base station i on subchannel n, is the signal-to-interference-noise ratio of user k and base station i on signal n at time t; C1 and C2 are binary variables, representing the association between the user, the base station and the channel respectively; C3 and C4 respectively indicate that a user can only select one base station and one channel at time t; C5 limits the transmission power of the base station to not exceed its maximum power.

[0069] Step S2: setting the state function, action function and reward function of each user device, and training the maximum throughput problem model.

[0070] In a specific embodiment of the present invention, in the UDN scenario, all participants, i.e., UEs, are selfish and hope to obtain the maximum long-term reward by selecting the best base station and channel. At any time t, the reward of the participant is affected by the current state of the network environment and the behavior of other participants. At the next moment, it will be converted to a completely new random state, which is affected by the previous state and the selected actions of all participants. Therefore, we can regard this optimization problem as a competitive game. By using the same reward for all agents, it is converted into a fully cooperative game to ensure the performance of the global network.

[0071] In a specific embodiment of the present invention, state: Let s k (t) represents the state observed by user k at time slot t, which is used to characterize the UDN environment. Public information (such as the action selection information of each FUE) can be obtained from the MBS at the beginning of each time slot. The user at position t can be represented as a two-dimensional vector (x k (t),y k (t)), record The user position is converted into a one-dimensional state storage. Each agent in the network needs to know the information of the entire network to determine the action selection, and each agent needs to determine the optimal strategy according to its own position. Therefore, the user k state consists of the channel selection matrix, the power selection matrix and the position state:

[0072] s k (t)={C(t),B(t),d k (t)} (8)

[0073] Among them, s k (t) is the state observed by user k in time slot t, C(t) is the set of channel selections for all users, and B(t) is the set of base station selections associated with all users.

[0074] In a specific embodiment of the present invention, the best associated base station and channel of the user are determined according to the base station location and signal strength in a dynamic network environment.

[0075] Action: In UDN, the key to user action design is to intelligently select the best associated base station and channel. This process requires users to comprehensively consider multiple factors such as base station location and signal strength in a dynamic network environment to ensure accurate selection of associated base stations. With intelligent resource allocation strategies, users can obtain more flexible and reliable connections in UDN, thereby significantly improving the overall performance of the network. Therefore, it can be expressed as:

[0076] a k (t) = {b k (t),c k (t)} (9)

[0077] Among them, a k (t) is the action selected by user k at time t; b k (t) is the base station selection of user k at time t; c k (t) is the channel selection of user k at time t.

[0078] In a specific embodiment of the present invention, based on the system throughput, the agent is motivated to take actions to improve the throughput.

[0079] Reward: The reward function is based on the system throughput, which incentivizes agents to take actions that improve throughput. At the same time, all agents share the same reward, forming a fully cooperative game to ensure overall network performance. In particular, when the user connection is beyond the service range of the associated base station, the reward is zero, ensuring that users take coverage into account when selecting base stations. It can be specifically expressed as follows:

[0080]

[0081] Among them, B is the base station index set, U is the user set, n is the selected channel, r i is the service radius of base station i, and d is the service range of the base station associated with the user connection.

[0082] Step S3: After the number of users changes dynamically, the similarity between the new user and the source scenario user is calculated based on the Euclidean distance, cosine similarity and Pearson correlation coefficient.

[0083] In a specific embodiment of the present invention, in step S3, the moving path and channel state of each user are determined according to the following formula:

[0084] U i = {Path i ,Ch i}

[0085] Among them, U i Information for each user; Path i represents the mobile path information of the i-th user; Chi represents the channel state information of the i-th user.

[0086] In a specific embodiment of the present invention, in step S3, a linear interpolation method is used to divide the moving path of each user from its starting position to its ending position into m sampling points, and the spatial similarity and moving direction similarity between the new user and the source scene user are calculated respectively based on the Euclidean distance and the cosine similarity.

[0087] In a specific embodiment of the present invention, in step S3, the similarity of the channel states of two users and the linear correlation of the channel gains are measured according to the Pearson correlation coefficient. For each subchannel, the Pearson correlation coefficient between the channel gains of the two users is calculated, and the average value is taken as the overall channel similarity.

[0088] In a specific embodiment of the present invention, two weights are designed to respectively calculate the comprehensive similarity of the spatial similarity and the moving direction similarity and the similarity determined according to the relative importance of the path and the channel.

[0089] Specifically, three main metrics are considered when calculating the similarity between users: Euclidean distance, cosine similarity, and Pearson correlation coefficient. Each metric emphasizes different aspects of similarity and plays a key role in different application scenarios. The following is a detailed introduction to these three metrics.

[0090] Euclidean distance is the most intuitive and easiest way to understand the distance between vectors. It is used to calculate the actual distance between two points in n-dimensional space. The calculation formula is as follows:

[0091]

[0092] Where A=(a1,a2,...,a n ), B=(b1,b2,...,b n ) represent two n-dimensional vectors respectively. At this time, the similarity between the vectors can be defined by formula (13). Among them, sim(A,B)∈(0,1), the larger the distance is, the closer it is to 0, and the lower the similarity is; on the contrary, the smaller the distance is, the closer it is to 1, and the higher the similarity is.

[0093]

[0094] Cosine similarity focuses on reflecting the difference between two vectors in direction. Specifically, it measures the cosine value of the angle between two vectors and determines whether the vectors are similar based on the cosine value. Formula (14) shows the calculation method of the cosine value between two vectors, so the calculation method of the cosine similarity between vector A and vector B can be deduced as shown in formula (15).

[0095] A·B=||A||||B||cosθ (13)

[0096]

[0097] Among them, the cosine similarity metric is usually used in positive space, that is, it is assumed that the two vectors have the same direction. Therefore, cossim(A,B)∈[0,1]. When the cosine similarity is 0, the two vectors are considered to be the least similar. When the cosine similarity is 1, the two vectors are considered to have exactly the same direction, which means they are considered to have the highest degree of similarity.

[0098] The Pearson correlation coefficient is an indicator to measure the linear relationship between two sets of data. Its value is between -1 and 1, where 1 indicates a perfect positive correlation, -1 indicates a perfect negative correlation, and 0 indicates no correlation. Its calculation formula is:

[0099]

[0100] Where r is the Pearson coefficient between vector A and vector B; A i and B i is a single observation of random variables A and B; and are the means of A and B respectively.

[0101] The ultra-dense network scenario considered in this chapter contains K users, and each user will move randomly along its mobile path. At any time t, there is a two-dimensional coordinate. The two-dimensional coordinate of old user i is expressed as (x i,t ,y i,t ), the two-dimensional coordinates of the new user j are expressed as (x j,t ,y j,t ). Therefore, when calculating the path similarity between old users and new users, we cannot only consider the start and end positions, but also take into account every position of the user in the path, so as to ensure that the user with the highest path similarity with the new user can be selected as the source user, ensuring the reliability of transfer learning.

[0102] In ultra-dense networks, the user's mobile path and channel status have a profound impact on network performance. In the scenario, each user has its own unique mobile path and channel status. In order to formalize this information, the information of each user can be represented as U i , more specifically

[0103] U i = {Path i ,Ch i} (16)

[0104] Among them, Path irepresents the mobile path information of the i-th user; Ch i represents the channel state information of the i-th user.

[0105] In order to train the model faster in a new environment, a similarity-based transfer learning strategy is proposed. In this section, we will introduce in detail how to calculate the comprehensive similarity between users and how to perform effective transfer learning based on this similarity.

[0106] The user's movement path is a key part of their behavior pattern. In this chapter, it is assumed that the movement paths of all users are straight lines. In order to more accurately determine the similarity between users, the Euclidean distance and cosine similarity are combined for calculation. First, the linear interpolation method is used to divide the movement path of each user from its starting position to the end position into m sampling points, which can provide a high-precision approximation of the user's movement pattern.

[0107]

[0108] For any two users A and B, the Euclidean distance and cosine similarity at each sampling point are calculated. In this way, not only their spatial similarity (through the Euclidean distance) is captured, but also the similarity in their movement direction (through the cosine similarity) can be known.

[0109]

[0110]

[0111] Finally, in order to integrate these two similarities, we use a weight w1 to weightedly combine them.

[0112]

[0113] In this way, we can ensure that the similarity of space and direction is taken into account, and balance the influence of the two by adjusting w1, so that the final similarity evaluation is more accurate.

[0114] In addition to the spatial path, the channel state also plays a crucial role when considering the similarity between users. In the system, each user can choose any of the available subchannels for communication. The gains of these subchannels follow the Rayleigh distribution, because in wireless communications, the signal fading caused by multipath effects often exhibits the characteristics of the Rayleigh distribution.

[0115] In order to measure the similarity of two users in channel state, the Pearson correlation coefficient is used to measure the linear correlation of their channel gains. For each subchannel, we calculate the Pearson correlation coefficient between the channel gains of two users and take the average as the overall channel similarity.

[0116]

[0117] in, and are the average Rayleigh fading of user a and user b on all sub-channels respectively.

[0118] Considering that both the mobile path and channel characteristics have an impact on the user's communication performance, these two similarities are combined to calculate a comprehensive similarity score. By using the weighted average method, the weight w2 is given according to the relative importance of the path and channel. The specific formula is as follows:

[0119]

[0120] Based on this similarity, it can be decided which user to migrate to, thereby training the model faster in the new environment.

[0121] Step S4: Migrate the strategy of the source scenario user with the highest similarity to the new user according to the transfer learning algorithm.

[0122] In a specific embodiment of the present invention, based on the above computational analysis of the similarity between old users and newly joined users in the source scene, the old users with the highest similarity to the newly joined users are selected from the historical scenes as the migration objects. Considering that the resource optimization for each old user in the source scene adopts a deep reinforcement learning algorithm, the transfer learning algorithm proposed in this section adopts the strategy and experience of the most similar users in the source scene to migrate.

[0123] Specifically, refer to Figure 3 As shown, the following are the steps and considerations of this transfer learning algorithm:

[0124] Similarity calculation: Calculate the comprehensive similarity between the new user and each old user in the source scene according to formula (23), and select the old user with the highest similarity to the new user as the migration object.

[0125] Strategy migration: Migrate the old user's strategy to the new user, which means that the new user will adopt the same or similar decision-making process as the old user with the highest similarity.

[0126] Experience reuse: Considering that deep reinforcement learning relies on the experience pool to train and update its strategy, part of the experience pool of old users with the highest similarity is migrated to the experience pool of new users. This provides new users with a richer learning starting point, enabling them to learn and adjust their strategies faster.

[0127] Training with a reduced exploration rate: Since the experience and strategy have been transferred, new users can start with a lower exploration rate, focusing on leveraging the transferred strategy. Over time, the exploration rate is gradually adjusted during the training process based on feedback from the new environment.

[0128] Please provide a simulation experiment. Figure 4 The effectiveness of different learning rate settings in deep reinforcement learning models is compared, including four values: 0.0001, 0.00001, 0.00005, and 0.000005. Analysis shows that the learning rate configuration of 0.00005 achieves the best convergence speed and performance stability during model training. In contrast, higher learning rates such as 0.0001 quickly improve performance in the early stage, but then the gain decreases; while lower learning rates such as 0.000005 show slow progress throughout the training cycle. Based on this, this study determines 0.00005 as the optimal learning rate to promote the model's rapid learning and stable performance in efficiently handling ultra-dense network resource allocation tasks.

[0129] Figure 5 The performance of deep reinforcement learning models with different neuron sizes in the ultra-dense network resource allocation task was compared. This study explored four network configurations: 32-32-32, 64-64-64, 128-128-128, and 256-256-256. The results showed that although the learning speeds of each configuration were similar at the beginning of training, as the training deepened, the 128-128-128 configuration showed significant advantages in performance, reflecting the effectiveness of moderate-sized network structures in dealing with resource allocation problems with complex decisions and dynamic environmental changes. Choosing a network that is too small may not fully capture the complexity of the environment, while a network that is too large may slow down learning efficiency due to computational burden. Therefore, a medium-sized network structure provides an ideal balance for the tasks in this chapter, taking into account both performance and computational efficiency.

[0130] Figure 6The performance comparison of different deep reinforcement learning strategies in a six-user scenario in an ultra-dense network environment is presented. Three training strategies are compared in this study: the first is basic deep reinforcement learning, with an initial exploration rate of 0.1; the second is enhanced exploration deep reinforcement learning, with an initial exploration rate of 0.5; and the third is transfer learning, which applies the old user strategy that is most similar to the existing one to the newly joined user and sets the initial exploration rate to 0.5. The experimental results reveal that although the basic deep reinforcement learning strategy converges slowly, it reaches stable performance after 1200 iterations, while the enhanced exploration deep reinforcement learning and transfer learning strategies achieve fast convergence within 600 iterations. At the same time, the transfer learning strategy not only accelerates the convergence process, but also achieves the best final performance, which reflects that the combination of appropriate exploration rate and effective knowledge transfer can significantly improve the learning efficiency and resource allocation ability of the model. In contrast, the performance of the enhanced exploration deep reinforcement learning strategy is slightly inferior, which may be due to the fact that the higher exploration rate fails to find a sufficiently optimized strategy within a given number of iterations, highlighting the need for a more delicate balance in the selection of exploration rate.

[0131] Figure 7 The performance of a basic deep reinforcement learning strategy and a transfer learning strategy in a seven-user scenario were compared in an ultra-dense network resource allocation task. The basic deep reinforcement learning strategy was trained from scratch, while the transfer learning strategy was adjusted based on the previous training of the six-user scenario to adapt to the newly added users. The results show that the transfer learning strategy quickly surpassed the basic strategy in the early stages of training, demonstrating the potential to accelerate adaptability to new scenarios by leveraging existing learning results. During the iteration process, the transfer learning strategy quickly reached a high performance level and maintained a high stability, while the other two deep reinforcement learning strategies showed slower performance improvement. This result further confirms the effectiveness of transfer learning in quickly adjusting resource allocation strategies in dynamic environments.

[0132] Figure 8 The average system throughput of different methods was compared in detail in the range of 700 to 900 iterations. The results show that in the middle of training, low-exploration deep reinforcement learning and transfer learning achieve higher system throughput faster than traditional deep reinforcement learning. Fig. 9 The figure shows the system throughput comparison between 1300 and 1500 iterations. In this late training stage, due to the low exploration of the low-exploration deep reinforcement learning strategy, it showed a relatively weak state in subsequent iterations, resulting in a lower system throughput. However, the transfer learning algorithm proposed in this paper can still maintain high performance in the later stages of training, surpassing traditional deep reinforcement learning and achieving a higher total system throughput.

[0133] Embodiment 2

[0134] An embodiment of the present invention provides a system for fast spectrum allocation in ultra-dense networks based on transfer learning, referring to Fig.10 As shown, including:

[0135] Model building module 10: used to build a maximum throughput problem model under spectrum reuse and co-channel interference conditions based on the application scenario of dense base station deployment in ultra-dense networks;

[0136] Function determination module 20: used to set the state function, action function and reward function of each user equipment, and train the maximum throughput problem model;

[0137] Similarity calculation module 30: used to calculate the similarity between the new user and the source scene user according to the Euclidean distance, cosine similarity and Pearson correlation coefficient after the number of users changes dynamically;

[0138] The strategy migration module 40 is used to migrate the strategy of the source scenario user with the highest similarity to the new user according to the migration learning algorithm.

[0139] In a specific embodiment of the present invention, the maximum system total throughput is calculated under the constraints of user association and sub-channel allocation:

[0140]

[0141] In the formula, is the transmission power of base station i on subchannel n, is the signal-to-interference-noise ratio of user k and base station i on signal n at time t; C1 and C2 are binary variables, representing the association between the user, the base station and the channel respectively; C3 and C4 respectively indicate that a user can only select one base station and one channel at time t; C5 limits the transmission power of the base station to not exceed its maximum power.

[0142] In a specific embodiment of the present invention, the user's position at t is represented as a two-dimensional vector (x k (t),y k (t)), each agent in the network needs to know the information of the entire network to decide the action selection, and each agent needs to determine the optimal strategy according to its own position. Therefore, the user k state is composed of the channel selection matrix, the power selection matrix and the position state:

[0143] s k (t)={C(t),B(t),d k (t)}

[0144] Among them, s k(t) is the state observed by user k in time slot t, C(t) is the set of channel selections for all users, and B(t) is the set of base station selections associated with all users.

[0145] In a specific embodiment of the present invention, the best associated base station and channel of the user are determined according to the base station location and signal strength in the dynamic network environment, which is expressed as:

[0146] a k (t) = {b k (t),c k (t)}

[0147] Among them, a k (t) is the action selected by user k at time t; b k (t) is the base station selection of user k at time t; c k (t) is the channel selection of user k at time t.

[0148] In a specific embodiment of the present invention, the agent is motivated to take actions to improve the throughput according to the system throughput, and the motivation function is expressed as:

[0149]

[0150] Among them, B is the base station index set, U is the user set, n is the selected channel, r i is the service radius of base station i, and d is the service range of the base station associated with the user connection.

[0151] In a specific embodiment of the present invention, the moving path and channel state of each user are determined according to the following formula:

[0152] U i = {Path i ,Ch i}

[0153] Among them, U i Information for each user; Path i represents the mobile path information of the i-th user; Ch i represents the channel state information of the i-th user.

[0154] In a specific embodiment of the present invention, a linear interpolation method is used to divide the moving path of each user from its starting position to its ending position into m sampling points, and the spatial similarity and moving direction similarity between the new user and the source scene user are calculated based on the Euclidean distance and cosine similarity.

[0155] In a specific embodiment of the present invention, the similarity of the channel states of two users and the linear correlation of the channel gains are measured according to the Pearson correlation coefficient. For each subchannel, the Pearson correlation coefficient between the channel gains of the two users is calculated, and the average value is taken as the overall channel similarity.

[0156] In a specific embodiment of the present invention, two weights are designed to respectively calculate the comprehensive similarity of the spatial similarity and the moving direction similarity and the similarity determined according to the relative importance of the path and the channel.

[0157] The present invention provides an electronic device, comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the fast spectrum resource optimization method based on transfer learning according to the first aspect.

[0158] Optionally, the above-mentioned electronic device may be a server.

[0159] In addition, an embodiment of the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the method for rapid spectrum resource optimization based on transfer learning as described in the first aspect.

[0160] The rapid development of network technology has promoted the densification of network infrastructure. In view of the problem that the number of users in ultra-dense networks is constantly changing and the trained allocation strategy is difficult to quickly adapt to network changes, a model is constructed to maximize the long-term network throughput after the dynamic increase of the number of users, and a fast resource allocation algorithm based on transfer learning is proposed. In the source scenario, each user has a fixed mobile path, and deep reinforcement learning is used to optimize the strategy of associating base stations and selecting channels. After the number of users changes dynamically, a comprehensive similarity calculation method is designed to analyze the similarity between the newly added users and the old users in the source scenario, and the strategy of the old users with the highest similarity is migrated to the new users as the initial strategy for further training and optimization. The simulation results show that the resource allocation algorithm using the transfer learning strategy has higher efficiency and effect in terms of the speed of learning the optimal strategy and optimizing the system throughput compared with the traditional deep reinforcement learning method.

[0161] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0162] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referenced to each other. The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the functions in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing terminal device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1The steps of the functions specified in one or more boxes. Although the preferred embodiments of the embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they know the basic creative concept. Therefore, the attached claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention. Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device including a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0163] The method and device provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principle and implementation mode of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation mode and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

[0164] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", "one specific embodiment" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A fast spectrum allocation method for ultra-dense networks based on transfer learning, It is characterized in that include: Step S1: Based on the application scenario of dense base station deployment in an ultra-dense network, a maximum throughput problem model under spectrum reuse and co-channel interference conditions is constructed; Step S2: setting the state function, action function and reward function of each user device, and training the maximum throughput problem model; Step S3: After the number of users changes dynamically, the similarity between the new user and the source scene user is calculated based on the Euclidean distance, cosine similarity and Pearson correlation coefficient; Step S4: Migrate the strategy of the source scenario user with the highest similarity to the new user according to the transfer learning algorithm.

2. The ultra-dense network fast spectrum allocation method based on transfer learning according to claim 1, It is characterized in that In step S1, the maximum system total throughput is calculated under the constraints of user association and subchannel allocation: In the formula, is the transmission power of base station i on subchannel n, is the signal-to-interference-noise ratio of user k and base station i on signal n at time t; C1 and C2 are binary variables, representing the association between the user, the base station and the channel respectively; C3 and C4 respectively indicate that a user can only select one base station and one channel at time t; C5 limits the transmission power of the base station to not exceed its maximum power.

3. The ultra-dense network fast spectrum allocation method based on transfer learning according to claim 1, It is characterized in that In step S2, the user's position t is represented as a two-dimensional vector (x k (t),y k (t)), each agent in the network needs to know the information of the entire network to decide the action selection, and each agent needs to determine the optimal strategy according to its own position. Therefore, the user k state is composed of the channel selection matrix, the power selection matrix and the position state: s k (t)={C(t),B(t),d k (t)} Among them, s k (t) is the state observed by user k in time slot t, C(t) is the set of channel selections for all users, and B(t) is the set of base station selections associated with all users.

4. The ultra-dense network fast spectrum allocation method based on transfer learning according to claim 3, It is characterized in that According to the base station location and signal strength in the dynamic network environment, the best associated base station and channel for the user are determined, which is expressed as: a k (t)={b k (t),c k (t)} Among them, a k (t) is the action selected by user k at time t; b k (t) is the base station selection of user k at time t; c k (t) is the channel selection of user k at time t.

5. The ultra-dense network fast spectrum allocation method based on transfer learning according to claim 4, It is characterized in that According to the system throughput, the agent is encouraged to take actions to improve the throughput. The incentive function is expressed as: Among them, B is the base station index set, U is the user set, n is the selected channel, r i is the service radius of base station i, and d is the service range of the base station associated with the user connection.

6. The ultra-dense network fast spectrum allocation method based on transfer learning according to claim 1, It is characterized in that In step S3, the moving path and channel state of each user are determined according to the following formula: U i ={Path i ,Ch i } Among them, U i Information for each user; Path i represents the mobile path information of the i-th user; Ch i represents the channel state information of the i-th user.

7. The ultra-dense network fast spectrum allocation method based on transfer learning according to claim 6, It is characterized in that In step S3, a linear interpolation method is used to divide the moving path of each user from its starting position to its ending position into m sampling points, and the spatial similarity and moving direction similarity between the new user and the source scene user are calculated based on the Euclidean distance and cosine similarity.

8. The ultra-dense network fast spectrum allocation method based on transfer learning according to claim 7, It is characterized in that In step S3, the similarity of the channel states of the two users and the linear correlation of the channel gains are measured according to the Pearson correlation coefficient. For each subchannel, the Pearson correlation coefficient between the channel gains of the two users is calculated, and the average value is taken as the overall channel similarity.

9. The ultra-dense network fast spectrum allocation method based on transfer learning according to claim 8, It is characterized in that Two weights are designed to respectively calculate the comprehensive similarity of the spatial similarity and the moving direction similarity and the similarity determined according to the relative importance of the path and the channel.

10. A fast spectrum allocation system for ultra-dense networks based on transfer learning, It is characterized in that include: Model building module: used to build a maximum throughput problem model under spectrum reuse and co-channel interference conditions based on the application scenario of dense base station deployment in ultra-dense networks; Function determination module: used to set the state function, action function and reward function of each user device and train the maximum throughput problem model; Similarity calculation module: used to calculate the similarity between new users and source scenario users based on Euclidean distance, cosine similarity and Pearson correlation coefficient after the number of users changes dynamically; Strategy migration module: used to migrate the strategy of the source scenario user with the highest similarity to the new user according to the migration learning algorithm.

Citation Information

Patent Citations

  • Unmanned aerial vehicle base station dynamic deployment method based on user trajectory prediction

    CN114710786A

  • Fast phase shift optimization method for intelligent reflection surface assisted unmanned aerial vehicle line patrol system

    CN117336735A

  • Internet of vehicles transmission scheduling method based on migration reinforcement learning

    CN117596700A

  • Multi-resource optimization method for isomerized semantic and bit communication network

    CN117858124A

  • Identifying source datasets that fit a transfer learning process for a target domain

    US20220027339A1