A method and system for resource allocation in an ultra-dense base station network system

Abstract

The application discloses a kind of super dense base station network system resource allocation method, system, it is related to 5G / 6G wireless communication network technical field.The super dense base station network system resource allocation method and system provided by the present application, by circular diffusion algorithm and K-means algorithm, the super dense base station network of target area established is adaptively hybrid clustered, effectively separates the base station of high-density aggregation, effectively balances the base station load and reduces intra-cluster interference;On this basis, the energy efficiency and spectral efficiency of the base station network after clustering are calculated, a joint reward function is constructed and a double-headed competitive deep Q network is trained, effectively realizing the joint optimization of energy efficiency and spectral efficiency, then using the trained double-headed competitive deep Q network to generate the resource allocation strategy of base station network, solving the dimension unfairness and Q value overestimation problem existing in the existing single network structure, making the resource allocation more accurate and more robust, so that the system achieves the best balance between energy efficiency and spectral efficiency.

Description

Technical Field

[0001] This application relates to the field of 5G / 6G wireless communication network technology, and in particular to a resource allocation method, system, medium and device for an ultra-dense base station network system. Background Technology

[0002] In the evolution of 5G / 6G ultra-dense networks (UDN), high-density deployment of low-power small base stations (LBS) within the coverage area of ​​macro base stations has become the mainstream technical solution to improve system spectrum reuse and regional capacity. However, with the increase in base station deployment density, co-channel interference between base stations becomes increasingly complex, and the accumulated circuit power consumption of massive base stations leads to low system energy efficiency.

[0003] To address this complex resource allocation challenge, existing technical solutions mainly fall into three categories: traditional mathematical optimization, network topology management, and basic artificial intelligence decision-making. In traditional mathematical optimization, early research often employed iterative water-filling algorithms or distributed power control strategies based on game theory. These methods establish rigorous mathematical models and utilize Lagrange multiplier methods or Nash equilibrium to solve for optimal resource allocation. To reduce the computational complexity of network-wide optimization, network topology management techniques are widely adopted, with dynamic clustering being the most typical. This type of technology typically divides a large-scale network into several independent sub-clusters based on the geographical coordinates of base stations or the channel gain matrix, using algorithms such as K-means and hierarchical clustering. This allows for local interference management within each cluster, aiming to achieve interference isolation and reduce signaling overhead. However, in real-world scenarios, base stations exhibit a physically non-uniform distribution characterized by "hotspot clustering and sparse edges." Traditional K-means clustering algorithms, which force the division into clusters of similar size, can lead to severe interference within hotspot clusters. Furthermore, simple geometric division cannot fit irregular hotspot boundaries, and existing base station clustering algorithms lack the ability to adaptively perceive base station density.

[0004] Furthermore, with the emergence of the demand for "green communication", optimizing energy efficiency (EE, Mb / J) and spectrum efficiency (SE, bps / Hz) has become a research hotspot. However, in wireless resource management, energy efficiency and spectrum efficiency are usually a pair of contradictory indicators that are mutually restrictive. Simply pursuing high spectrum efficiency often requires increasing transmit power, which leads to a decrease in energy efficiency, and vice versa.

[0005] In dealing with multi-objective optimization problems, existing technologies typically employ the weighted sum method, which transforms EE and SE into a single scalar objective function by setting weight factors. This approach, focusing on optimizing a single metric, struggles to achieve an optimal balance between the two and fails to meet the requirements of 6G "green communication." Furthermore, to address the dynamic and time-varying characteristics of wireless environments, Q-learning reinforcement learning based on lookup tables has gradually replaced static optimization algorithms. However, EE and SE exhibit significant numerical differences (e.g., EE might be 0.5, while SE might be 10), and existing technologies lack effective mechanisms to eliminate this dimensional unfairness, leading to the failure of joint optimization. Moreover, in complex interference environments, the single network structure of reinforcement learning is prone to Q-value overestimation, resulting in unstable network training and unsustainable resource allocation strategies. Summary of the Invention

[0006] Therefore, it is necessary to provide a resource allocation method, system, medium, and device for ultra-dense base station network systems to address the aforementioned technical problems.

[0007] The following technical solution is adopted in this specification: This specification provides a resource allocation method for an ultra-dense base station network system, including: Based on resource block bandwidth, transmission power, and circuit static power consumption, an ultra-dense base station network system for the target area is established. After clustering the dense base stations of the ultra-dense base station network in the target area using the circular diffusion algorithm, the remaining base stations are then clustered using the K-means algorithm. Calculate the energy efficiency and spectral efficiency of the clustered ultra-dense base station network system, construct a joint reward function by performing reward shaping and normalization weighting on the energy efficiency and the spectral efficiency, and train a dual-head competitive deep Q network through the joint reward function; The current state data of the clustered ultra-dense base station network system is obtained, and the current state data is input into the trained dual-head contention deep Q network to generate Q-value vectors. The resource allocation strategy of the ultra-dense base station network system in the target area is obtained according to the Q-value vectors, and resource blocks are allocated to system users based on the resource allocation strategy.

[0008] Optionally, after clustering the dense base stations of the ultra-dense base station network in the target area using the circular diffusion algorithm, the remaining base stations are then clustered using the K-means algorithm, including: Traverse all base stations of the ultra-dense base station network in the target area and establish a base station density matrix; The base station density matrix is ​​sorted by density value and convolution is performed. An initial diffusion center is selected, and circular iterative diffusion is performed according to the diffusion speed and step size to group the base stations in the diffusion area into a cluster. The remaining base stations are clustered using the K-means algorithm.

[0009] Optionally, the dual-head competitive deep Q-network includes a network input layer, a shared feature extraction layer, and independent energy efficiency evaluation heads and spectral efficiency evaluation heads. The network input layer receives state data from the base station system. The shared feature extraction layer is a fully connected layer with 128 neurons and ReLU activation, used to extract general environmental state features from the state data. The energy efficiency evaluation heads and spectral efficiency evaluation heads adopt fully connected neural network layers, used to learn the system's action strategy from the environmental state features and evaluate the system state value.

[0010] Optionally, both the energy efficiency evaluation head and the spectral efficiency evaluation head include a state value flow branch and an action advantage flow branch. The state value flow branch and the action advantage flow branch are combined through a Dueling aggregation layer and output energy efficiency Q-value vector and spectral efficiency Q-value vector, respectively.

[0011] Optionally, the energy efficiency and spectral efficiency of the ultra-dense base station network system after clustering include: Based on resource block bandwidth Calculate user rate , Channel gain; Based on the user rate Transmit power and circuit static power consumption Calculate energy efficiency , ; Spectral efficiency .

[0012] Optionally, a joint reward function is constructed by weighting and normalizing the energy efficiency and the spectral efficiency, specifically including: Regarding the energy efficiency and spectral efficiency Perform reward-based plastic surgery to obtain energy efficiency rewards. and spectrum efficiency bonus ,Right now: Penalty is a penalty item. This is the magnification factor; The energy efficiency reward and spectrum efficiency rewards Perform normalized weighting to construct a joint reward function. ,in, As a weighting factor, As a reward for energy efficiency, The spectral efficiency is rewarded, while Penalty is penalized.

[0013] Optionally, the step of obtaining the current state data of the clustered ultra-dense base station network system, inputting the current state data into the trained dual-head contention deep Q-network to generate a Q-value vector, obtaining the resource allocation strategy of the target area ultra-dense base station network system based on the Q-value vector, and allocating resource blocks to system users based on the resource allocation strategy includes: The current state data of the clustered ultra-dense base station network system is obtained and logarithmically normalized to obtain a high-dimensional vector. This vector is then input into the trained dual-head competitive deep Q network to generate energy efficiency Q-value vectors and spectral efficiency Q-value vectors, which are then subjected to Min-Max normalization. The normalized energy efficiency Q-value vector and the spectral efficiency Q-value vector are weighted and summed to obtain the composite Q-value vector. Resource block indexes are obtained based on the synthesized Q-value vectors, and resources are allocated to users of the ultra-dense base station network system according to the resource block indexes.

[0014] This specification provides a resource allocation system for an ultra-dense base station network system, including: A module is established to build an ultra-dense base station network system for a target area based on resource block bandwidth, transmission power, and circuit static power consumption. The clustering module uses the circular diffusion algorithm to cluster the dense base stations in the ultra-dense base station network of the target area, and then uses the K-means algorithm to cluster the remaining base stations. The computation and training module is used to calculate the energy efficiency and spectral efficiency of the clustered ultra-dense base station network system, construct a joint reward function by performing reward shaping and normalization weighting on the energy efficiency and the spectral efficiency, and train a dual-head competitive deep Q network through the joint reward function. The generation and allocation module is used to obtain the current state data of the clustered ultra-dense base station network system, input the current state data into the trained dual-head contention deep Q network to generate Q-value vectors, obtain the resource allocation strategy of the target area ultra-dense base station network system according to the Q-value vectors, and allocate resource blocks to system users based on the resource allocation strategy.

[0015] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described resource allocation method for an ultra-dense base station network system.

[0016] This specification provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described resource allocation method for an ultra-dense base station network system.

[0017] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects: The resource allocation method and system for ultra-dense base station networks provided in this specification adaptively hybridizes and clusters the ultra-dense base station network in the target area using the circular diffusion algorithm and the K-means algorithm. This ensures that the base stations within each cluster are relatively concentrated and that the number of base stations within each cluster is kept as equal as possible, thereby evenly distributing the number of base stations in each cluster. This effectively separates high-density clustered base stations, effectively balances the base station load, alleviates the problem of excessive subcarrier reuse, and reduces intra-cluster interference. Based on this, the energy efficiency and spectral efficiency of the clustered base station network are calculated and reward-shaped and normalized weighted. A joint reward function is then constructed and a dual-headed contention deep Q-network is trained to effectively achieve joint optimization of energy efficiency and spectral efficiency. Then, based on the current state data of the system, the trained dual-headed contention deep Q-network generates a Q-value vector, and the resource block index of the base station network is obtained based on the Q-value vector. Resource blocks are then allocated to users accordingly. This solves the problems of dimensional unfairness and Q-value overestimation existing in the existing single network structure, making resource allocation more accurate and robust, thereby achieving the best balance between energy efficiency and spectral efficiency. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0019] Figure 1 This document provides a flowchart illustrating a resource allocation method for an ultra-dense base station network system. Figure 2 This is a schematic diagram of the adaptive hybrid clustering process for ultra-dense base station networks in a target area using the circular diffusion algorithm and the K-means algorithm, as provided in this manual. Figure 3 This is a schematic diagram illustrating the simulation results of block division of the LBS base station distribution area provided in this specification. Figure 4 This is a schematic diagram of the density matrix provided in this specification; Figure 5 This is a schematic diagram of the density matrix after convolution calculation provided in this specification. Figure 6This is a schematic diagram of the LBS clustering results using the circular diffusion combined with the K-means algorithm provided in this manual. Figure 7 This is a schematic diagram of the LBS clustering results using the pure K-means algorithm provided in this manual; Figure 8 This is a schematic diagram of the LBS clustering results using the K-means algorithm based on hierarchical clustering, as provided in this manual. Figure 9 This is a schematic diagram of the structure of the dual-head competitive deep Q-network provided in this specification; Figure 10 This is a flowchart illustrating the process of constructing a joint reward function based on energy efficiency and spectral efficiency, as provided in this specification. Figure 11 This document provides a flowchart illustrating the process of generating a resource allocation strategy for the target area's ultra-dense base station network system using a trained dual-head contention deep Q-network, and allocating resource blocks to system users based on the resource allocation strategy. Figure 12 The overall energy efficiency and spectral efficiency changes and loss curves during the simulation experiment training provided in this manual; Figure 13 This manual provides individual energy efficiency curves for each cluster of the system in the simulation experiments. Figure 14 This document provides individual spectral efficiency curves for each cluster of the system in the simulation experiments. Figure 15 This specification provides a schematic diagram of a resource allocation system for an ultra-dense base station network system. Figure 16 This is a schematic diagram of a computer device used to implement a resource allocation method for an ultra-dense base station network system, as provided in this specification. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this application.

[0021] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0022] This invention provides a resource allocation method for an ultra-dense base station network system. Figure 1 The flowchart of this method is shown, and it specifically includes the following steps: S101. Based on resource block bandwidth, transmission power and circuit static power consumption, establish an ultra-dense base station network system for the target area.

[0023] For a specific target area (such as a residential area, train station, shopping mall, etc.), a model of an ultra-dense base station network system that conforms to the physical reality of the target area is first constructed, including: Configure resource block bandwidth: Bandwidth per resource block (RB) .

[0024] System total power consumption model: System total power consumption It consists of the transmit power of the activated base station and the static power consumption of the basic circuitry of all base stations: in, For transmission power, This refers to the circuit's static power consumption. This indicates the set of active base stations.

[0025] In this embodiment of the invention, a resource block (RB) is a wireless spectrum resource unit with a bandwidth of 180kHz. It is the core operational object for the system to make multi-user resource allocation decisions. The final allocation strategy is to assign a specific resource block number to each user, aiming to achieve an optimal balance between network energy efficiency and spectrum efficiency. Each resource block has unique channel capacity, gain (signal quality), and neighboring cell interference tolerance for different users at different times. This dynamic information constitutes the state input for system decision-making. The base station needs to allocate these resource blocks to users within its coverage area so that users can transmit data.

[0026] S102. After clustering the dense base stations of the ultra-dense base station network in the target area using the circular diffusion algorithm, the remaining base stations are then clustered using the K-means algorithm.

[0027] Circular diffusion algorithms leverage the spatial density characteristics of ultra-dense base stations. By selecting a high-density region as the center, the diffusion speed is controlled by the area of ​​the diffusion region and the number of base stations within that region. This ensures that after clustering, the number of base stations in different density clusters is as similar as possible, allowing users within each cluster to receive the most equal bandwidth resources, thus improving user fairness. However, circular diffusion algorithms cannot cluster all base stations. After clustering the high-density base stations, the remaining sparsely distributed base stations require the assistance of the K-means algorithm for clustering. Therefore, combining circular diffusion with the K-means algorithm can effectively achieve adaptive clustering of base stations in ultra-dense, unevenly distributed base station scenarios.

[0028] Specifically, such as Figure 2As shown, step S102 includes: S1021. Traverse all base stations of the ultra-dense base station network in the target area and establish a base station density matrix; S1022. Sort the density values ​​and perform convolution calculation on the base station density matrix, select an initial diffusion center, and perform circular iterative diffusion according to the diffusion speed and step size to group the base stations in the diffusion area into a cluster. S1023. Perform clustering of the remaining base stations according to the K-means algorithm.

[0029] In this embodiment, firstly, all base stations within the target area are traversed to obtain the upper and lower boundaries of the distribution area in the horizontal and vertical directions, and then the area is divided into M equal parts to obtain... For each base station block, create an identical M×M square matrix, denoted as the base station density matrix. , .in, This represents the density value of the base station block in the i-th row and j-th column, i j Then, the density values ​​are arranged in descending order until the sum of the selected density values ​​exceeds 20% of the total number of base stations. The center of the block corresponding to the selected density value is then used as the initial diffusion center for the circular diffusion algorithm. If there are identical density values, the density matrix is ​​convolved and then compared.

[0030] The convolution calculation process is similar to the CNN convolution process, using a density matrix. The i-th row and j-th column 3×3 matrix centered Described as a matrix , convolution kernel Recorded as For the matrix , Convolution calculation is performed using the following formula: in, It is a matrix The Line number Column elements, It is a matrix The Line number Column element, i j ; After completing matrix operations The value at the corresponding position, i j Traverse the density matrix Use all elements in the array to complete the above convolution calculation.

[0031] The reason for performing the above convolution calculation is that for blocks with the same density value but a larger value after convolution, it means that there are more surrounding base stations for the block location corresponding to that density value. Therefore, the center of the block is more suitable as the initial diffusion center of the circular diffusion algorithm.

[0032] After selecting the initial diffusion center, the diffusion region is updated incrementally from the diffusion center according to the diffusion speed and step size to achieve circular iterative diffusion. All base stations contained in the diffusion region after diffusion termination are divided into a cluster. The step size is calculated as follows: Where counts is the number of base stations within the circular diffusion cluster. The weights represent the number of base stations within the circular diffusion cluster, where area represents the circular diffusion area. This is the weight of the area of ​​the circular diffusion cluster; the "1" in the formula is to avoid a zero denominator during calculation. Iterate gradually until the diffusion rate is less than... or reach All base stations within the diffusion area after the diffusion ends are treated as a cluster. When diffusion areas overlap, base stations will be assigned to the nearest cluster based on their distance from the diffusion center.

[0033] After the circular diffusion algorithm completes the clustering of high-density base stations, the remaining sparsely distributed base stations need to be clustered using the K-means algorithm. The four corners of the quadrilateral distribution area of ​​the ultra-dense base stations are selected as the starting positions for the K-means algorithm, which then performs clustering of the remaining base stations. This completes the clustering of all base stations within the target area.

[0034] Ultra-dense base station networks are defined as LBS base station densities ranging from 100 to 2000 per km². This invention selects 1000 base stations per km² as the LBS base station density. Taking a square area with dimensions of 400m as an example, assuming 160 LBS base stations exist within this area, the locations of the LBS base stations are randomly generated using a combination of normal and uniform distributions, and the distribution area of ​​the LBS base stations is divided into blocks. Simulation results are as follows: Figure 3 As shown, there are several high-density regions in the LBS distribution, where LBS points are densely distributed; while outside these regions, the distribution density of LBS points is significantly reduced, exhibiting sparse characteristics.

[0035] like Figure 3 As shown, the LBS distribution area is divided into 256 blocks of 16×16. Then, a density matrix of the same dimension is created. All blocks are traversed, and the number of LBS instances is counted and filled into the density matrix. The density matrix is ​​as follows. Figure 4As shown; the density matrix after convolution is as follows Figure 5 As shown. After convolution, the numerical values ​​in the resulting matrix can more effectively reflect the number of LBSs surrounding the corresponding block.

[0036] Then, the density values ​​are arranged in descending order until the sum of the selected density values ​​exceeds 20% of the total number of LBSs. The center of the block corresponding to the selected density value is used as the initial diffusion center for the circular diffusion algorithm. Diffusion proceeds gradually from the initial diffusion center according to the diffusion speed and step size, achieving circular diffusion clustering. The remaining sparsely distributed LBSs are clustered using the K-means algorithm. The final clustering result combining circular diffusion and the K-means algorithm is shown below. Figure 6 As shown. By Figure 6 As can be seen, the clustering results include 3 clusters generated based on the circular diffusion algorithm and 4 clusters generated based on the K-means algorithm. In terms of distribution, the circular diffusion clustering results exhibit relatively concentrated LBS points and maintain a similar number of LBS points within each cluster. Therefore, the circular diffusion algorithm can effectively separate high-density clustered LBS base stations, thereby reducing the imbalance in the number of LBS points within each cluster. Ultimately, the combination of circular diffusion and the K-means algorithm achieves adaptive sensing of base station density.

[0037] To verify the effectiveness of this invention, this embodiment also includes a comparative example of clustering using a pure K-means algorithm and a hierarchical clustering-based K-means algorithm under the same settings. The LBS clustering results using the pure K-means algorithm are as follows: Figure 7 As shown, the LBS clustering results using the K-means algorithm based on hierarchical clustering are as follows: Figure 8 As shown, the K-means clustering algorithm has two significant characteristics in LBS clustering: First, the algorithm tends to aggregate spatially dense LBS into a single, excessively large cluster. The improved K-means algorithm based on hierarchical clustering not only retains this characteristic but also shows a tendency to be overly sensitive to discrete points, often dividing spatially isolated LBS into separate clusters.

[0038] Table 1 shows the data for three clustering algorithms using the same LBS distribution in terms of the number of clusters, overall variance, number of LBSs within a cluster, and variance of the number of LBSs.

[0039] Table 1 Comparison of Three Clustering Algorithms As shown in Table 1, the K-means algorithm based on hierarchical clustering has a large variance in the number of LBSs in each cluster, and the distribution of LBSs is extremely uneven. In clusters with too many LBSs, excessive reuse of the same subcarrier can easily occur. This excessive reuse of subcarriers can cause excessive intra-cluster interference, greatly reducing intra-cluster channel capacity and lowering the overall system capacity. In contrast, the clustering algorithm combining circular diffusion and K-means in this invention distributes the number of LBSs in each cluster more evenly, alleviating the problem of excessive subcarrier reuse from the source, effectively balancing the base station load, reducing intra-cluster interference, improving system capacity, and providing a better network topology foundation for subsequent resource allocation.

[0040] S103. Calculate the energy efficiency and spectral efficiency of the clustered ultra-dense base station network system. After reward-shaping and normalization weighting of the energy efficiency and spectral efficiency, construct a joint reward function. Train a dual-head competitive deep Q-network through the joint reward function.

[0041] Based on the hybrid clustering of circular diffusion and K-means in the aforementioned ultra-dense base station network, this invention constructs a feature-decoupled dual-head competitive deep Q-network based on a deep neural network architecture to simultaneously optimize the inherently conflicting objectives of system energy efficiency (EE) and spectral efficiency (SE). Specifically, as shown... Figure 9 As shown, the Dual-Head Dueling Deep Q-Network (DQN) includes a network input layer, a shared feature extraction layer, and independent energy efficiency evaluation heads and spectral efficiency evaluation heads. The network input layer receives the state data of the base station system. The shared feature extraction layer extracts general environmental state features from the state data. The energy efficiency evaluation heads and spectral efficiency evaluation heads learn the system's action strategy from the environmental state features and evaluate the system's state value. Both the energy efficiency evaluation head and the spectral efficiency evaluation head include a state value flow branch and an action advantage flow branch. Each branch can use a fully connected neural network layer. The state value flow branch and the action advantage flow branch are combined through a Dueling aggregation layer and output energy efficiency Q-value vectors and spectral efficiency Q-value vectors, respectively.

[0042] The network input layer receives the current environmental state vector of the base station system. The data is then logarithmically normalized to form high-dimensional (64-dimensional) Channel State Information (CSI) data, with each dimension corresponding to a sub-channel resource block. In this embodiment, the environment state vector... At least include:

[0043] Local channel characteristics The current user is in all Channel gain (or signal-to-interference-plus-noise ratio, SINR) on each sub-channel. Interference characteristics Interference power levels of adjacent base stations on each sub-channel; Historical Action Strategy The resource allocation strategy at the previous moment.

[0044] Environment state vector Measured and reported by the base station and passed Preprocessing is performed. The raw data measured at the base station.

[0045] The shared feature extraction layer can be a deep convolutional neural network, specifically a fully connected layer with 128 neurons and ReLU activation. The shared feature extraction layer extracts features from the environmental state vector. The shared feature extraction layer extracts common environmental features (such as deep fading locations and congested frequency bands) for use by two subsequent dedicated heads. Since the physical characteristics of wireless channels (such as path loss and shadowing) and the distribution of electromagnetic environments (such as neighboring cell interference) are objective physical facts, their impact on energy efficiency and spectral efficiency is fundamentally consistent (e.g., deep fading frequency bands cause both metrics to deteriorate simultaneously). Therefore, the shared feature extraction layer is responsible for abstracting these common environmental state representations from the original high-dimensional CSI data, such as locating sub-channel indices in deep fading and identifying high-interference-intensity spectral regions. These refined general feature vectors are then fed into independent energy efficiency evaluation and spectral efficiency evaluation heads, allowing subsequent networks to focus on learning differentiated decision-making strategies for specific objectives without repeatedly learning fundamental channel physics.

[0046] The dedicated head for energy efficiency assessment contains an independent state-value stream. Branch paths and action advantage flow Branch lines are specifically used to assess energy efficiency. Used to assess the overall energy-saving potential (energy consumption) of the system under the current condition. Used to evaluate the relative contribution of selecting a specific sub-channel to energy efficiency. The dedicated header for spectral efficiency evaluation contains a separate state value stream. Branch paths and action advantage flow Branch circuits are specifically used to evaluate spectral efficiency.

[0047] The state value flow branch is a sub-network that outputs a scalar (single-value) and evaluates the overall quality of the current wireless environment (baseline score). The action advantage flow branch is a sub-network that outputs a vector (multi-value, dimension equal to the number of sub-channels) and evaluates the performance of each sub-channel relative to the average level (relative score). The outputs of the state value flow branch and the action advantage flow branch of each dedicated head are combined through a Dueling aggregation layer, and then output energy efficiency Q-value vectors respectively. and spectral efficiency Q-value vector The Q-value of the Dueling aggregation layer is calculated using the following formula:

[0048] in, , The size of the action space (64 dimensions).

[0049] The aforementioned dual-head competitive deep Q-network model decouples the learning of 'environment state' and 'action selection', thereby achieving faster convergence in complex ultra-dense base station network interference environments. It eliminates the problem of Q-value overestimation in existing algorithms, resulting in more stable model training and a more robust generated resource allocation strategy. Of course, before using the aforementioned dual-head competitive deep Q-network model, it must be constrained by the joint reward function constructed based on energy efficiency and spectral efficiency according to this invention, as well as the environment state vector. Pre-training is performed using the action strategy data from the previous moment.

[0050] Furthermore, in step S103 above, the steps of calculating the energy efficiency and spectral efficiency of the clustered ultra-dense base station network system include: Based on resource block bandwidth Calculate user rate , Channel gain; Based on user rate Transmit power and circuit static power consumption Calculate energy efficiency , ; Spectral efficiency .

[0051] in, This represents the base station transmit power, a system configuration parameter (e.g., 1.0 W), which is not dynamically adjusted with changes in the channel to reduce hardware control complexity. This refers to the fixed power consumption required for the base station to maintain the operation of baseband processing, radio frequency links, and heat dissipation systems when it is not transmitting signals. In this embodiment, the power is set to 5.0 W according to the hardware specifications. express

[0052] The number of active users is counted in real time by the base station's Media Access Control (MAC) layer scheduler, representing the total number of users currently connected to the base station and requesting data transmission.

[0053] Energy efficiency Spectrum efficiency is the ratio of the data transfer rate obtained by a user on a resource block to the total power consumed (transmit power + circuit power consumption), measuring the "cost-effectiveness" of using that resource block; It represents the ratio of the data transfer rate obtained by a user on a certain resource block to the bandwidth of that resource block, and measures the utilization efficiency of that resource block.

[0054] Furthermore, such as Figure 10 As shown, in step S103 above, a joint reward function is constructed by weighting and normalizing the energy efficiency and spectral efficiency after reward shaping, specifically including: S1031, Regarding energy efficiency and spectral efficiency Perform reward-based plastic surgery to obtain energy efficiency rewards. and spectrum efficiency rewards ,Right now: Penalty is a penalty item. This is the magnification factor; S1032, Energy Efficiency Rewards and spectrum efficiency rewards Perform normalized weighting to construct a joint reward function. ,in, is the weighting factor, and Penalty is the penalty term.

[0055] In this embodiment, the amplification factor Scale_EE is set to 20.0; when the user rate When the speed is Mbps, Penalty is -5.0; otherwise, it is +2.0. This applies to energy efficiency. Computational and spectral efficiency Rewards , Nonlinear amplification and penalized shaping are performed, followed by Min-Max dynamic range normalization. Forcing the reward to fit within the same range (usually [0, 1]) allows assigning equal weights to both rewards, preventing gradient vanishing and thus facilitating the process. , Constructed joint reward function To better train dual-head competitive deep Q-networks.

[0056] S104. Obtain the current state data of the clustered ultra-dense base station network system, input the current state data into the trained dual-head competition deep Q network to generate Q-value vectors, obtain the resource allocation strategy of the target area ultra-dense base station network system according to the Q-value vectors, and allocate resource blocks to system users based on the resource allocation strategy.

[0057] like Figure 11As shown, step S104 specifically includes: S1041. Obtain the current state data of the clustered ultra-dense base station network system and perform log normalization to obtain a high-dimensional vector. Then, input the vector into the trained dual-head competitive deep Q network to generate the energy efficiency Q value vector and the spectral efficiency Q value vector, and perform Min-Max normalization on them respectively. S1042. The normalized energy efficiency Q-value vector and spectral efficiency Q-value vector are weighted and summed to obtain the composite Q-value vector. S1043. Obtain the resource block index based on the synthesized Q-value vector, and allocate resource blocks to users of the ultra-dense base station network system according to the resource block index.

[0058] pass , Constructed joint reward function The trained dual-head competitive deep Q-network effectively achieves joint optimization of energy efficiency and spectral efficiency, solving the problems of dimensional unfairness and overestimation of Q-values ​​generated by existing single network structures. This helps to make subsequent resource allocation more accurate and robust, thereby enabling the system to achieve the best balance between energy efficiency and spectral efficiency.

[0059] First, obtain the current state data of the clustered ultra-dense base station network system. After performing logarithmic normalization to obtain a high-dimensional vector, it is input into the trained dual-head competitive deep Q-network to generate energy efficiency Q-value vectors. and spectral efficiency Q-value vector Then, the Min-Max normalization function is used. The energy efficiency Q-value vector output by the network and spectral efficiency Q-value vector The two indicators are mapped to the dimensionless interval [0, 1]: in, and The dynamic maintenance method is used to obtain the following: The system maintains an "experience pool" or "sliding window" that records the observations in the most recent N steps (e.g., the most recent 1000 steps). The value is then updated in real time: ; and Obtain it in the same way.

[0060] Then, the normalized energy efficiency Q-value vector and the spectral efficiency Q-value vector are weighted and summed to obtain the composite Q-value vector. By employing the dual mechanism of signal amplification and Q-value normalization for Scale_EE, the gradient weight of energy efficiency EE is forcibly increased, thus solving the problem that energy efficiency optimization is masked by spectral efficiency in traditional algorithms and resolving the curse of dimensions.

[0061] Finally, a resource allocation strategy is obtained based on the synthesized Q-value vector. The synthesized Q-value vector provides a mapping relationship of "user -> resource block". That is, for each user in the network, the optimal resource block index is output (e.g., User1→RB15, User2→RB42). Then, the corresponding resource block is allocated according to the resource block index. This resource allocation strategy can allocate the optimal communication sub-channel for each user under the current system state (channel state, interference level) and achieve Pareto optimality of the whole network energy efficiency and spectrum efficiency under the constraint of minimum user rate.

[0062] Furthermore, embodiments of the present invention also provide an ultra-dense base station network after clustering using the aforementioned circular diffusion algorithm combined with the K-means algorithm. Figure 6 Simulation experiment, the system physical parameters are as follows: subcarrier bandwidth Total number of sub-channels 64, transmission power 1.0 W (30 dBm), circuit static power consumption 5.0 W, background noise power Minimum rate threshold .

[0063] Under the above physical parameters, a Double DQN mechanism was adopted, and the following key training parameters were used: maximum training episodes = 3000, learning rate = 0.0005, and discount factor. 0.9, Batch size = 64, Experience replay pool Buffer = 2000, Termination condition is reaching 3000 rounds, using a joint reward function. A dual-head competitive deep Q-network was trained, updating the EE head and SE head separately. The overall energy efficiency and spectral efficiency changes and loss curves during training are shown below. Figure 12 As shown in the figure; the trained dual-head competitive deep Q-network achieves the optimal balance between energy efficiency (EE) and spectral efficiency (SE) through joint optimization of energy efficiency and spectral efficiency. Simulation results are shown in the figure. Figure 13 , 14 The diagrams show the individual energy efficiency curves and spectral efficiency curves for each cluster of the system.

[0064] based on Figure 1 The resource allocation method for the ultra-dense base station network system shown herein adaptively hybridizes and clusters the ultra-dense base station network in the target area using the circular diffusion algorithm and the K-means algorithm. This ensures that the base stations within each cluster are relatively concentrated and that the number of base stations within each cluster is kept as equal as possible, thereby evenly distributing the number of base stations in each cluster. This effectively separates high-density clustered base stations, effectively balances the base station load, alleviates the problem of excessive subcarrier reuse, and reduces intra-cluster interference. Based on this, the energy efficiency and spectral efficiency of the clustered base station network are calculated and reward-shaped and normalized weighted. A joint reward function is then constructed and a dual-headed contention deep Q-network is trained to effectively achieve joint optimization of energy efficiency and spectral efficiency. Then, based on the current state data of the system, the trained dual-headed contention deep Q-network generates a Q-value vector, and the resource block index of the base station network is obtained based on the Q-value vector. Resource blocks are then allocated to users accordingly. This solves the problems of dimensional unfairness and Q-value overestimation existing in the current single network structure, making resource allocation more accurate and robust, thus achieving the best balance between energy efficiency and spectral efficiency.

[0065] When applying the resource allocation method for ultra-dense base station network systems provided in this manual, it is not necessary to consider... Figure 1 The steps shown are executed in sequence. The specific execution order of each step can be determined as needed, and this manual does not impose any restrictions on it.

[0066] The above describes one or more embodiments of a resource allocation method for an ultra-dense base station network system provided in this specification. Based on the same idea, this specification also provides a corresponding resource allocation system for an ultra-dense base station network system, such as... Figure 15 As shown.

[0067] Figure 15 This specification provides a schematic diagram of a resource allocation system for an ultra-dense base station network system, which includes: Module 1201 is established to build an ultra-dense base station network system for the target area based on resource block bandwidth, transmission power and circuit static power consumption. Clustering module 1202 is used to cluster the dense base stations of the ultra-dense base station network in the target area using the circular diffusion algorithm, and then use the K-means algorithm to cluster the remaining base stations. The calculation and training module 1203 is used to calculate the energy efficiency and spectral efficiency of the clustered ultra-dense base station network system, construct a joint reward function after reward shaping and normalization weighting of the energy efficiency and the spectral efficiency, and train a dual-head competitive deep Q network through the joint reward function. The generation and allocation module 1204 is used to obtain the current state data of the clustered ultra-dense base station network system, input the current state data into the trained dual-head competition deep Q network to generate Q-value vectors, obtain the resource allocation strategy of the target area ultra-dense base station network system according to the Q-value vectors, and allocate resource blocks to system users based on the resource allocation strategy.

[0068] Specific limitations regarding the resource allocation system for ultra-dense base station network systems can be found in the limitations on resource allocation methods for ultra-dense base station network systems described above, and will not be repeated here. Each module in the aforementioned resource allocation system for ultra-dense base station network systems can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0069] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 The provided method for allocating resources in an ultra-dense base station network system.

[0070] This instruction manual also provides Figure 16 The schematic diagram of the computer device shown is as follows: Figure 16 As shown, at the hardware level, this computer device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then executes it to achieve the above. Figure 1 The provided method for allocating resources in an ultra-dense base station network system.

[0071] The resource allocation method and system for ultra-dense base station networks provided in this specification adaptively hybridizes and clusters the ultra-dense base station network in the target area using the circular diffusion algorithm and the K-means algorithm. This ensures that the base stations within each cluster are relatively concentrated and that the number of base stations within each cluster is kept as equal as possible, thereby evenly distributing the number of base stations in each cluster. This effectively separates high-density clustered base stations, effectively balances the base station load, alleviates the problem of excessive subcarrier reuse, and reduces intra-cluster interference. Based on this, the energy efficiency and spectral efficiency of the clustered base station network are calculated, a joint reward function is constructed, and a dual-head contention deep Q-network is trained to effectively achieve joint optimization of energy efficiency and spectral efficiency. Then, based on the current state data of the system, the trained dual-head contention deep Q-network generates a Q-value vector, and the resource block index of the base station network is obtained according to the Q-value vector, and resource blocks are allocated to users accordingly. This solves the problems of dimensional unfairness and Q-value overestimation existing in the existing single network structure, making resource allocation more accurate and robust, thereby achieving the best balance between energy efficiency and spectral efficiency in the system.

[0072] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0073] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A resource allocation method for an ultra-dense base station network system, characterized in that, include: Based on resource block bandwidth, transmission power, and circuit static power consumption, an ultra-dense base station network system for the target area is established. After clustering the dense base stations of the ultra-dense base station network in the target area using the circular diffusion algorithm, the remaining base stations are then clustered using the K-means algorithm. Calculate the energy efficiency and spectral efficiency of the clustered ultra-dense base station network system, construct a joint reward function by performing reward shaping and normalization weighting on the energy efficiency and the spectral efficiency, and train a dual-head competitive deep Q network through the joint reward function; The current state data of the clustered ultra-dense base station network system is obtained, and the current state data is input into the trained dual-head competition deep Q network to generate Q-value vectors. The resource allocation strategy of the ultra-dense base station network system in the target area is obtained according to the Q-value vectors, and resource blocks are allocated to system users based on the resource allocation strategy.

2. The resource allocation method for an ultra-dense base station network system as described in claim 1, characterized in that, After clustering the dense base stations of the ultra-dense base station network in the target area using the circular diffusion algorithm, the remaining base stations are then clustered using the K-means algorithm, including: Traverse all base stations of the ultra-dense base station network in the target area and establish a base station density matrix; The base station density matrix is ​​sorted by density value and convolution is performed. An initial diffusion center is selected, and circular iterative diffusion is performed according to the diffusion speed and step size to group the base stations in the diffusion area into a cluster. The remaining base stations are clustered using the K-means algorithm.

3. The resource allocation method for an ultra-dense base station network system as described in claim 1, characterized in that, The dual-head competitive deep Q-network includes a network input layer, a shared feature extraction layer, and independent energy efficiency evaluation heads and spectral efficiency evaluation heads. The network input layer receives state data from the base station system. The shared feature extraction layer is a fully connected layer with 128 neurons and ReLU activation, used to extract general environmental state features from the state data. The dedicated heads for energy efficiency assessment and spectral efficiency assessment employ fully connected neural network layers to learn the system's action strategy from environmental state characteristics and evaluate the system's state value.

4. The resource allocation method for an ultra-dense base station network system as described in claim 3, characterized in that, Both the energy efficiency assessment head and the spectral efficiency assessment head include a state value flow branch and an action advantage flow branch. The state value flow branch and the action advantage flow branch are combined through a Dueling aggregation layer and output energy efficiency Q-value vector and spectral efficiency Q-value vector, respectively.

5. The resource allocation method for an ultra-dense base station network system as described in claim 4, characterized in that, The energy efficiency and spectral efficiency of the ultra-dense base station network system after clustering include: Based on resource block bandwidth Calculate user rate , Here, i represents the channel gain, and i represents the resource block. Based on the user rate Transmit power and circuit static power consumption Calculate energy efficiency , ; Spectral efficiency .

6. The resource allocation method for an ultra-dense base station network system as described in claim 5, characterized in that, A joint reward function is constructed by weighting and normalizing the energy efficiency and the spectral efficiency, specifically including: Regarding the energy efficiency and spectral efficiency Perform reward-based plastic surgery to obtain energy efficiency rewards. and spectrum efficiency rewards ,Right now: Penalty is a penalty item. This is the magnification factor; The energy efficiency reward and spectrum efficiency rewards Perform normalized weighting to construct a joint reward function. ,in, is the weighting factor, and Penalty is the penalty term.

7. The resource allocation method for an ultra-dense base station network system as described in claim 6, characterized in that, The process of acquiring current state data of the clustered ultra-dense base station network system, inputting the current state data into a trained dual-head contention deep Q-network to generate a Q-value vector, obtaining the resource allocation strategy of the target area ultra-dense base station network system based on the Q-value vector, and allocating resource blocks to system users based on the resource allocation strategy includes: The current state data of the clustered ultra-dense base station network system is obtained and logarithmically normalized to obtain a high-dimensional vector. This vector is then input into the trained dual-head competitive deep Q network to generate energy efficiency Q-value vectors and spectral efficiency Q-value vectors, which are then subjected to Min-Max normalization. The normalized energy efficiency Q-value vector and the spectral efficiency Q-value vector are weighted and summed to obtain the composite Q-value vector. Resource block indexes are obtained based on the synthesized Q-value vector, and resource blocks are allocated to users of the ultra-dense base station network system according to the resource block indexes.

8. A resource allocation system for an ultra-dense base station network system, characterized in that, include: A module is established to build an ultra-dense base station network system for a target area based on resource block bandwidth, transmission power, and circuit static power consumption. The clustering module uses the circular diffusion algorithm to cluster the dense base stations in the ultra-dense base station network of the target area, and then uses the K-means algorithm to cluster the remaining base stations. The computation and training module is used to calculate the energy efficiency and spectral efficiency of the clustered ultra-dense base station network system, construct a joint reward function by performing reward shaping and normalization weighting on the energy efficiency and the spectral efficiency, and train a dual-head competitive deep Q network through the joint reward function. The generation and allocation module is used to obtain the current state data of the clustered ultra-dense base station network system, input the current state data into the trained dual-head contention deep Q network to generate Q-value vectors, obtain the resource allocation strategy of the target area ultra-dense base station network system according to the Q-value vectors, and allocate resource blocks to system users based on the resource allocation strategy.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 7.

10. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any one of claims 1 to 7.

Images