Communication network spectrum dynamic adjustment optimization method based on reinforcement learning

Abstract

The invention discloses a communication network spectrum dynamic adjustment and optimization method based on reinforcement learning, and relates to the technical field of communication network intelligent optimization. According to the communication network spectrum dynamic adjustment optimization method based on reinforcement learning, the operation state data of multiple access nodes in multiple historical sampling periods are obtained, a reinforcement learning model is trained, reasoning is carried out based on the current communication network state, and the communication network spectrum dynamic adjustment optimization is realized. According to the method, the spectrum configuration strategy of the communication network is trained through the reinforcement learning model, the historical and current operation state data of the multiple access nodes are combined, the channel allocation, frequency band switching and interference coordination scheme is dynamically optimized, and the optimal spectrum configuration action including channel allocation, frequency band switching and interference coordination is output. And the spectrum use efficiency is improved. The method is especially suitable for an ultra-dense network scene, has good adaptivity and global cooperation capability, can significantly reduce interference overlapping and resource idling phenomena, and enhances communication quality and system stability.

Description

Technical Field

[0001] This invention relates to the field of intelligent optimization technology for communication networks, specifically a method for dynamic adjustment and optimization of communication network spectrum based on reinforcement learning. Background Technology

[0002] With the continuous evolution of 5G and future 6G communication systems, the number of access nodes in communication networks is growing exponentially. Especially with the dense access of smart terminals and large-scale IoT devices, spectrum resources face the challenge of coexistence of high scarcity and complex interference. Traditional spectrum allocation strategies are mostly based on fixed rules, static allocation, or centralized control methods, which make it difficult to maintain the optimal balance between spectrum utilization and communication quality in dynamic and multi-interference environments.

[0003] The limitations of existing technologies include at least the following problems: First, in ultra-dense communication network environments, the scheduling and allocation of spectrum resources still heavily rely on manual rules or static models, lacking effective dynamic optimization capabilities when facing multiple access nodes, complex interference environments, and dynamic service demands. Most existing solutions only consider the network state at a specific moment or perform local optimization on a single node, lacking overall modeling and analysis of historical communication performance across cycles and nodes. This results in low spectrum resource allocation efficiency, frequent system interference, and large fluctuations in communication quality, severely restricting the improvement of network capacity and user experience.

[0004] Secondly, traditional spectrum scheduling methods typically separate configuration actions from network performance, making it difficult to achieve a closed-loop learning mechanism that optimizes future strategies based on historical scheduling results. This results in limited model learning capabilities, weak generalization ability, and difficulty in adapting to complex changes in the wireless environment. Furthermore, spectrum allocation methods that partially rely on heuristic algorithms experience a sharp increase in computational complexity when the policy space is large and scheduling objectives are diverse, making it difficult to guarantee timeliness. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method for dynamic adjustment and optimization of communication network spectrum based on reinforcement learning. This method solves the problems of static and rigid spectrum configuration strategies, lack of historical scheduling effect feedback mechanisms, and difficulty in achieving multi-node collaborative optimization in ultra-dense network scenarios in existing communication networks.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for dynamic adjustment and optimization of communication network spectrum based on reinforcement learning, comprising the following steps: acquiring historical communication network data, wherein the historical communication network data includes historical operating status data of several access nodes for several historical sampling periods; training a preset reinforcement learning model based on the historical communication network data; and, after training, acquiring current communication network data, wherein the current communication network data includes current operating status data of several access nodes in the current sampling period; and inputting the current communication network data into the reinforcement learning model for optimization processing to obtain the optimal spectrum configuration actions of the communication network, including channel allocation, frequency band switching, and interference area resource coordination strategies.

[0007] Furthermore, the historical operating status data includes historical spectrum occupancy rate, historical communication success rate, historical average signal-to-noise ratio, and historical average interference intensity.

[0008] Further, based on the historical communication network data, the specific steps for training the preset reinforcement learning model are as follows: The historical operating status data of each access node in each historical sampling period is taken as a state input; the actual spectrum configuration action adopted by the access node in that historical sampling period is obtained as the action output, where the spectrum configuration action originates from the historical spectrum scheduling record corresponding to the node; based on the operating status data, the spectrum utilization efficiency index and interference sensitivity index of the access node in that sampling period are calculated respectively, and fused analysis is performed to obtain the corresponding reward value; the state input, action output, and reward value are combined to construct a training sample set; the training sample set is input into the reinforcement learning model for training to establish the optimal mapping relationship between the operating status data and the spectrum configuration action.

[0009] Furthermore, the specific steps for calculating the spectrum utilization efficiency index of a certain access node under a certain historical sampling period are as follows: read the historical communication success rate and historical average signal-to-noise ratio of a certain access node under a certain historical sampling period, and perform normalization processing; perform comprehensive analysis on the normalized historical communication success rate and historical average signal-to-noise ratio to obtain the spectrum utilization efficiency index of a certain access node under a certain historical sampling period.

[0010] Furthermore, the specific formula for calculating the spectrum utilization efficiency index of a certain access node under a certain historical sampling period is as follows: ;in, , , The parameters are, in order, the spectrum utilization efficiency index, historical communication success rate, and historical average signal-to-noise ratio of a certain access node in a certain historical sampling period. This is a communication success rate adjustment factor stored in the database.

[0011] Furthermore, the specific steps for calculating the interference sensitivity index of a certain access node under a certain historical sampling period are as follows: read the historical spectrum occupancy rate and historical average interference intensity of a certain access node under a certain historical sampling period, and perform normalization processing; perform comprehensive analysis on the normalized historical spectrum occupancy rate and historical average interference intensity to obtain the interference sensitivity index of a certain access node under a certain historical sampling period.

[0012] Furthermore, the specific formula for calculating the interference sensitivity index of a certain access node under a certain historical sampling period is as follows: ;in, , , The parameters are, in order, the interference sensitivity index, historical spectrum occupancy rate, and historical average interference intensity of a certain access node under a certain historical sampling period. It is a natural constant. These are the interference enhancement factors stored in the database.

[0013] Furthermore, the specific steps to obtain the reward value of a certain access node under a certain historical sampling period are as follows: read the spectrum utilization efficiency index and interference sensitivity index of a certain access node under a certain historical sampling period; input the spectrum utilization efficiency index and interference sensitivity index into the preset reward fusion analysis model to obtain the reward value of a certain access node under a certain historical sampling period.

[0014] Furthermore, the pre-defined reward fusion analysis model is as follows: ;in, , , The values, in order, are the reward value, spectrum utilization efficiency index, and interference sensitivity index of a certain access node in a certain historical sampling period. , The following are the spectral efficiency adjustment coefficients and interference sensitivity adjustment coefficients stored in the database, respectively.

[0015] Furthermore, the current operating status data includes the current spectrum occupancy rate, current communication success rate, current average signal-to-noise ratio, and current average interference intensity.

[0016] The present invention has the following beneficial effects:

[0017] (1) The reinforcement learning-based method for dynamic adjustment and optimization of spectrum in communication networks breaks through the shortcomings of static and rigid spectrum configuration in traditional communication networks by introducing a reinforcement learning model. It can dynamically adjust the allocation of spectrum resources based on the historical and current status data of access nodes. In actual deployment, this method can effectively identify the changing trend of spectrum utilization of each node and deduce the optimal spectrum configuration action accordingly, including channel allocation, frequency band switching and interference area resource coordination. In this way, while ensuring communication quality, it can maximize spectrum utilization efficiency and reduce resource idleness or overlapping interference. Especially in ultra-dense network environments with dense deployment of multiple access nodes, this method has good adaptability and self-optimization ability, which can significantly improve the utilization of spectrum resources and reduce the overall load pressure and energy consumption of the network. It has high practical engineering promotion value.

[0018] (2) The reinforcement learning-based method for dynamic adjustment and optimization of communication network spectrum comprehensively introduces multi-dimensional state parameters such as communication success rate, signal-to-noise ratio, spectrum occupancy rate, and interference intensity during the reinforcement learning training process. By constructing the spectrum utilization efficiency index and the interference sensitivity index, the state input not only has the ability to express the quality of the communication link, but also takes into account the feedback characteristics of the network interference environment. At the same time, the design of the reward function is no longer limited to the traditional single throughput target, but establishes a reward model in the way of dual index fusion, which makes the model training more realistic and directional. The above mechanism makes the training process closer to the actual network operation, and also makes the final output of spectrum scheduling action have a quantifiable and traceable basis, thereby enhancing the interpretability and fine control capability of the entire scheduling system and solving the problem of insufficient transparency of the traditional black box scheduling model in actual engineering deployment.

[0019] (3) The reinforcement learning-based communication network spectrum dynamic adjustment optimization method can take into account the spectrum coordination relationship between multiple access nodes. Through full-cycle modeling and training of historical scheduling data, it learns the scheduling strategy performance of each node under different interference environments and resource occupation, thereby effectively avoiding the system performance bottleneck caused by local optima. In the current data input stage, this invention supports the parallel collection of state data by each access node for joint reasoning, so that the optimization results have strong global coordination and strategy stability. In the environment of complex interference and limited resources, it can accurately identify interference sources and coordinate and adjust resource configuration schemes to ensure that the overall network performance is not dragged down by individual nodes, significantly improve the overall robustness of the system, and have good adaptability to 5G ultra-dense deployment applications.

[0020] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0021] Figure 1 This is a flowchart of the reinforcement learning-based method for dynamic adjustment and optimization of communication network spectrum according to the present invention.

[0022] Figure 2 This is a flowchart illustrating the specific steps involved in calculating the spectrum utilization efficiency index of a certain access node under a certain historical sampling period in the reinforcement learning-based dynamic adjustment and optimization method for communication network spectrum of this invention.

[0023] Figure 3 This is a flowchart illustrating the specific steps involved in obtaining the reward value of a certain access node under a certain historical sampling period in the reinforcement learning-based communication network spectrum dynamic adjustment and optimization method of the present invention. Detailed Implementation

[0024] Please see Figure 1 This invention provides a technical solution: a method for dynamic adjustment and optimization of communication network spectrum based on reinforcement learning, comprising the following steps: acquiring historical communication network data, which includes historical operating status data of several access nodes for several historical sampling periods; training a preset reinforcement learning model based on the historical communication network data; and after training, acquiring current communication network data, which includes current operating status data of several access nodes in the current sampling period; and inputting the current communication network data into the reinforcement learning model for optimization processing to obtain the optimal spectrum configuration action of the communication network, including channel allocation, frequency band switching, and interference area resource coordination strategy.

[0025] Spectrum configuration actions include two categories: single-node actions and multi-node joint actions, corresponding to single resource scheduling and coordinated interference management strategies, respectively. Specifically, a single-node action can be represented as a triple:

[0026] An = (fn, cn, sn);

[0027] Where fn represents the frequency band number currently recommended for switching, cn is the channel number, and sn is the interference coordination strategy type (such as frequency avoidance, time slot backoff, or power suppression). Each action output by the reinforcement learning model is a combination configuration of one or more nodes, the action space is a discrete space, and the set of elements expands dynamically according to the device capabilities.

[0028] In actual deployment, historical communication network data is periodically reported to the network control center by access nodes (such as micro base stations and cell access points) in the communication system. The sampling period can be set according to the load status and policy scheduling mechanism of the communication system, generally ranging from 100ms to 10s. Each access node records its own spectrum configuration decisions (such as the use of channel number, frequency band number, and interference management policy) in each period and uploads them synchronously. Its behavior records constitute the historical scheduling data of spectrum configuration actions.

[0029] Current operational status data includes current spectrum occupancy, current communication success rate, current average signal-to-noise ratio, and current average interference intensity.

[0030] During the model deployment phase, the communication system periodically collects communication status data within the current sampling period, including the current spectrum occupancy rate, communication success rate, signal-to-noise ratio, and interference intensity of each access node. After standardized preprocessing, this data is input into the trained reinforcement learning model. Based on the current state input, the model infers the optimal set of spectrum configuration actions, where each action corresponds to a spectrum decision for an access node, including specifying a channel number, suggesting frequency band switching, and interference avoidance strategies (such as power suppression, time avoidance, and spectrum hopping). The inference results are uploaded to the network control center, and the control strategy unit issues instructions to the access nodes for execution.

[0031] During the inference phase, the reinforcement learning model takes the current state set of all access nodes in the communication network as input and outputs a set of spectrum configuration actions corresponding to the number of nodes, with each dimension corresponding to the optimal spectrum configuration suggestion for an access node. Therefore, the policy output in each round is a vectorized set of spectrum configuration policies, suitable for network-wide synchronous scheduling and coordination control.

[0032] Historical operational status data includes historical spectrum occupancy, historical communication success rate, historical average signal-to-noise ratio, and historical average interference intensity.

[0033] Specifically, the steps for training the pre-defined reinforcement learning model based on historical communication network data are as follows: The historical operational status data of each access node in each historical sampling period is taken as a state input; the actual spectrum configuration action taken by the access node in that historical sampling period is obtained as the action output, with the spectrum configuration action derived from the node's corresponding historical spectrum scheduling record; based on the operational status data, the spectrum utilization efficiency index and interference sensitivity index of the access node in that sampling period are calculated and fused to obtain the corresponding reward value; the state input, action output, and reward value are combined to construct a training sample set; the training sample set is input into the reinforcement learning model for training to establish the optimal mapping relationship between operational status data and spectrum configuration actions.

[0034] The reinforcement learning model can be a policy optimization model based on DQN (Deep Q-Network) or DDPG (Deep Deterministic Policy Gradient), which includes a state input layer, a hidden layer, and an action output layer. The hidden layer can be a 2-4 layer fully connected structure, and the activation function is ReLU or tanh. During training, an empirical replay mechanism and a fixed Q-objective policy are used to improve stability and convergence speed.

[0035] For each access node in each historical sampling period, a state vector is constructed based on its corresponding normalized state data. The actual spectrum configuration action executed by the node in that period (which can be found in its spectrum scheduling records) is used as the action vector. Then, the spectrum utilization efficiency index and interference sensitivity index are calculated based on its state data, and the reward value corresponding to that state-action is calculated using a pre-defined reward fusion analysis model. These three elements together form a training sample triple (s, a, r), which is used for policy iteration training of the reinforcement learning model. During data preprocessing, to avoid interference from historical extreme communication conditions on the training results, those with historical interference intensity or signal-to-noise ratio exceeding a set threshold (e.g., Z-Score > 3) are discarded or smoothed.

[0036] In this implementation scheme, a meticulously designed state-action-reward triplet sample construction method is used to achieve deep modeling and behavioral backtracking of complex communication network operation states during reinforcement learning model training. Specifically, the operational state data of each access node in each historical sampling period is extracted as state input and combined with the actual spectrum scheduling records as action output. At the same time, the corresponding spectrum utilization efficiency index and interference sensitivity index are calculated, and the reward value is obtained through the fusion model, thus forming training samples with business real-world significance. This training sample set not only has the ability to truly reflect actual strategies, but also enhances the robustness of the model by removing extreme interference data and avoiding overfitting to historical sporadic situations. In addition, by introducing experience replay and fixed target network mechanisms, the stability and convergence efficiency of the reinforcement learning model are further improved. Compared with traditional single-index driven or simulated label driven learning mechanisms, the model trained by this method has stronger policy transferability and generalization ability, and is suitable for spectrum scheduling decisions under different network densities and interference intensities.

[0037] Specifically, such as Figure 2 As shown, the specific steps for calculating the spectrum utilization efficiency index of an access node under a certain historical sampling period are as follows: read the historical communication success rate and historical average signal-to-noise ratio of an access node under a certain historical sampling period, and perform normalization processing; perform comprehensive analysis on the normalized historical communication success rate and historical average signal-to-noise ratio to obtain the spectrum utilization efficiency index of an access node under a certain historical sampling period.

[0038] The normalization process employs the range normalization (Min-Max Scaling) method.

[0039] Xnorm=(X−Xmin) / (Xmax−Xmin);

[0040] Here, Xmin and Xmax represent the minimum and maximum values ​​of the feature in the training set, respectively. To improve the robustness of the system, upper and lower pruning thresholds can be set to prevent outliers from affecting the normalization results.

[0041] The specific formula for calculating the spectrum utilization efficiency index of a certain access node in a certain historical sampling period is as follows: ;in, , , The following are, in order: the spectrum utilization efficiency index of a certain access node in a certain historical sampling period, the historical communication success rate (after normalization), and the historical average signal-to-noise ratio (after normalization). The communication success rate adjustment factor is stored in the database, and in this embodiment, the value is 1.2.

[0042] The communication success rate adjustment factor has a range of values: [0.2, 2.0].

[0043] This implementation scheme, based on two core indicators—communication success rate and average signal-to-noise ratio (SNR)—introduces a range normalization mechanism and an adjustment factor to construct a calculation model. This model can more reasonably reflect the spectrum utilization effect of access nodes under historical sampling periods. Compared with traditional methods that directly use raw indicators or empirical scoring methods, this method first normalizes the communication success rate and SNR to ensure that the indicators under different access nodes or different sampling periods have a uniform scale, significantly improving the accuracy of cross-node horizontal comparisons. Furthermore, the introduction of an adjustment factor can dynamically adjust the weight of the communication success rate in the comprehensive evaluation according to the needs of the scenario, enhancing the model's adaptability to the QoS requirements of specific services. In addition, to prevent interference from extreme values ​​during the normalization process, an upper and lower limit pruning mechanism is set to effectively avoid the model being biased towards historical abnormal samples. Overall, this spectrum utilization efficiency index has the triple advantages of computational simplicity, indicator sensitivity, and system stability, laying a high-quality foundation for subsequent reward modeling and policy learning.

[0044] Specifically, the steps for calculating the interference sensitivity index of an access node under a certain historical sampling period are as follows: read the historical spectrum occupancy rate and historical average interference intensity of an access node under a certain historical sampling period, and perform normalization processing; perform comprehensive analysis on the normalized historical spectrum occupancy rate and historical average interference intensity to obtain the interference sensitivity index of an access node under a certain historical sampling period.

[0045] The specific formula for calculating the interference sensitivity index of a certain access node under a certain historical sampling period is as follows: ;in, , , The following are, in order: interference sensitivity index of a certain access node in a certain historical sampling period, historical spectrum occupancy (normalized), and historical average interference intensity (normalized). This is a natural constant, and in this embodiment, it is taken as 2.71. The interference enhancement factor is stored in the database, and in this embodiment, it is set to 0.4.

[0046] The value range of the interference enhancement factor is [0.1, 1.0].

[0047] This implementation plan addresses the resource coordination challenges in ultra-dense network (UDN) scenarios, such as high spectrum occupancy and strong adjacent-channel interference. It constructs a sensitivity evaluation model that integrates spectrum occupancy and interference intensity. First, a range normalization method is used to unify the index scale, ensuring comparability between different sampling periods and access nodes, thus avoiding heteroscedastic interference from the original index. Second, an exponential function structure with a natural constant base is introduced, making the index more sensitive to high-interference scenarios and amplifying the interference response intensity in critical locations such as edge areas and heavily loaded areas. Simultaneously, by introducing an adjustable interference enhancement factor, users or the system can flexibly adjust the influence weight of the interference factor according to different service scenarios, thereby improving the model's adaptability to differentiated scenarios. Furthermore, the index's calculation logic is simple, and its source is clear, facilitating rapid integration into existing networks. This enables highly efficient and sensitive interference identification and dynamic response, significantly promoting subsequent reward modeling and reinforcement learning strategy convergence.

[0048] Specifically, such as Figure 3 As shown, the specific steps to obtain the reward value of a certain access node under a certain historical sampling period are as follows: read the spectrum utilization efficiency index and interference sensitivity index of a certain access node under a certain historical sampling period; input the spectrum utilization efficiency index and interference sensitivity index into the preset reward fusion analysis model to obtain the reward value of a certain access node under a certain historical sampling period.

[0049] The spectrum utilization efficiency index essentially measures the effective communication quality carried by a unit of spectrum resources; a higher value indicates more efficient utilization of the frequency band. The interference sensitivity index reflects the node's sensitivity to spectrum interference in the current environment; a higher value indicates severe spectrum congestion or interference, requiring avoidance as much as possible. These two constitute the core variables of the reward function; the former provides positive incentives, while the latter incurs negative penalties, guiding the reinforcement learning strategy to converge to an efficient, low-interference spectrum scheduling strategy.

[0050] The pre-defined reward fusion analysis model is as follows: ;in, , , The values, in order, are the reward value, spectrum utilization efficiency index, and interference sensitivity index of a certain access node in a certain historical sampling period. , The values ​​are, in order, the spectral efficiency adjustment coefficient and the interference sensitivity adjustment coefficient stored in the database, which are 1.0 and 1.5 in this embodiment.

[0051] Where tanh is the hyperbolic tangent function.

[0052] The range of the spectral efficiency adjustment coefficient is [0.5, 2.0].

[0053] The interference sensitivity adjustment coefficient has a range of values: [0.5, 2.0].

[0054] In multi-access node collaborative spectrum scheduling scenarios, to enhance the global coordination capability of reinforcement learning strategies, the system can optionally introduce an adjacency matrix between access nodes, constructing spatial correlations between nodes based on the network topology graph. For example, if the service areas of two nodes overlap, the system designates them as adjacent nodes and assigns them higher scheduling weights in the interference coordination strategy. This topology-aware mechanism is particularly suitable for high-density deployed ultra-dense wireless network (UDN) scenarios, improving edge communication quality and spectrum utilization efficiency.

[0055] In this implementation scheme, a dual-indicator driving mechanism of spectrum utilization efficiency index and interference sensitivity index is adopted in the reward function design. This effectively solves the learning path drift problem caused by target ambiguity or reward bias in traditional policy learning. The spectrum utilization efficiency index is used to positively incentivize efficient communication behavior, while the interference sensitivity index suppresses resource selection due to interference surges through a negative penalty mechanism. Together, they construct a reinforcement learning policy optimization orientation that seeks to maximize benefits and minimize harms. This can effectively guide the learning model to converge toward a high-performance, low-conflict spectrum allocation space. In addition, adjustable weight coefficients are introduced into the reward fusion function, allowing for flexible adjustment of the policy orientation according to different network deployment scenarios, improving adaptability and transferability. Furthermore, the system supports topology modeling between access nodes and constructs a spatial correlation graph based on the adjacency matrix. This enables the learning policy to consider the performance of a single node while also possessing multi-node interference perception and collaborative scheduling capabilities. It is particularly suitable for complex network structures such as ultra-dense cells and cellular edges, significantly improving the overall utilization efficiency of spectrum resources and system throughput performance.

[0056] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0057] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for optimizing dynamic spectrum adjustment of a communication network based on reinforcement learning, characterized in that, The method comprises the following steps: acquiring historical communication network data, the historical communication network data comprising historical running state data of a plurality of access nodes in a plurality of historical sampling periods; training a preset reinforcement learning model based on the historical communication network data; after the training is completed, acquiring current communication network data, the current communication network data comprising current running state data of the plurality of access nodes in a current sampling period; inputting the current communication network data into the reinforcement learning model for optimization processing to obtain an optimal spectrum configuration action of the communication network, the optimal spectrum configuration action comprising a channel allocation, a frequency band switching and an interference region resource coordination strategy.

2. The method of claim 1, wherein, The historical running state data comprises a historical spectrum occupancy rate, a historical communication success rate, a historical average signal-to-noise ratio and a historical average interference intensity. 3.The method of claim 1, wherein, The specific steps of training the preset reinforcement learning model based on the historical communication network data are as follows: inputting the historical running state data of each access node in each historical sampling period in the historical communication network data as a state input respectively; acquiring a spectrum configuration action actually adopted by the access node in the historical sampling period as an action output, the spectrum configuration action being derived from a historical spectrum scheduling record corresponding to the node; based on the running state data, calculating a spectrum utilization efficiency index and an interference sensitivity index of the access node in the sampling period respectively, and performing fusion analysis to obtain a corresponding reward value; combining the state input, the action output and the reward value to construct a training sample set; inputting the training sample set into the reinforcement learning model for training to establish an optimal mapping relationship between the running state data and the spectrum configuration action.

4. The method of claim 3, wherein, The specific steps of calculating the spectrum utilization efficiency index of a certain access node in a certain historical sampling period are as follows: reading the historical communication success rate and the historical average signal-to-noise ratio of the certain access node in the certain historical sampling period, and performing normalization processing; performing comprehensive analysis on the normalized historical communication success rate and the historical average signal-to-noise ratio to obtain the spectrum utilization efficiency index of the certain access node in the certain historical sampling period.

5. The method of claim 4, wherein, The specific formula of calculating the spectrum utilization efficiency index of a certain access node in a certain historical sampling period is as follows: ； wherein, , , are, in sequence, a spectral usage efficiency index, a historical communication success rate, a historical average signal-to-noise ratio of a certain access node at a certain historical sampling period, is a communication success rate adjustment factor stored in the database.

6. The method of claim 3, wherein the method further comprises: The specific steps of calculating the interference sensitivity index of a certain access node in a certain historical sampling period are as follows: reading the historical spectrum occupancy rate and the historical average interference intensity of the certain access node in the certain historical sampling period, and performing normalization processing; performing comprehensive analysis on the normalized historical spectrum occupancy rate and the historical average interference intensity to obtain the interference sensitivity index of the certain access node in the certain historical sampling period.

7. The method of claim 6, wherein the method further comprises: The specific formula of calculating the interference sensitivity index of a certain access node in a certain historical sampling period is as follows: ； wherein, , , are, in order, the interference sensitivity index, the historical spectrum occupancy, the historical average interference strength of a certain access node at a certain historical sampling period, is a natural constant, is the interference enhancement factor stored in the database. 8.The method of claim 3, wherein, The specific steps of obtaining the reward value of a certain access node in a certain historical sampling period are as follows: reading the spectrum utilization efficiency index and the interference sensitivity index of the certain access node in the certain historical sampling period; inputting the spectrum utilization efficiency index and the interference sensitivity index into a preset reward fusion analysis model to obtain the reward value of the certain access node in the certain historical sampling period. 9.The method of claim 8, wherein, The preset reward fusion analysis model is specifically as follows: ； wherein, , , are, in sequence, a reward value, a spectrum usage efficiency index, an interference sensitivity index of a certain access node at a certain historical sampling period, , are, in sequence, a spectrum efficiency adjustment coefficient, an interference sensitivity adjustment coefficient stored in the database. 10.The method of claim 1, wherein, The current running state data includes current spectrum occupancy, current communication success rate, current average signal-to-noise ratio and current average interference intensity.

Images