Power control method and device of communication power supply, computer readable storage medium and communication power supply

Abstract

A power control method and device of a communication power supply, a computer readable storage medium and the communication power supply, the method comprising: acquiring a multi-dimensional state parameter related to a current communication link; inputting the multi-dimensional state parameter into a pre-trained reinforcement learning model, outputting a discrete power gain adjustment action through the reinforcement learning model; and adjusting the current transmission power according to the power gain adjustment action to obtain and execute a final transmission power; wherein the transmission power comprises radio frequency power of a signal transmitted by the communication power supply. The application realizes adaptive intelligent control of the transmission power of the communication power supply, effectively improves energy efficiency while ensuring communication reliability.

Description

Technical Field

[0001] This application relates to the field of communication technology, specifically to a power control method, apparatus, computer-readable storage medium, and communication power supply for a communication power supply. Background Technology

[0002] Among various communication power supplies, controlling the transmit power is one of the key factors affecting communication reliability and energy efficiency. Whether it is a base station, a mobile terminal, an IoT node, or a communication power module, there is a need to dynamically adjust the transmit power according to changes in the communication environment in order to minimize power consumption while ensuring communication quality.

[0003] Currently, power control in communication power supplies is mainly achieved through preset rules. Common technical solutions include: fixed power schemes that transmit signals at constant power, distance adjustment schemes that use simple linear mapping based on communication distance, and multi-parameter adjustment schemes that incorporate some channel parameters for manual rule optimization. These schemes all rely on static preset logic and lack the ability to dynamically perceive and adaptively adjust to complex communication environments.

[0004] As communication networks continue to expand and application scenarios become increasingly complex, the limitations of traditional power control methods are becoming more and more apparent. On the one hand, communication link conditions (such as channel quality and interference levels) are significantly time-varying and uncertain, making it difficult for preset rules to accurately adapt to various scenarios. On the other hand, there is an inherent contradiction between communication reliability and equipment energy efficiency. Traditional solutions cannot effectively optimize energy consumption while ensuring communication quality, resulting in communication power supplies operating in a suboptimal state for extended periods, leading to energy waste.

[0005] Therefore, how to achieve adaptive intelligent control of the transmission power of communication power supplies to significantly improve energy efficiency while ensuring communication reliability has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] This application provides a power control method, apparatus, computer-readable storage medium, and communication power supply for communication power supplies, which can solve the problem in the prior art that it is impossible to effectively optimize energy consumption while ensuring communication quality, resulting in energy waste.

[0007] In a first aspect, embodiments of this application provide a power control method for a communication power supply, the power control method for the communication power supply comprising: Obtain multi-dimensional state parameters related to the current communication link; wherein, the multi-dimensional state parameters include channel state parameters, communication service requirement parameters, and historical performance parameters; The multidimensional state parameters are input into a pre-trained reinforcement learning model, and the reinforcement learning model outputs discrete power gain adjustment actions; wherein, the reinforcement learning model is trained through a multi-objective reward function, which includes at least a first reward term for evaluating whether communication is successful and a second reward term for evaluating the energy efficiency of transmission power; According to the power gain adjustment action, the current transmit power is adjusted to obtain and execute the final transmit power; wherein, the transmit power includes the radio frequency power of the communication power supply transmit signal.

[0008] The aforementioned method achieves comprehensive perception of the communication link status by acquiring multi-dimensional state parameters including channel state, communication service requirements, and historical performance. It trains a reinforcement learning model using a multi-objective reward function that includes communication success rewards and energy efficiency rewards, enabling the model to proactively optimize transmit power efficiency while ensuring communication reliability. Furthermore, it dynamically adjusts the current transmit power by outputting discrete power gain adjustment actions, achieving adaptive control of the transmit power. This application solves the technical problems of traditional power control methods that rely on preset rules, cannot dynamically adapt to channel changes, and struggle to balance communication reliability and energy efficiency. It enables intelligent adaptive power adjustment in complex and ever-changing communication environments, effectively reducing transmit power and improving equipment energy efficiency while ensuring communication quality.

[0009] In conjunction with the first aspect, in one implementation, the step of adjusting the gain of the current transmit power according to the power gain adjustment action to obtain and execute the final transmit power includes: The base transmit power is determined based on the multidimensional state parameters of the current communication link. The initial transmission power is obtained by superimposing the base transmission power with the gain value corresponding to the power gain adjustment action; The initial transmit power is limited to obtain the final transmit power.

[0010] In conjunction with the first aspect, in one implementation, the reinforcement learning model includes a deep Q-network model, and the method for updating the deep Q-network model includes: Build an experience replay pool to store historical interaction data. Each piece of historical interaction data should include at least the current state, the action performed, the reward obtained, and the next state. Several historical interaction data points are randomly or sampled from the experience replay pool according to priority, and the network parameters of the deep Q network model are iteratively updated.

[0011] In conjunction with the first aspect, in one implementation, the method for updating the deep Q-network model further includes: Construct a target network with the same structure as the current network, but whose parameter updates lag behind the current network; The target Q-value is calculated using the target network, and a loss function is calculated based on the error between the predicted Q-value of the current network and the target Q-value. The parameters of the current network are adjusted with the goal of minimizing the loss function.

[0012] In conjunction with the first aspect, in one embodiment, the multi-objective reward function further includes a third reward item for evaluating data transmission efficiency; the first reward item, the second reward item, and the third reward item are weighted and combined according to preset weights to form the multi-objective reward function.

[0013] In conjunction with the first aspect, in one implementation, the first reward item takes a positive value when communication is successful and a negative value when communication fails; the second reward item is negatively correlated with the current transmission power; and the third reward item is positively correlated with the throughput of the current communication link.

[0014] In conjunction with the first aspect, in one implementation, the discrete power gain adjustment action output by the reinforcement learning model includes: The reinforcement learning model is used to evaluate the value of each candidate power gain adjustment action in the preset discrete action space to obtain the evaluation result. Based on the evaluation results and the preset selection strategy, a candidate power gain adjustment action is selected as the power gain adjustment action to be executed. The preset selection strategy includes an ε-greedy strategy, which includes first randomly selecting a candidate power gain adjustment action with a first probability, and then selecting the candidate power gain adjustment action with the largest Q value in the current state with a second probability, wherein the first probability is less than the second probability.

[0015] Secondly, embodiments of this application provide a power control device for a communication power supply, the power control device for the communication power supply comprising: The acquisition module is used to acquire multi-dimensional state parameters related to the current communication link; wherein, the multi-dimensional state parameters include channel state parameters, communication service requirement parameters, and historical performance parameters; An adjustment module is used to input the multidimensional state parameters into a pre-trained reinforcement learning model and output discrete power gain adjustment actions through the reinforcement learning model; wherein, the reinforcement learning model is trained by maximizing a multi-objective reward function, the multi-objective reward function includes at least a first reward term for evaluating whether communication is successful and a second reward term for evaluating the energy efficiency of transmission power; An execution module is used to adjust the current transmit power gain according to the power gain adjustment action to obtain and execute the final transmit power; wherein the transmit power includes the radio frequency power of the communication power supply transmit signal.

[0016] Thirdly, embodiments of this application provide a communication power supply, which includes a processor, a memory, and a power control program stored in the memory and executable by the processor, wherein when the power control program is executed by the processor, it implements the steps of the power control method of the communication power supply described above.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing a power control program, wherein when the power control program is executed by a processor, it implements the steps of the power control method for the communication power supply described above.

[0018] The beneficial effects of the technical solutions provided in this application include: This invention achieves comprehensive perception of the communication link status by acquiring multi-dimensional state parameters including channel state, communication service requirements, and historical performance. It trains a reinforcement learning model using a multi-objective reward function that includes at least communication success rewards and energy efficiency rewards, enabling the model to proactively optimize transmit power efficiency while ensuring communication reliability. It dynamically adjusts the current transmit power by outputting discrete power gain adjustment actions, achieving adaptive power control. Furthermore, by determining the base transmit power based on the communication distance and performing amplitude limiting, the accuracy of power adjustment and hardware adaptability are improved. Iterative updates to the deep Q-network model by constructing an experience replay pool and a target network enhance the stability and convergence efficiency of model training. By introducing a third reward term for evaluating data transmission efficiency and weighting it with the first two terms, multi-dimensional collaborative optimization of communication success rate, energy efficiency, and data transmission efficiency is further achieved. The use of an ε-greedy strategy to balance exploration and utilization enhances the model's adaptability to dynamic communication environments. This invention effectively solves the technical problems of traditional power control methods, such as reliance on preset rules, inability to dynamically adapt to channel changes, and difficulty in balancing communication reliability and energy efficiency, achieving intelligent adaptive control of communication power supply transmit power. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating an embodiment of the power control method for a communication power supply according to this application. Figure 2 This is a functional module schematic diagram of an embodiment of the power control device for a communication power supply according to this application; Figure 3 This is a schematic diagram of the hardware structure of the communication power supply involved in the embodiments of this application. Detailed Implementation

[0020] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0021] First, some of the technical terms used in this application will be explained to help those skilled in the art understand this application.

[0022] In a first aspect, embodiments of this application provide a power control method for a communication power supply. It should be noted that the execution subject of the method described in this invention can be a communication device. The communication device may include a power supply for communication, and may be a mobile terminal, base station, IoT terminal, data center equipment, etc. The method can be executed by a processor, microcontroller, digital signal processor, or dedicated power control chip built into the communication device.

[0023] In one embodiment, reference is made to Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the power control method for a communication power supply according to this application. Figure 1 As shown, the power control method for communication power supplies includes: Step S10: Obtain multi-dimensional state parameters related to the current communication link; In some embodiments, the multidimensional state parameters may include channel state parameters, communication service requirement parameters, and historical performance parameters; the current communication link may refer to the wireless communication path for signal transmission between the communication power supply and other communication devices (such as base stations and routers), and its quality is affected by various factors such as channel environment, service requirements, and historical operating status. To accurately perceive the current link status, this method collects multidimensional state parameters related to the current communication link in real time through a state perception module.

[0024] The state awareness module is deployed inside the communication equipment and is tightly integrated with the equipment's hardware resources. Specifically, the state awareness module may include a channel state acquisition unit, a communication demand acquisition unit, and a historical information statistics unit. The channel status acquisition unit can measure and acquire channel status parameters in real time through sensors and signal detection circuits built into the communication module. The communication module, which can be deployed inside the communication device, is the basic hardware unit for the device to transmit and receive wireless signals. For example, the acquired channel status parameters may include: a signal-to-noise ratio (SNR) of 25 dB, a path loss of 25 dB, a noise power of 50 (normalized value), and an interference level of 0.1 (0 indicates no interference, 1 indicates strong interference). These parameters reflect the current quality status of the wireless channel.

[0025] The communication requirements acquisition unit can obtain communication service requirement parameters from the communication protocol stack or application layer. For example, the obtained communication service requirement parameters include: communication distance of 50 m, QoS requirement of 0.8 (1 being the highest level), and data priority of 0.7 (1 being high priority). These parameters reflect the service level requirements of the current communication task.

[0026] The historical information statistics unit can statistically analyze historical communication performance based on a sliding window (e.g., a window size of 10 communication cycles). For example, the obtained historical performance parameters include: a recent communication success rate of 0.96, a recent retransmission count of 1 (normalized to 0.1), and a recent energy consumption of 0.2 (normalized value). These parameters reflect the recent operating status of the device.

[0027] The aforementioned 10-dimensional state parameters, after being standardized (e.g., the normalization process described above), constitute the multi-dimensional state vector at the current moment, and are input into the subsequent reinforcement learning decision module to generate the optimal power control strategy.

[0028] Step S20: Input the multidimensional state parameters into the pre-trained reinforcement learning model, and output discrete power gain adjustment actions through the reinforcement learning model; In some embodiments, the reinforcement learning model is pre-trained using a multi-objective reward function, which includes at least a first reward term for evaluating communication success and a second reward term for evaluating transmit power efficiency. This embodiment uses a Deep Q-Network (DQN) as the reinforcement learning model. Its network structure is as follows: the input layer contains 10 neurons, corresponding to a 10-dimensional state vector; the hidden layers consist of 4 layers with 64, 128, 64, and 32 neurons respectively, all using the ReLU activation function, and a dropout layer (with a dropout rate of 0.2) is set after the first two hidden layers to prevent overfitting; the output layer contains 7 neurons, using a linear activation function, corresponding to the Q-values ​​of 7 discrete power gain actions. This model has been pre-trained using historical running data and can accurately evaluate the long-term benefits of each action based on the current state.

[0029] The 10-dimensional state vector collected and standardized in step S10 is input into the reinforcement learning model, with the following specific values: After receiving the aforementioned state vector, the reinforcement learning model performs forward propagation calculations to evaluate the value of each candidate power gain adjustment action in a preset discrete action space, and outputs the Q value corresponding to each action. The discrete action space may contain 7 power gain levels: 0 dB, 1 dB, 2 dB, 3 dB, 4 dB, 5 dB, and 6 dB.

[0030] In this embodiment, the Q-value evaluation result output by the model is as follows: The Q value above indicates that, under the current conditions, choosing a 1 dB gain (action a2) yields the highest expected cumulative reward, meaning the model predicts that this action can achieve a good balance between ensuring communication success and energy efficiency. However, choosing a gain level that is too high or too low may lead to a decrease in long-term returns due to increased risk of communication failure or excessive energy consumption.

[0031] This embodiment employs an ε-greedy strategy for action selection, with a preset exploration probability ε = 0.1. At each decision point, the system generates a random number p between 0 and 1 and compares it with ε: If p < ε, then enter exploration mode and randomly select one action from the 7 candidate actions; If p ≥ ε, then enter the exploitation mode and select the action with the largest Q value in the current state.

[0032] In this embodiment, assuming the generated random number p=0.06, which is less than ε=0.1, the system enters exploration mode and randomly selects an action from 7 candidate actions. Assuming the random selection result is action a3 (gain 2 dB), the power gain adjustment action to be executed is 2 dB.

[0033] At another decision point, if the generated random number p=0.15 is greater than ε=0.1, then the utilization mode is entered, and the action a2 (gain 1 dB) with the largest Q value is selected as the power gain adjustment action to be executed.

[0034] The reinforcement learning decision module ultimately outputs a discrete power gain adjustment action, which is a specific gain level selected from a preset action space. This action will be passed to the power execution module (e.g., the main control chip for a communication power supply) for subsequent final transmit power calculation.

[0035] In this embodiment, the output power gain adjustment action is 2 dB (corresponding to exploration mode) or 1 dB (corresponding to utilization mode), and the specific value depends on the random number generation result.

[0036] Through the above process, the present invention realizes intelligent power decision based on the current link state, which can dynamically adjust the transmission power to optimize energy efficiency while ensuring communication reliability.

[0037] In some embodiments, taking the reinforcement learning model as an example including a deep Q-network model, the update method of the deep Q-network model may include: constructing an experience replay pool to store historical interaction data, each piece of historical interaction data including at least the current state, the action performed, the reward obtained, and the next state; randomly or by priority sampling several pieces of historical interaction data from the experience replay pool, and iteratively updating the network parameters of the deep Q-network model.

[0038] It should be noted that the experience replay pool is a first-in-first-out queue with a capacity of N. When the amount of stored data exceeds the capacity, the earliest experience data is discarded.

[0039] In this embodiment, the capacity of the experience replay pool is set to N=10000. Each piece of historical interaction data can be stored in the form of a 5-tuple: in: The state at time j, i.e., the 10-dimensional state vector; : The action performed at time j, i.e., the power gain level selected from the discrete action space; The immediate reward value obtained after performing an action is calculated by a multi-objective reward function; The state transitioned to in the next moment after the action is performed; : Task completion indicator, indicating whether the current communication cycle has ended (e.g., communication task completed or failed and terminated).

[0040] During model training, the agent interacts with the environment to generate experience data. Suppose a particular interaction occurs as follows: This piece of experience data is stored in the experience replay pool. As the interaction continues, the experience replay pool gradually accumulates a large amount of historical experience covering different scenarios.

[0041] Once the amount of data stored in the experience replay pool reaches a preset threshold (32 in this embodiment), the model training process is initiated. In each training step, a mini-batch of experience data is randomly sampled from the experience replay pool.

[0042] This embodiment employs a random sampling strategy, meaning each piece of experience has an equal probability of being sampled, and the batch size B = 32. The small batch of sampled data is denoted as: The sampled mini-batch data is input into the deep Q-network for parameter updates. The update process can be based on the Bellman optimality equation, achieved by minimizing the loss function: Calculate the target Q-value: For each experience, calculate the target Q-value using the target network. in, This is a discount factor (which can be set to 0.99 in this embodiment), representing the degree of importance attached to future rewards; if If true (indicating a terminated state), the target Q value only includes the immediate reward.

[0043] Calculate the loss function: Use the mean squared error (MSE) to calculate the error between the current network prediction and the target Q value. Backpropagation parameter update: Using the Adam optimizer, gradient descent is performed on the loss function with a learning rate of 0.001 to update the parameters of the current Q-network. .

[0044] Repeat the above sampling and update process until the model converges or reaches the preset number of training epochs. During training, the loss function value gradually decreases, indicating that the Q-value prediction of the current network is becoming more and more accurate.

[0045] Even after the trained model is deployed to a real communication device, it continues to be updated online. 1. After each communication is executed, the experience data generated in this interaction will be recorded. Stored in the experience replay pool; 2. When the amount of data in the experience replay pool reaches 32 records, an online learning update is triggered; 3. Perform the above sampling and update process; 4. Through continuous online learning, the model can adapt to the dynamic changes in the communication environment and continuously optimize the power control strategy.

[0046] Through the above training and update mechanism, the deep Q-network model of this invention can learn the optimal power control strategy from historical experience and continuously evolve in practical applications to adapt to complex and ever-changing communication environments.

[0047] In some embodiments, the method for updating the deep Q-network model further includes: constructing a target network with the same structure as the current network but whose parameter updates lag behind the current network; calculating a target Q-value using the target network, and calculating a loss function based on the error between the predicted Q-value of the current network and the target Q-value; and adjusting the parameters of the current network with the goal of minimizing the loss function.

[0048] It should be noted that, to ensure the stability of model training, this method constructs two deep Q-networks with identical structures but different parameter update strategies: 1. Current Network: denoted as The parameters are It is used to evaluate the Q-value of each action in real time and participates in action selection and loss function calculation. The network updates its parameters through backpropagation at each training step.

[0049] 2. Target Network: denoted as... The parameters are Its network structure is exactly the same as the current network (4 hidden layers, number of neurons 64→128→64→32, 10 neurons in the input layer, and 7 neurons in the output layer). The target network does not participate in real-time decision-making, but is only used to calculate the target Q value, and its parameter updates lag behind the current network.

[0050] At the start of training, the parameters of the target network are initialized to be the same as those of the current network, i.e. .

[0051] This embodiment uses a soft update strategy to update the target network, that is, in each training step, the target network parameters slowly approach the current network parameters according to the following formula: in, The soft update coefficient is set to 0.01 in this embodiment. This means that the target network only absorbs 1% of the current network's update each time, while the remaining 99% retains the original parameters, thus making the changes in the target network more gradual and providing a stable target value for training.

[0052] After sampling a small batch of data from the experience replay pool, the target network is used to calculate the target Q-value corresponding to each experience. Taking a sampled experience data point from the above embodiment as an example: Current status SNR = 25 dB, communication distance = 50 m, recent success rate = 0.96, ... Execute action Gain 1 dB Instant rewards 0.916 Next state SNR = 26 dB, communication distance = 50 m, recent success rate = 0.97, ... Task completion indicator : false (communication not finished) The formula for calculating the target Q value is: The calculation steps are as follows: 1. Enter the next state: Input the target network, and the target network outputs the Q-values ​​for 7 actions: The corresponding action range is 0 dB to 6 dB.

[0053] 2. Select the largest Q value: Choose the largest value from the above 7 Q values, i.e. (Corresponding action 1 dB).

[0054] 3. Discounted Future Rewards: This embodiment sets a discount factor. This indicates that future rewards are slightly less important than immediate rewards. (Calculation) .

[0055] 4. Consider the termination state: due to The communication has not yet ended, therefore future reward items are reserved. If so, the future reward item will be 0.

[0056] 5. Calculate the target Q value: .

[0057] Using the current network computation in state Next action The predicted Q-value. Input the current network, output the Q-values ​​of 7 actions, and select the corresponding action. Q value (with a gain of 1 dB): The loss function uses mean squared error (MSE) to calculate the difference between the predicted Q-value and the target Q-value: For this rule of thumb, the loss for a single sample is: The loss value for the current batch is obtained by averaging the losses calculated from all 32 sampled empirical data. .

[0058] To minimize the loss function To achieve the goal, the Adam optimizer is used for backpropagation to update the parameters of the current network. In this embodiment, the learning rate of the Adam optimizer is set to 0.001.

[0059] After the update is complete, update the target network parameters according to the soft update policy: The table below shows the changes in the target Q-value, predicted Q-value, and loss value over three consecutive training steps: As training progresses, the predicted Q-value gradually approaches the target Q-value, and the loss function continuously decreases, indicating that the current network is becoming increasingly accurate in estimating the value of actions.

[0060] Through the aforementioned dual-network structure and loss function optimization mechanism, the deep Q-network model of this invention can stably learn the optimal power control strategy, avoiding the overestimation and divergence problems commonly found in single-network training.

[0061] In some embodiments, the multi-objective reward function further includes a third reward term for evaluating data transmission efficiency; the first reward term, the second reward term, and the third reward term are weighted and combined according to preset weights to form the multi-objective reward function.

[0062] It should be noted that in this embodiment, the multi-objective reward function adopts a weighted summation form and consists of three reward terms: in: This is the first reward item, used to evaluate whether the communication was successful; This is the second award item, used to evaluate the energy efficiency of transmission power; This is the third reward item, used to evaluate data transmission efficiency; These are the weighting coefficients for each reward item, used to balance the importance of different optimization objectives.

[0063] In this embodiment, based on the optimization requirements of the communication scenario, the weighting coefficient is set as follows: , , This weighting configuration reflects a design philosophy that prioritizes communication reliability while also considering energy efficiency and data transmission efficiency.

[0064] First Reward The value reflects whether the communication task was successfully completed, and its selection rules are as follows: "Communication success" is defined as the receiver successfully demodulating the signal and the bit error rate being lower than a preset threshold (e.g., 10). -6 "Communication failure" is defined as the receiver failing to receive the signal correctly or the bit error rate exceeding the threshold.

[0065] Taking the communication result in the above embodiment as an example, the success rate of this communication was 97%, and the receiving end successfully demodulated the signal. Therefore, the communication was determined to be successful. .

[0066] Second Reward This reflects the energy efficiency of the current transmission power. The design philosophy is: under the premise of ensuring successful communication, the lower the transmission power, the higher the energy efficiency bonus. The specific calculation formula is as follows: in: This is the current final transmission power; The preset minimum transmit power (5 dBm in this embodiment); The preset maximum transmit power is 20 dBm in this embodiment.

[0067] This formula normalizes the actual power to the [0,1] interval. The lower the power, the smaller the normalized value and the higher the energy efficiency bonus; the higher the power, the larger the normalized value and the lower the energy efficiency bonus. When the actual power equals the minimum power, When the actual power equals the maximum power, .

[0068] Taking the final transmit power of 9 dBm in Example 3 as an example: Third reward item This reflects the current data transmission efficiency of communication, and its design philosophy is: the higher the throughput, the higher the reward. The specific calculation formula is as follows: in: This represents the actual throughput. The target throughput (i.e., the throughput expected to be achieved).

[0069] When communication is successful, the throughput reward is the ratio of the actual throughput to the target throughput, but not exceeding 1.0; when communication fails, the throughput is 0, and therefore the reward is 0.

[0070] Taking the communication results in the above embodiment as an example, the ratio of actual throughput to expected throughput is 0.98, therefore: The three reward items are weighted and combined according to preset weights to obtain the comprehensive reward value: The overall reward value is the immediate reward obtained after performing the power control action, and it will be stored in the experience replay pool for subsequent model training.

[0071] To more comprehensively demonstrate the effectiveness of multi-objective reward functions, the table below provides examples of reward calculation in different communication scenarios: As can be seen from the table above: In close-range, high-quality communication scenarios, the overall reward is relatively high (0.916) due to moderate power and high throughput. In long-distance, low-quality communication scenarios, although communication is successful and throughput meets the standard, the high power consumption results in a low energy efficiency reward and a decrease in the overall reward (0.750). In communication failure scenarios, regardless of how low the power is, the overall reward is always negative, guiding the model to avoid selecting actions that lead to communication failure. In ultra-low power communication scenarios, although the throughput decreases slightly, the energy efficiency reward is extremely high, and the overall reward reaches the highest level (0.949), reflecting the effect of balancing multiple objectives.

[0072] It should be noted that the above weighting coefficients can be dynamically adjusted according to the optimization needs of different application scenarios. For example: In battery-powered IoT devices, the weighting of energy efficiency rewards can be increased (e.g., ), further strengthening the energy conservation orientation; In high-throughput scenarios such as real-time video transmission, the weight of throughput rewards can be increased (e.g., Prioritize data transmission efficiency; In mission-critical communications, the weight of rewards for successful communications can be increased (e.g., This ensures maximum communication reliability.

[0073] Through the design of the multi-objective reward function described above, the reinforcement learning model of this invention can simultaneously optimize three key indicators—communication success rate, energy efficiency, and throughput—during the training process, and ultimately learn the optimal power control strategy to adapt to different scenario requirements.

[0074] Step S30: Adjust the current transmit power according to the power gain adjustment action to obtain and execute the final transmit power; wherein, the transmit power includes the radio frequency power of the communication power supply transmit signal.

[0075] In some embodiments, the power execution module first calculates the base transmit power based on the communication distance from the multidimensional state parameters of the current communication link. In this embodiment, taking a communication distance of 50 m as an example, the base transmit power is calculated to be 8 dBm according to a preset linear mapping relationship (base power increases linearly with distance). This mapping relationship is pre-stored in the power calculation unit to ensure that the base power can meet the basic communication requirements at different distances.

[0076] Basic transmit power P The base is superimposed on the gain value G corresponding to the power gain adjustment action output in step S20. In this embodiment, the gain value is 1dB, therefore the initial transmit power Pinitial is: Pinitial=Pbase+ G =8dBm + 1dB = 9dBm To ensure that the transmit power meets the hardware operating specifications of the communication power supply and to avoid damage to the RF front-end due to excessive power or communication failure due to insufficient power, the initial transmit power is limited. In this embodiment, the preset transmit power operating range is 5 dBm to 20 dBm. Since 9 dBm is within this range, no adjustment is required, and the final transmit power is 9 dBm.

[0077] If the calculated initial transmit power exceeds the operating range, it is limited to the boundary value. For example, if the base power is 18 dBm and the gain is 5 dB, the summed power is 23 dBm, and the final transmit power after limiting is 20 dBm; if the base power is 5 dBm and the gain is -2 dB (if negative gain exists), the summed power is 3 dBm, and the final transmit power after limiting is 5 dBm.

[0078] The calculated final transmit power (9 dBm) is converted into a control signal recognizable by the RF front end by the power execution unit, thereby driving the power amplifier in the power execution module (deployed in the communication device) to transmit the communication signal at that power value. In this embodiment, the transmit power refers to the RF power of the communication power supply transmitting the signal through the antenna, which directly determines the signal coverage and reception quality.

[0079] After signal transmission, the power execution module monitors and records the communication results and feeds them back to the state awareness module. In this embodiment, after transmission at a power of 9 dBm, the feedback communication success rate was 97%, the ratio of actual throughput to expected throughput was 0.98, and the number of retransmissions was 0. These results will be used as historical performance parameters to update the state awareness module and provide a basis for subsequent reward calculation and online model learning.

[0080] Through the above steps, the present invention achieves complete closed-loop control from state perception and intelligent decision-making to power execution, dynamically adjusting the transmission power according to the current link state, and optimizing energy efficiency while ensuring communication quality.

[0081] In some embodiments, the first reward item takes a positive value when communication is successful and a negative value when communication fails; the second reward item is negatively correlated with the current transmit power; and the third reward item is positively correlated with the throughput of the current communication link.

[0082] Specifically, the first reward item Used to evaluate the success or failure of communication, its value is determined by the following rule: a positive value indicates successful communication, and a negative value indicates communication failure. The model is forced to prioritize successful communication and avoid selecting power actions that might lead to failure. In this embodiment, the specific values ​​are as follows: The design intent of this value selection rule is to reinforce actions that lead to successful communication by providing positive rewards and penalize actions that lead to communication failure by providing negative rewards, so that the model prioritizes power control strategies that can guarantee communication reliability during the learning process.

[0083] Second Reward The energy efficiency used to evaluate transmit power is determined by a rule that it is negatively correlated with the current transmit power; that is, the lower the transmit power, the higher the energy efficiency reward, and vice versa. Under the premise of ensuring successful communication, the incentive model selects the lowest possible power level.

[0084] In this embodiment, a linear negative correlation function is used to implement this rule: in: The current final transmit power (unit: dBm); The preset minimum transmit power is set to 5 dBm in this embodiment; The preset maximum transmit power is set to 20 dBm in this embodiment.

[0085] This function normalizes the transmit power to the [0,1] interval, when the actual power equals the minimum power. When the actual power equals the maximum power, When the actual power is between the two values, the energy efficiency bonus decreases linearly with the increase of power, strictly satisfying the "negative correlation" value rule.

[0086] Third reward item The reward is used to evaluate data transmission efficiency, and its value is determined by a positive correlation with the throughput of the current communication link; that is, the higher the throughput, the higher the reward, and the lower the throughput, the lower the reward. The incentive model aims for higher data transmission efficiency while ensuring successful communication.

[0087] In this embodiment, a ratio function is used to implement this rule, and it is calculated only when communication is successful: in: Actual throughput (unit: Mbps); The target throughput is the expected throughput (in Mbps).

[0088] When communication is successful, the throughput reward is the ratio of the actual throughput to the target throughput, but with a maximum of 1.0; when communication fails, the actual throughput is 0, so the reward value is 0. This rule ensures that the higher the throughput (the closer it is to or the closer it is to the target value), the higher the reward, strictly satisfying the "positive correlation" value rule.

[0089] To clearly illustrate the above value selection rules, this embodiment sets up three different communication scenarios to calculate the value of each reward item respectively: Scenario A: High-quality short-range communication Communication result: Successful (97% success rate) Transmit power: 9 dBm Actual throughput / target throughput: 0.98 Calculation of each reward item: (Positive value for successful communication) (Lower power consumption, higher energy efficiency bonus) (Higher reward for throughput close to target) Scenario B: Long-distance low-quality communication (Continued from Example 2) Communication result: Successful (92% success rate) Transmit power: 17.5 dBm Actual throughput / Target throughput: 1.0 Calculation of each reward item: (Positive value for successful communication) (Higher power consumption, lower energy efficiency bonus) (If the throughput target is met, the maximum reward will be applied.) Scenario C: Communication Failure Scenario Communication result: Failed (the receiver failed to demodulate the signal correctly) Transmit power: 12 dBm Actual throughput / target throughput: 0 Calculation of each reward item: (Negative values ​​are taken for communication failures) (The power is moderate, but the communication failure renders the reward meaningless.) (Communication failed, throughput is 0) Secondly, embodiments of this application also provide a power control device 00 for a communication power supply.

[0090] In one embodiment, reference is made to Figure 2 , Figure 2 This is a functional module diagram of an embodiment of the power control device 00 for a communication power supply according to this application. Figure 2 As shown, the power control device for the communication power supply includes: The acquisition module 01 is used to acquire multi-dimensional state parameters related to the current communication link; wherein, the multi-dimensional state parameters include channel state parameters, communication service requirement parameters, and historical performance parameters; The adjustment module 02 is used to input the multidimensional state parameters into a pre-trained reinforcement learning model and output discrete power gain adjustment actions through the reinforcement learning model; wherein, the reinforcement learning model is trained by maximizing a multi-objective reward function, and the multi-objective reward function includes at least a first reward term for evaluating whether communication is successful and a second reward term for evaluating the energy efficiency of transmission power; The execution module 03 is used to adjust the current transmit power according to the power gain adjustment action in order to obtain and execute the final transmit power; wherein, the transmit power includes the radio frequency power of the communication power supply transmit signal.

[0091] The functions of each module in the power control device of the above-mentioned communication power supply correspond to the steps in the embodiment of the power control method of the above-mentioned communication power supply, and their functions and implementation processes will not be described in detail here.

[0092] Thirdly, embodiments of this application provide a communication power supply, which may include a communication power supply 20. The communication power supply 20 may be a mobile terminal, base station, access point, Internet of Things terminal, data center equipment, or other device with wireless communication function and requiring power control.

[0093] Reference Figure 3 , Figure 3 This is a schematic diagram of the hardware structure of the communication power supply 20 involved in the embodiments of this application. In this embodiment, the communication power supply 20 may include a processor 21, a memory 22, a communication interface 23, a communication bus 24, and a communication power supply 25. The power source for the communication power supply 20 may be mains power, a battery, etc.

[0094] The communication bus 24 can be of any type and is used to interconnect the processor 21, the memory 22 and the communication interface 23.

[0095] The communication interface 23 includes input / output (I / O) interfaces, physical interfaces, and logical interfaces for interconnecting devices within the communication power supply, as well as interfaces for interconnecting the communication power supply with other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.

[0096] The memory 22 can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0097] Processor 21 can be a general-purpose processor. This processor can call the power control program stored in memory 22 and execute the power control method for the communication power supply provided in this embodiment. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the power control program is called can be found in various embodiments of the power control method for the communication power supply in this application, and will not be repeated here.

[0098] Those skilled in the art will understand that Figure 3The hardware structure shown does not constitute a limitation of this application and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0099] Fourthly, embodiments of this application also provide a computer-readable storage medium.

[0100] The present application has a power control program stored on a computer-readable storage medium, wherein when the power control program is executed by the processor 21, it implements the steps of the power control method for the communication power supply as described above.

[0101] The method implemented when the power control program is executed can be referred to in various embodiments of the power control method of the communication power supply of this application, and will not be repeated here.

[0102] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0103] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.

[0104] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.

[0105] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0106] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0107] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0108] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A power control method for a communication power supply, characterized in that, The power control method for the communication power supply includes: Obtain multi-dimensional state parameters related to the current communication link; wherein, the multi-dimensional state parameters include channel state parameters, communication service requirement parameters, and historical performance parameters; The multidimensional state parameters are input into a pre-trained reinforcement learning model, and the reinforcement learning model outputs discrete power gain adjustment actions; wherein, the reinforcement learning model is trained through a multi-objective reward function, which includes at least a first reward term for evaluating whether communication is successful and a second reward term for evaluating the energy efficiency of transmission power; According to the power gain adjustment action, the current transmit power is adjusted to obtain and execute the final transmit power; wherein, the transmit power includes the radio frequency power of the communication power supply transmit signal.

2. The power control method for a communication power supply as described in claim 1, characterized in that, The step of adjusting the gain of the current transmit power according to the power gain adjustment action includes: The base transmit power is determined based on the multidimensional state parameters of the current communication link. The initial transmission power is obtained by superimposing the base transmission power with the gain value corresponding to the power gain adjustment action; The initial transmit power is limited to obtain the final transmit power.

3. The power control method for a communication power supply as described in claim 1, characterized in that, The reinforcement learning model includes a deep Q-network model, and the update method of the deep Q-network model includes: Build an experience replay pool to store historical interaction data. Each piece of historical interaction data should include at least the current state, the action performed, the reward obtained, and the next state. Several historical interaction data points are randomly or sampled from the experience replay pool according to priority, and the network parameters of the deep Q network model are iteratively updated.

4. The power control method for a communication power supply as described in claim 3, characterized in that, The update method for the deep Q-network model also includes: Construct a target network with the same structure as the current network, but whose parameter updates lag behind the current network; The target Q-value is calculated using the target network, and a loss function is calculated based on the error between the predicted Q-value of the current network and the target Q-value. The parameters of the current network are adjusted with the goal of minimizing the loss function.

5. The power control method for a communication power supply as described in claim 1, characterized in that, The multi-objective reward function also includes a third reward term for evaluating data transmission efficiency; the first reward term, the second reward term, and the third reward term are weighted and combined according to preset weights to form the multi-objective reward function.

6. The power control method for a communication power supply as described in claim 5, characterized in that, The first reward item takes a positive value when communication is successful and a negative value when communication fails; the second reward item is negatively correlated with the current transmission power; and the third reward item is positively correlated with the throughput of the current communication link.

7. The power control method for a communication power supply as described in claim 1, characterized in that, The discrete power gain adjustment action output by the reinforcement learning model includes: The reinforcement learning model is used to evaluate the value of each candidate power gain adjustment action in the preset discrete action space to obtain the evaluation result. Based on the evaluation results and the preset selection strategy, a candidate power gain adjustment action is selected as the power gain adjustment action to be executed. The preset selection strategy includes an ε-greedy strategy, which includes first randomly selecting a candidate power gain adjustment action with a first probability, and then selecting the candidate power gain adjustment action with the largest Q value in the current state with a second probability, wherein the first probability is less than the second probability.

8. A power control device for a communication power supply, characterized in that, The power control device for the communication power supply includes: The acquisition module is used to acquire multi-dimensional state parameters related to the current communication link; wherein, the multi-dimensional state parameters include channel state parameters, communication service requirement parameters, and historical performance parameters; An adjustment module is used to input the multidimensional state parameters into a pre-trained reinforcement learning model and output discrete power gain adjustment actions through the reinforcement learning model; wherein, the reinforcement learning model is trained by maximizing a multi-objective reward function, the multi-objective reward function includes at least a first reward term for evaluating whether communication is successful and a second reward term for evaluating the energy efficiency of transmission power; An execution module is used to adjust the current transmit power gain according to the power gain adjustment action to obtain and execute the final transmit power; wherein the transmit power includes the radio frequency power of the communication power supply transmit signal.

9. A communication power supply, characterized in that, The communication power supply includes a processor, a memory, and a power control program stored in the memory and executable by the processor, wherein when the power control program is executed by the processor, it implements the steps of the power control method of the communication power supply as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a power control program, wherein when the power control program is executed by a processor, it implements the steps of the power control method for a communication power supply as described in any one of claims 1 to 7.

Images