A method for optimizing energy efficiency when deploying ultra-reliable low-latency devices in large quantities

Abstract

This invention provides an energy efficiency optimization method for the large-scale deployment of ultra-reliable low-latency (URLLC) devices, comprising: Step 1: Calculating the short packet collision probability and short packet reception failure probability of a single device, and calculating the packet loss rate of the device based on the calculated short packet collision probability and short packet reception failure probability; Step 2: Calculating the packet loss rate of each device, and grouping devices with similar communication environments into one category based on the calculated packet loss rate; Step 3: Optimizing the energy efficiency of all devices categorized in Step 2 using a deep reinforcement learning method. The beneficial effects of this invention are: 1. The method can effectively reduce the transmission power of sensing devices sending short packets and the number of times the same short packets are repeatedly sent, significantly improving the communication energy efficiency of URLLC devices and extending their battery life; 2. The method of this invention uses a classification method, which significantly shortens the training time of deep reinforcement learning.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and in particular to a method for optimizing energy efficiency when deploying a large number of ultra-reliable low-latency devices. Background Technology

[0002] URLLC is an important service category supported by fifth-generation (5G) mobile communication networks. Based on its high reliability and low latency performance, URLLC can be applied to the monitoring and early warning of emergencies. By installing batteries in sensing devices, these devices can be deployed on a large scale in complex, unmanned environments using URLLC communication services, saving labor costs. According to the specifications of the 3rd Generation Partnership Project (3GPP), an URLLC device needs to complete short packet transmission within a communication transmission latency of less than 1 millisecond, and the packet loss probability of the short packets should not exceed 10%. -5 .

[0003] To reduce transmission latency, URLLC devices typically transmit short data packets of a few tens of bytes to control latency. To reduce packet loss, URLLC devices employ repetitive transmission techniques, sending the same short packets consecutively within adjacent time periods. While continuously sending the same short packets reduces packet loss, it increases transmission latency and shortens the battery life of the URLLC sensing device. To ensure URLLC performance, existing technologies, based on a single cellular base station environment, propose methods to control transmit power to improve communication energy efficiency and extend battery life.

[0004] Existing technical solutions assume that the number of retransmissions for all users is fixed, that is, to retransmit as many times as possible within a specified 1 millisecond, thereby improving the energy efficiency of the device by reducing the transmission power. However, too many retransmissions will limit further improvements in energy efficiency.

[0005] Furthermore, existing technical solutions assume that all devices send short packets at every moment, an assumption that does not align with the monitoring scenario of URLLC sensing devices. Devices only send short packets to the central system for early warning when they detect a sudden event. Therefore, the short packet transmission of URLLC sensing devices is sporadic; that is, devices do not send short packets at every moment, but only report data when a sudden event occurs.

[0006] Existing technical solutions only consider the communication environment of a single cellular base station. In reality, large-scale URLLC equipment is often deployed in multiple adjacent cellular base stations. If equipment in adjacent base stations uses the same communication frequency band, there will be inter-cell interference.

[0007] Therefore, existing energy efficiency (EE) optimization techniques are not suitable for environments with large-scale URLLC equipment deployments within adjacent base stations. Summary of the Invention

[0008] To address the problems in the prior art, this invention provides a method for optimizing energy efficiency when deploying a large number of ultra-reliable, low-latency devices.

[0009] This invention provides a method for optimizing energy efficiency when deploying a large number of ultra-reliable, low-latency devices, comprising the following steps:

[0010] Step 1: Calculate the short packet collision probability and the short packet reception failure probability of a single device, and calculate the packet loss rate of the device based on the calculated short packet collision probability and the short packet reception failure probability.

[0011] Step 2: Calculate the packet loss rate for each device, and group devices with similar communication environments into one category based on the calculated packet loss rate;

[0012] Step 3: Optimize the energy efficiency of all devices classified in Step 2 using deep reinforcement learning.

[0013] As a further improvement of the present invention, step two specifically includes:

[0014] Step 1: Initialize the device's power P and retransmission count K;

[0015] Step 2: Calculate the packet loss rate for each device;

[0016] Step 3: Group devices with a packet loss rate less than or equal to the set value into one category;

[0017] Step 4: Determine if there are still any unclassified devices. If so, proceed to the next step; otherwise, end the classification process.

[0018] Step 5: Determine if the number of retransmissions K has reached the maximum. If so, increase the device's transmit power P by ΔP milliwatts and then return to step 2. Otherwise, increment the number of retransmissions K by 1 and then return to step 2.

[0019] As a further improvement of the present invention, step three specifically includes:

[0020] Step S1: The devices classified in Step 2 are treated as a learning agent for each category. All learning agents obtain the current environment state of step g from the communication environment. Any learning agent inputs the current environment state into its own trained neural network to obtain the corresponding action. The actions of all learning agents work together on the communication environment. The communication environment returns the effect or reward after the learning agent's action is executed. The communication environment will enter a new environment state in the next step g+1 due to the influence of the action.

[0021] Step S2: The learning agent combines this information with the environmental state and selected action input in step g to form an experience sample and saves it in its own experience storage pool. At the same time, the learning agent will randomly draw a number of experience samples from the experience storage pool at set intervals for training its own neural network. After training, each learning agent will select an appropriate action after obtaining the current state of the environment, so that the long-term energy efficiency of the entire system can reach the optimal level.

[0022] As a further improvement of the present invention, in step S1, the actions of the learning agent include the number of repeated transmissions K and the transmission power P, and the environmental state input to the learning agent includes the short packet collision count, the short packet reception failure count, and the number of devices currently included in the learning agent.

[0023] As a further improvement of the present invention, in step S2, the training process of learning the agent's own neural network includes:

[0024] Step Y1: Start the communication network and initialize the learning agent network parameters;

[0025] Step Y2: Update the network environment and categorize the equipment for each base station;

[0026] Step Y3: Determine if all rounds have been completed. If yes, end; otherwise, proceed to the next step.

[0027] Step Y4: Begin a new moment, learn agent selection actions;

[0028] Step Y5: After all learning agents have selected actions, they interact with the communication environment;

[0029] Step Y6: Update the communication network and categorize the equipment for each base station;

[0030] Step Y7: Learn to sample using a proxy and train the neural network;

[0031] Step Y8: Determine if the current game has ended. If yes, execute step Y3; otherwise, execute step Y4.

[0032] As a further improvement of the present invention, in step 3, the packet loss rate is less than or equal to 10%.-5 The equipment is classified into one category.

[0033] The present invention also discloses a system for optimizing energy efficiency when deploying a large number of ultra-reliable low-latency devices, comprising: a memory, a processor, and a computer program stored in the memory, wherein the computer program is configured to implement the steps of the method described in the present invention when invoked by the processor.

[0034] The present invention also discloses a computer-readable storage medium storing a computer program configured to implement the steps of the method described in the present invention when invoked by a processor.

[0035] The beneficial effects of this invention are: 1. The method of this invention can effectively reduce the transmission power of sensing devices sending short packets and the number of times the same short packets are repeatedly sent, significantly improving the communication energy efficiency of URLLC devices and extending their battery life; 2. The method of this invention adopts a classification method, which significantly shortens the training time of deep reinforcement learning. Attached Figure Description

[0036] Figure 1 This is a system model diagram of the present invention;

[0037] Figure 2 This is a flowchart illustrating the equipment classification process for each base station in this invention;

[0038] Figure 3 This is a flowchart illustrating the interaction between the Agent and the communication environment in this invention.

[0039] Figure 4 This is a flowchart of the Agent neural network training process of the present invention. Detailed Implementation

[0040] This invention relates to the field of communication technology, primarily focusing on optimizing communication energy efficiency when a large number of ultra-reliable and low-latency communications (URLLC) devices are deployed across multiple adjacent cellular base stations. Currently, URLLC sensing devices can be equipped with batteries and deployed on a large scale in unattended locations to monitor emergencies and send short messages to a central system for early warning. However, unattended environments are complex, such as automated factories with numerous production lines and intelligent delivery vehicles shuttling back and forth, leading to complex and poor communication environments that severely impact the communication quality of URLLC devices and prolong their information transmission time. To meet the ultra-high reliability and low latency communication requirements of URLLC devices while extending their battery life, this invention provides a method for optimizing communication energy efficiency when deploying a large number of URLLC devices based on deep reinforcement learning. Specifically, this invention uses deep reinforcement learning to learn the communication environment of all URLLC devices and assesses the required transmission power and the number of times the same short packets need to be repeatedly transmitted based on the communication environment. This invention can effectively reduce the transmission power of sensing devices sending short packets and the number of times the same short packets are repeatedly sent, significantly improving the communication energy efficiency of URLLC devices and extending their battery life.

[0041] Furthermore, this invention proposes a device classification method that groups devices with similar communication environments into one category. This classification method significantly reduces the training time for deep reinforcement learning.

[0042] This invention provides a method for optimizing energy efficiency when deploying ultra-reliable low-latency devices on a large scale, comprising the following steps:

[0043] Step 1: Calculate the short packet collision probability and the short packet reception failure probability of a single device, and calculate the packet loss rate of the device based on the calculated short packet collision probability and the short packet reception failure probability.

[0044] Step 2: Calculate the packet loss rate for each device, and group devices with similar communication environments into one category based on the calculated packet loss rate;

[0045] Step 3: Optimize the energy efficiency of all devices classified in Step 2 using deep reinforcement learning.

[0046] This invention considers a large number of URLLC devices deployed within multiple adjacent cellular base stations. The system model is as follows: Figure 1 As shown, this includes multiple URLLC devices and two adjacent cellular base stations. The two adjacent base stations use the same channel, and the vehicle moves randomly within the environment, constantly affecting the communication environment of the devices. Figure 1As shown, devices 1 through 5 send short packets at specific times, each selecting a different channel. Devices 1 and 2, within the same base station, simultaneously select channel 2, resulting in a channel collision; devices 3 and 5, at different base stations, select channel 3, causing inter-station interference. Channel collisions directly lead to the failure to receive short packets sent by the devices; inter-station interference degrades the communication environment of the devices, reducing the success rate of the base station in receiving short packets. Therefore, a device's packet loss rate consists of two parts: the probability of short packet collisions and the success rate of short packet reception.

[0047] For a device n at base station m, the signal-to-interference-plus-noise ratio (SINR) received by base station m from the device can be expressed as:

[0048]

[0049] Among them, P n,m β represents the transmit power of device n. n,m α represents the path loss between device n and base station m. n,m h represents the shadow fading coefficient between device n and base station m. n,m X represents the fast fading coefficient between device n and base station m. j,f This indicates whether device f from other base stations f has a short packet transmission at that moment and has selected the same channel as device n. Therefore, given the SINR threshold γ... th Afterwards, base station m receives the short packet from device n, if γ n,m ≥γ th If the short packet is successfully received, then the short packet reception is successful; otherwise, the short packet reception fails. What is the probability of short packet reception failure for device n in base station m? It can be represented as

[0050]

[0051] The short packet collision probability of device n in base station m can be expressed as:

[0052]

[0053] in, This indicates the probability that, within base station m, in addition to device n, there are i other devices that simultaneously send short packets. Let represent the probability that device i does not choose the same channel as device n, and K represent the number of times the same short packet is repeatedly sent. After calculating the short packet collision probability and the short packet reception failure probability of device n, the packet loss probability of this device can be expressed as:

[0054]

[0055] in, This represents the probability that when device n continuously transmits K identical short packets, j short packets will not cause channel collisions with other devices.

[0056] Next, the packet loss rate of each device is calculated, and devices with similar channel environments are grouped together. The specific classification process is as follows: Figure 2 As shown, the specific steps include:

[0057] Step 1: Initialize the device's power P and retransmission count K;

[0058] Step 2: Calculate the packet loss rate for each device;

[0059] Step 3: Ensure the packet loss rate is less than or equal to 10%. -5 The equipment is classified into one category;

[0060] Step 4: Determine if there are still any unclassified devices. If so, proceed to the next step; otherwise, end the classification process.

[0061] Step 5: Determine if the number of retransmissions K has reached the maximum. If so, increase the device's transmit power P by ΔP milliwatts and then return to step 2. Otherwise, increment the number of retransmissions K by 1 and then return to step 2.

[0062] After all base station equipment has been categorized, this invention employs deep reinforcement learning to optimize the energy efficiency of all equipment, specifically including:

[0063] Step S1: Treat each of the categorized devices as a learning agent. All agents obtain the current environment state S from the communication environment. g Any agent c inputs the current environmental state into its trained neural network to obtain the corresponding action (Action, Action). c,m,g All actions of the learning agents work together in the communication environment, which returns the effect or reward (Reward, R) of the learning agent's actions. g+1 Furthermore, in the next step g+1, the communication environment enters a new environment state S due to the influence of the action. g+1 The interaction process between the Agent and the communication environment described above is as follows: Figure 3 As shown. The Agent's Action includes the number of repeated transmissions K and the transmission power P, and the input Agent's state S. g This includes statistics on the number of short packet collisions, the number of short packet reception failures, the number of devices currently included in the Agent, and the reward R returned by the communication environment. g+1 Current energy efficiency (Unit: Packets / Joule)

[0064] Step S2: When the communication environment returns reward R g+1 and the new state S g+1 Then, Agent c will combine this information (reward information returned by the communication environment, and environment state information from step g+1) with the state S input from the previous step (step g). g and the selected action A c,m,g These samples are merged into a single experience sample and stored in the agent's own experience storage pool. Simultaneously, the agent periodically draws several experience samples from the experience storage pool for training its own neural network. The specific training process is as follows: Figure 4 As shown, the training process of the neural network that learns the agent itself includes:

[0065] Step Y1: Start the communication network and initialize the learning agent network parameters;

[0066] Step Y2: Update the network environment and categorize the equipment for each base station;

[0067] Step Y3: Determine if all rounds have been completed. If yes, end; otherwise, proceed to the next step.

[0068] Step Y4: Begin a new moment, learn agent selection actions;

[0069] Step Y5: After all learning agents have selected actions, they interact with the communication environment;

[0070] Step Y6: Update the communication network and categorize the equipment for each base station;

[0071] Step Y7: Learn to sample using a proxy and train the neural network;

[0072] Step Y8: Determine if the current game has ended. If yes, proceed to step Y3; otherwise, proceed to step Y4.

[0073] A complete training cycle for the agent consists of several episodes, each containing several steps. During this extended period, at each step, the agent selects an action based on the current environment and receives new state information and a reward from the environment. Subsequently, the agent draws samples from an experience pool as input to train its neural network. Training continues until all episodes are completed. After training, each agent, upon receiving the state of the current environment, selects an appropriate action to optimize the long-term energy efficiency of the entire system.

[0074] The present invention also discloses a system for optimizing energy efficiency when deploying a large number of ultra-reliable low-latency devices, comprising: a memory, a processor, and a computer program stored on the memory, wherein the computer program is configured to implement the steps of the method described in the present invention when called by the processor.

[0075] The present invention also discloses a computer-readable storage medium storing a computer program configured to implement the steps of the method described in the present invention when invoked by a processor.

[0076] The beneficial effects of this invention are: 1. The method of this invention can effectively reduce the transmission power of sensing devices sending short packets and the number of times the same short packets are repeatedly sent, significantly improving the communication energy efficiency of URLLC devices and extending their battery life; 2. The method of this invention adopts a classification method, which significantly shortens the training time of deep reinforcement learning.

[0077] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A method for optimizing energy efficiency when deploying a large number of ultra-reliable low-latency devices, characterized in that, Includes the following steps: Step 1: Calculate the short packet collision probability and the short packet reception failure probability of a single device, and calculate the packet loss rate of the device based on the calculated short packet collision probability and the short packet reception failure probability. Step 2: Calculate the packet loss rate for each device, and group devices with similar communication environments into one category based on the calculated packet loss rate; Step 3: Optimize the energy efficiency of all devices classified in Step 2 using deep reinforcement learning methods; Step two specifically involves: Step 1: Initialize the device's power P and retransmission count K; Step 2: Calculate the packet loss rate for each device; Step 3: Group devices with a packet loss rate less than or equal to the set value into one category; Step 4: Determine if there are still any unclassified devices. If so, proceed to the next step; otherwise, end the classification process. Step 5: Determine whether the number of retransmissions K has reached the maximum. If so, increase the device's transmit power P by ∆P milliwatts and then return to step 2. Otherwise, increment the number of retransmissions K by 1 and then return to step 2. Step three specifically involves: Step S1: The devices classified in Step 2 are treated as a learning agent for each category. All learning agents obtain the current environment state of step g from the communication environment. Any learning agent inputs the current environment state into its own trained neural network to obtain the corresponding action. The actions of all learning agents work together on the communication environment. The communication environment returns the effect or reward after the learning agent's action is executed. The communication environment will enter a new environment state in the next step g+1 due to the influence of the action. Step S2: The learning agent merges the reward information returned by the communication environment, the environment state information of step g+1, the environment state input in step g, and the selected action into an experience sample and saves it in its own experience storage pool. At the same time, the learning agent will randomly draw a number of experience samples from the experience storage pool at set intervals for training its own neural network. After training, each learning agent will select an appropriate action after obtaining the current state of the environment, so that the long-term energy efficiency of the entire system can reach the optimal level.

2. The method of claim 1, wherein, In step S1, the actions of the learning agent include the number of repeated transmissions K and the transmission power P. The environmental state input to the learning agent includes the short packet collision count, the short packet reception failure count, and the number of devices currently included in the learning agent.

3. The method of claim 1, wherein, In step S2, the training process of the neural network for learning the agent itself includes: Step Y1: Start the communication network and initialize the learning agent network parameters; Step Y2: Update the network environment and categorize the equipment for each base station; Step Y3: Determine if all rounds have been completed. If yes, end; otherwise, proceed to the next step. Step Y4: Begin a new moment, learn agent selection actions; Step Y5: After all learning agents have selected actions, they interact with the communication environment; Step Y6: Update the communication network and categorize the equipment for each base station; Step Y7: Learn to sample using a proxy and train the neural network; Step Y8: Determine if the current game has ended. If yes, execute step Y3; otherwise, execute step Y4.

4. The method of energy efficiency optimization for massive deployment of ultra-reliable and low-latency devices according to claim 1, wherein, In the step 3, the devices with the packet loss rate less than or equal to 10 -5 are classified into one category. 5.A system for energy efficiency optimization in massive deployment of ultra-reliable and low latency devices, the system comprising: include: A memory, a processor, and a computer program stored on the memory, the computer program being configured to implement the steps of the method of any one of claims 1-4 when invoked by the processor.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program configured to implement the steps of the method according to any one of claims 1-4 when invoked by a processor.

Images