Power terminal retransmission strategy model training method, power terminal retransmission strategy model determining method and related device

Abstract

The invention belongs to the field of power communication, and discloses a power terminal retransmission strategy model training method, a power terminal retransmission strategy model determining method and a related device.In the training stage, all power terminals train a global retransmission strategy model through local data, and only model parameters are uploaded to a central server; and the privacy security and independence of the power grid data are fundamentally ensured. The central server adopts a weighted aggregation algorithm based on an attention mechanism, intelligent fusion is carried out according to quality differences of terminal model parameters, and the convergence speed and the training efficiency of a global model are effectively improved. And in the strategy determination stage, the power terminal acquires communication state data in real time, and dynamically outputs an optimal retransmission scheme in the next time period through the trained retransmission strategy model. According to the invention, on the premise of following the data security specification of the power industry, the data transmission success rate is remarkably improved, the end-to-end transmission delay and network overhead are effectively reduced, and meanwhile, the training burden of a single node is relieved through a distributed computing architecture.

Description

Technical Field

[0001] This invention belongs to the field of power communication and relates to a training method, determination method and related apparatus for a power terminal retransmission strategy model. Background Technology

[0002] The power system communication network is the nerve center of the smart grid, carrying the transmission of critical business data such as protection and control, data acquisition, and status monitoring. Its communication environment often faces complex challenges such as strong electromagnetic interference, multipath fading, and high noise, leading to high packet loss rates and decreased reliability. While traditional Automatic Repeat Quest (ARQ) mechanisms and their improved algorithms are simple and easy to use, they suffer from fixed retransmission strategies, inability to adapt to harsh, time-varying environments, and frequent retransmissions exacerbating network congestion and latency.

[0003] In recent years, some studies have attempted to apply artificial intelligence algorithms such as reinforcement learning (RL) to retransmission decisions, learning the optimal retransmission strategy through the interaction between the agent and the environment. However, these methods typically require the centralized collection of large amounts of environmental data for training, which presents two major challenges: First, power production control data is highly sensitive, and centralized collection poses a risk of privacy breaches, which does not comply with power industry safety regulations; second, power terminals are widely distributed, and the electromagnetic environments, network loads, and service types they face are significantly heterogeneous, making it difficult for a centralized global model to achieve optimal performance in all local scenarios. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a training method, determination method and related apparatus for a power terminal retransmission strategy model.

[0005] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, the present invention provides a method for training a retransmission strategy model for power terminals, comprising: an iterative update step until a preset iteration termination condition is reached; a central server sending the current global retransmission strategy model as a training retransmission strategy model to each power terminal; wherein the update step comprises: the central server sending the current global retransmission strategy model to each power terminal; the power terminal training the current global retransmission strategy model based on local data to obtain a local training retransmission strategy model and sending the model parameters of the local training retransmission strategy model to the central server; and the central server updating the current global retransmission strategy model based on the model parameters of the local training retransmission strategy model sent by each power terminal, using an aggregation method based on an attention mechanism.

[0006] Optionally, the global retransmission strategy model is constructed based on a reinforcement learning model; the state space of the global retransmission strategy model is: downlink channel block error rate, average downlink channel quality index, distance between transceiver devices, and number of resource blocks; the action space of the global retransmission strategy model is: A0~A8; where A0 is the 5G HARQ transmission scheme, A1 is the time-domain delay strategy with a delay time of 1 time slot, A2 is the time-domain delay strategy with a delay time of 2 time slots, A3 is the retransmission strategy with a repetition count of 1, A4 is the retransmission strategy with a repetition count of 2, A5 is the hybrid strategy with a delay time of 1 time slot and a repetition count of 1, A6 is the hybrid strategy with a delay time of 1 time slot and a repetition count of 2, A7 is the hybrid strategy with a delay time of 1 time slot and a repetition count of 2, and A8 is the hybrid strategy with a delay time of 2 time slots and a repetition count of 2.

[0007] Optionally, the reward function of the global retransmission strategy model is: when ≤ hour:

[0008] in, The downlink channel block error rate for the 5G HARQ transmission scheme. The preset downlink channel block error rate threshold. This is the reward function.

[0009] when > hour:

[0010]

[0011]

[0012]

[0013]

[0014]

[0015] in, For the basic reward function, The downlink channel block error rate for the current retransmission strategy. This is the reward coefficient; To enhance the reward function, This is the reward function for the average downlink channel quality index. The distance reward function between the transmitting and receiving devices. The reward function is the number of resource blocks. This is the average downlink channel quality metric. The distance between transceiver devices This represents the number of resource blocks.

[0016] Optionally, when the power terminal trains the current global retransmission strategy model based on local data, it uses the Q-Learning method for training and employs a classical neural network to fit the current Q value in the Q-Learning method.

[0017] Optionally, the central server updates the current global retransmission strategy model based on the model parameters of the locally trained retransmission strategy model sent by each power terminal, using an attention-based aggregation method, including: The aggregate weight of each power terminal is obtained by the following formula:

[0018] in, For power terminals No. The aggregation weight for each aggregation period. For viewing window Internal power terminal The total number of requests for all content uploaded. To observe the size of the window, This refers to the number of power terminals.

[0019] Update the current global retransmission strategy model using the following formula:

[0020] in, For the first The model parameters of the global retransmission strategy model after aggregation in each aggregation cycle. For power terminals No. Model parameters of the local training retransmission strategy model for each aggregation cycle.

[0021] In a second aspect, the present invention provides a method for determining a retransmission strategy for a power terminal, comprising: the power terminal acquiring downlink channel block error rate, average downlink channel quality index, distance between transceiver devices, and number of resource blocks for the current time period to obtain communication status data for the current time period; the power terminal obtaining a retransmission strategy for the next time period by training a retransmission strategy model based on the communication status data for the current time period; wherein the training retransmission strategy model is obtained based on the above-mentioned power terminal retransmission strategy model training method.

[0022] In a third aspect, the present invention provides a power terminal retransmission strategy model training system, comprising a control module, a central server, and several power terminals; the control module is used to control the iterative update steps of the central server and power terminals until a preset iteration termination condition is reached, and to control the central server to send the current global retransmission strategy model as a training retransmission strategy model to each power terminal; wherein, the update step includes: the central server sending the current global retransmission strategy model to each power terminal; the power terminal training the current global retransmission strategy model based on local data to obtain a local training retransmission strategy model and sending the model parameters of the local training retransmission strategy model to the central server; the central server updating the current global retransmission strategy model based on the model parameters of the local training retransmission strategy model sent by each power terminal, using an aggregation method based on an attention mechanism.

[0023] In a fourth aspect, the present invention provides a power terminal retransmission strategy determination system, comprising: a data acquisition module, used by the power terminal to acquire downlink channel block error rate, average downlink channel quality index, distance between transceiver devices, and number of resource blocks for the current time period, thereby obtaining communication status data for the current time period; and a strategy determination module, used by the power terminal to obtain a retransmission strategy for the next time period based on the communication status data for the current time period by training a retransmission strategy model; wherein the training retransmission strategy model is obtained based on the aforementioned power terminal retransmission strategy model training method.

[0024] In a fifth aspect, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the power terminal retransmission strategy model training method described above, or the steps of the power terminal retransmission strategy determination method described above.

[0025] In a sixth aspect, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the above-described power terminal retransmission strategy model training method or the steps of the above-described power terminal retransmission strategy determination method.

[0026] Compared with the prior art, the present invention has the following beneficial effects: This invention presents a training method for a power terminal retransmission strategy model. In terms of privacy protection and data utilization, it employs a federated learning framework. Each power terminal only needs to upload its locally trained model parameters to the central server, without sharing original power business data. This fundamentally ensures the independence and confidentiality of power grid data from different regions, meeting the stringent data security requirements of the power industry. Simultaneously, by distributively utilizing the local data of each power terminal to train the retransmission strategy model, it overcomes the limitations of data silos, maximizing the extraction and utilization of effective data value within the power system. Regarding model training efficiency, each terminal can perform targeted training based on the characteristics of its complex local electromagnetic environment. This not only significantly reduces the communication overhead caused by relying solely on centralized training from the central server but also effectively reduces the training burden on individual nodes and improves training quality through distributed computing. Finally, the model parameters of the locally trained retransmission strategy model sent by each power terminal are sent to the central server. An attention-based aggregation method is used to aggregate these parameters, fully considering the quality of model parameters from different power terminals, effectively accelerating the convergence speed of the global retransmission strategy model and improving training efficiency. Attached Figure Description

[0027] Figure 1 This is a flowchart of the update steps in an embodiment of the present invention.

[0028] Figure 2 This is a flowchart of the power terminal retransmission strategy determination method according to an embodiment of the present invention.

[0029] Figure 3 This is a block diagram of the power terminal retransmission strategy model training system according to an embodiment of the present invention.

[0030] Figure 4 This is a block diagram of the power terminal retransmission strategy determination system according to an embodiment of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] The present invention will now be described in further detail with reference to the accompanying drawings: See Figure 1 In one embodiment of the present invention, a training method for a power terminal retransmission strategy model is provided, which can effectively improve the training quality and efficiency of the retransmission strategy model, thereby supporting more accurate determination of the retransmission strategy.

[0034] Specifically, the training method for the power terminal retransmission strategy model of the present invention includes the following steps: The iterative update process continues until the preset iteration termination condition is met. The central server then uses the current global retransmission strategy model as the training retransmission strategy model and sends it to each power terminal.

[0035] The update step includes: S1: The central server sends the current global retransmission strategy model to each power terminal.

[0036] S2: The power terminal trains the current global retransmission strategy model based on local data, obtains the locally trained retransmission strategy model, and sends the model parameters of the locally trained retransmission strategy model to the central server.

[0037] S3: The central server updates the current global retransmission strategy model based on the model parameters of the locally trained retransmission strategy model sent by each power terminal, using an aggregation method based on an attention mechanism.

[0038] This invention presents a training method for a power terminal retransmission strategy model. In terms of privacy protection and data utilization, it employs a federated learning framework. Each power terminal only needs to upload its locally trained model parameters to the central server, without sharing original power business data. This fundamentally ensures the independence and confidentiality of power grid data from different regions, meeting the stringent data security requirements of the power industry. Simultaneously, by distributively utilizing the local data of each power terminal to train the retransmission strategy model, it overcomes the limitations of data silos, maximizing the extraction and utilization of effective data value within the power system. Regarding model training efficiency, each terminal can perform targeted training based on the characteristics of its complex local electromagnetic environment. This not only significantly reduces the communication overhead caused by relying solely on centralized training from the central server but also effectively reduces the training burden on individual nodes and improves training quality through distributed computing. Finally, the model parameters of the locally trained retransmission strategy model sent by each power terminal are sent to the central server. An attention-based aggregation method is used to aggregate these parameters, fully considering the quality of model parameters from different power terminals, effectively accelerating the convergence speed of the global retransmission strategy model and improving training efficiency.

[0039] Explanatory purposes, the power terminal in this invention generally refers to a power service terminal or an MEC (Multi-access Edge Computing) terminal that connects multiple power service terminals. An MEC terminal can be understood as a small cloud computing center deployed at the network edge.

[0040] Specifically, MEC is deployed in power scenarios to serve local power service terminals. It collects communication information from these terminals within its service area for local retransmission strategy training, and finally uploads the information to a central server for aggregation. When distributing the trained retransmission strategy model, the central server can distribute the model to the MEC terminal, which in turn distributes it to the power service terminals within its range.

[0041] The core idea of ​​this invention's power terminal retransmission strategy model training method is to employ a federated learning algorithm to maximize the use of effective data in the power system while ensuring the privacy and security of power business data. The retransmission strategy model is trained locally on the power business terminal or MEC using a training algorithm. The model is then aggregated in the cloud using a federated aggregation algorithm based on an attention mechanism. Finally, the aggregated global retransmission strategy model is distributed to the power business terminal or MEC, improving the utilization rate of effective data while reducing the local training cost of the power business terminal or MEC. Specifically, considering the privacy and security issues of power business terminal users, federated learning (FL) collaboratively trains the global model using local data from distributed power business terminals or MECs. Compared to centralized training methods, it can significantly reduce user privacy risks and substantially reduce communication costs.

[0042] In one possible implementation, the global retransmission strategy model is constructed based on a reinforcement learning model.

[0043] Specifically, a global retransmission policy model is initialized on the central server, including the model's initial parameters. Here, a reinforcement learning model based on a deep learning neural network is considered, which can learn the optimal retransmission policy through the interaction between the agent and the environment. Therefore, the global retransmission policy model is initialized as follows: This includes the parameters of the neural network. , used to represent the weights and biases of each neuron.

[0044] For a power terminal, it collects local power business communication data from its service area and performs in-depth reinforcement training locally to obtain a local training retransmission strategy model. This local training retransmission strategy model determines which retransmission strategy the power terminal should choose in the current environment.

[0045] For a global retransmission strategy model built on a reinforcement learning model, when performing local training at the power terminal, the state space and action space for local training must first be defined.

[0046] In one possible implementation, the state space of the global retransmission strategy model is: downlink channel block error rate, average downlink channel quality index, distance between transceiver devices, and number of resource blocks.

[0047] Explained, for state space, the processing delay of the receive decoder has a significant impact on the transmission round-trip time (RTT). (Return) The more errors there are, the more data errors occur during transmission, leading to an increase in the number of retransmissions required. This further exacerbates the latency accumulation at the receiving decoding end. Therefore, reducing the downlink channel block error rate is crucial. It helps improve the reliability of information transmission and reduce delays.

[0048] in, The calculation method is as follows:

[0049] in, The number of negative acknowledgments returned. This represents the number of confirmations returned.

[0050] In addition to considering the downlink channel block error rate, the design state space also needs to consider the state space design state space. In addition to this important indicator, the average downlink channel quality indicator was also considered. ), the distance between transceiver devices and the number of resource blocks ( For example, the distance between the transceiver devices ranges from 0 to 2000 m, and the number of resource blocks ranges from 0 to 120.

[0051] In one possible implementation, the action space of the global retransmission strategy model is: A0~A8; where A0 is the 5G HARQ transmission scheme, A1 is the time-domain delay strategy with a delay time of 1 time slot, A2 is the time-domain delay strategy with a delay time of 2 time slots, A3 is the retransmission strategy with a repetition count of 1, A4 is the retransmission strategy with a repetition count of 2, A5 is a hybrid strategy with a delay time of 1 time slot and a repetition count of 1, A6 is a hybrid strategy with a delay time of 1 time slot and a repetition count of 2, A7 is a hybrid strategy with a delay time of 1 time slot and a repetition count of 2, and A8 is a hybrid strategy with a delay time of 2 time slots and a repetition count of 2.

[0052] An interpretable, simple, and efficient action space design can reduce convergence difficulty and improve training speed. In this embodiment, the action space is designed as A0~A8, containing nine discrete program actions. The 5G HARQ (Hybrid Automatic Repeat Request) transmission scheme is a key mechanism that combines forward error correction and automatic repeat, employing soft combining technology to improve data transmission reliability and efficiency.

[0053] Explanatory: While more retransmissions generally lead to a higher success rate for packet transmission, the receiving end's confirmation... The waiting time for signal retransmission significantly impacts retransmission latency. Therefore, to conserve time and resources required for retransmission, this invention limits the number of transmissions to a maximum of three: one normal transmission plus two consecutive retransmissions. Properly selecting the designed optimized transmission scheme can avoid waste due to channel resource redundancy when channel quality is good, and avoid long delays and high bit error rates caused by repeated retransmissions when channel quality is poor. For example, when… In cases of poor performance, the probability of information transmission failure is high. In such situations, a time-domain delay strategy (delaying by one or two time slots) can be used to send information, thereby improving the success rate of transmission. This is because an appropriate delay transmission scheme can avoid transmission failures in cases of poor performance. The environment is such that there is a greater probability of lowering the average. This also frees up more limited channel resources. It's worth noting that while the time-domain delay strategy may seem like a temporary waste of time, it actually reduces the average number of retransmissions because it avoids interference. Therefore, it subsequently reduces the overall transmission time and average delay. When users are far apart, there is a high probability of information transmission failure. In this case, a retransmission strategy (retransmitting 2 or 3 times) can be used to send information to improve the success rate of information transmission and reduce the average transmission delay. Because a proper retransmission mechanism can save on receiver acknowledgments... The waiting time required for the original retransmission after the signal is received is reduced, thereby lowering the average waiting time. It is worth noting that while using a continuous retransmission mechanism temporarily borrows channel resources, the increased transmission success rate significantly reduces the number of retransmissions required, thus compensating for the lost channel resources. Furthermore, when the distance is long and... When both are poor, a hybrid strategy can be chosen, which combines time-domain delay strategy and repetitive transmission strategy. The selection of a transmission scheme should not focus on a single parameter, but requires comprehensive consideration. The distance between the transceiver devices and Factors such as these.

[0054] In one possible implementation, the reward function of the global retransmission strategy model is: when ≤ hour:

[0055] in, The downlink channel block error rate for the 5G HARQ transmission scheme. The preset downlink channel block error rate threshold. This is the reward function.

[0056] when > hour:

[0057]

[0058]

[0059]

[0060]

[0061]

[0062] in, For the basic reward function, The downlink channel block error rate for the current retransmission strategy. This is the reward coefficient; To enhance the reward function, This is the reward function for the average downlink channel quality index. The distance reward function between the transmitting and receiving devices. The reward function is the number of resource blocks. This is the average downlink channel quality metric. The distance between the transceiver devices. This represents the number of resource blocks.

[0063] Interpretive design of reward functions enables the model to select those that can reduce average values ​​across various transmission environments. The optimal solution, when transmission does not meet the requirements If required, use A1~A8, and if the requirements are met... If required, the original A0 should be used. Therefore, the maximum tolerance can be initially set. Threshold, denoted as It can be adjusted as needed. In this implementation method, the simulated environment... Threshold set to =0.098. A0's The parameter is denoted as A1~A8 The parameter is denoted as The design of the reward function is mainly based on Is it greater than .

[0064] when Value not greater than In this case, two reward rules can be selected in the reward function, set as follows: (The following conditions must be met) The required state is ≤ The corresponding reward function is:

[0065] In this situation, choosing A0 will result in a positive reward, while choosing A1 to A8 will result in a penalty.

[0066] In dissatisfaction The required state, i.e. > The corresponding reward function is:

[0067] in, Basic rewards, To enhance rewards, if the conditions are not met... Choosing A0 under the threshold condition will result in a penalty, while choosing A1~A8 will be penalized according to... Key factors contributing to change will be subject to appropriate rewards or penalties. Among these, those reflecting... Reduced basic reward It can be represented as:

[0068] in, The reward coefficient is set to [value] in this embodiment. Retransmission strategies A1~A8 The smaller the value, the higher the corresponding base reward. The larger.

[0069] Enhancement Rewards The definition is as follows:

[0070] in, This is the reward function for the average downlink channel quality index. The distance reward function between the transmitting and receiving devices. The resource block quantity reward function consists of reward coefficients and their normalization system:

[0071]

[0072]

[0073] Specifically, for Factor, when choosing a time-domain delay strategy, The worse the performance, the greater the reward, and when the repeated transmission strategy is chosen, The better the performance, the greater the reward. However, the opposite is true for the distance between the transmitting and receiving devices and the number of resource blocks. The greater the distance between the transmitting and receiving devices and the number of resource blocks, the greater the reward for repeated transmissions; conversely, the smaller the distance between the transmitting and receiving devices and the number of resource blocks, the greater the reward for delayed transmissions.

[0074] In one possible implementation, when the power terminal trains the current global retransmission strategy model based on local data, it uses the Q-Learning method for training and employs a classical neural network to fit the current Q value in the Q-Learning method.

[0075] Interpretively, during local training, the powerful learning capabilities of deep reinforcement learning are leveraged to obtain better retransmission schemes. In Q-Learning, the interaction between the agent and the environment is determined by the update process of the Q-Table, which basically adopts the TD Bellman equation, uses the sequential differencing method to set the reward function mechanism, and obtains the optimal policy by updating the TD Bellman equation. The update of the TD Bellman equation is as follows:

[0076] in, For learning rate, For immediate gains, This refers to the discount rate (or attenuation rate). For memory efficiency, represent the next state. The maximum Q value under all actions.

[0077] Specifically The larger the value, the less the effect of previous training is retained. The larger the value, The greater the role of past experience, the more importance is placed on it. Conversely, The smaller the size, the more focused they are on immediate gains. The agent's goal is to select the next state. To maximize the reward after an action, so that you can be in the state next time. At that time, it can continue to receive rewards by choosing the correct action.

[0078] Meanwhile, since the state space is a continuous state space, this embodiment uses a classical neural network to fit the state. The local model parameters obtained from local training are then uploaded to the central server for aggregation.

[0079] In one possible implementation, the central server updates the current global retransmission strategy model based on the model parameters of the locally trained retransmission strategy model sent by each power terminal, using an attention-based aggregation method. This includes obtaining the aggregation weights of each power terminal using the following formula:

[0080] in, For power terminals No. The aggregation weight for each aggregation period. For viewing window Internal power terminal The total number of requests for all content uploaded. To observe the size of the window, This refers to the number of power terminals.

[0081] Update the current global retransmission strategy model using the following formula:

[0082] in, For the first The model parameters of the global retransmission strategy model after aggregation in each aggregation cycle. For power terminals No. Model parameters of the local training retransmission strategy model for each aggregation cycle.

[0083] Explanatoryly, given the differences in computing power and cached data quality among various power terminals, it is unreasonable to integrate the parameters of each edge model on an average basis according to the classic FL framework.

[0084] In this implementation, an attention-weighted aggregation mechanism is introduced, using the activity level index of each power terminal to measure the contribution of the edge model to the global model. This allows for the assignment of different weight values ​​to the model parameters of the locally trained retransmission strategy model sent by each power terminal, as detailed below:

[0085] Then, the global model is aggregated using the following formula:

[0086] Finally, after all the power terminals upload the model parameters of their locally trained retransmission strategy models to the central server and then perform global aggregation, the resulting global retransmission strategy model parameters are distributed to the power terminals to update the model parameters of the current locally trained retransmission strategy models in the power terminals.

[0087] See Figure 2 In another embodiment of the present invention, a method for determining a retransmission strategy for a power terminal is provided, which can effectively determine a suitable retransmission strategy based on the current state of the power terminal to improve communication quality.

[0088] Specifically, the method for determining the power terminal retransmission strategy includes the following steps: S11: The power terminal obtains the downlink channel block error rate, average downlink channel quality index, distance between transceiver devices, and number of resource blocks for the current time period, and obtains the communication status data for the current time period.

[0089] S12: The power terminal obtains the retransmission strategy for the next time period by training the retransmission strategy model based on the communication status data of the current time period.

[0090] The training retransmission strategy model is obtained based on the above-mentioned power terminal retransmission strategy model training method.

[0091] The method of this invention can, under the premise of strictly protecting data privacy, enable each power terminal to dynamically select the optimal retransmission strategy that adapts to its local complex electromagnetic environment through distributed collaborative learning of federated learning, thereby significantly improving the data transmission success rate and reducing end-to-end latency and network overhead.

[0092] The following are embodiments of the apparatus of the present invention, which can be used to execute embodiments of the method of the present invention. For details not disclosed in the apparatus embodiments, please refer to the embodiments of the method of the present invention.

[0093] See Figure 3 In another embodiment of the present invention, a power terminal retransmission strategy model training system is provided, which can be used to implement the above-mentioned power terminal retransmission strategy model training method. Specifically, the power terminal retransmission strategy model training system includes a control module, a central server and several power terminals.

[0094] The control module is used to control the iterative update steps of the central server and power terminals until the preset iteration termination condition is reached, and to control the central server to send the current global retransmission strategy model as the training retransmission strategy model to each power terminal.

[0095] The update steps include: the central server sending the current global retransmission strategy model to each power terminal; the power terminal training the current global retransmission strategy model based on local data to obtain a locally trained retransmission strategy model and sending the model parameters of the locally trained retransmission strategy model to the central server; the central server updating the current global retransmission strategy model based on the model parameters of the locally trained retransmission strategy model sent by each power terminal using an attention-based aggregation method.

[0096] In one possible implementation, the global retransmission strategy model is constructed based on a reinforcement learning model; the state space of the global retransmission strategy model is: downlink channel block error rate, average downlink channel quality index, distance between transceiver devices, and number of resource blocks; the action space of the global retransmission strategy model is: A0~A8; where A0 is a 5GHARQ transmission scheme, A1 is a time-domain delay strategy with a delay time of 1 time slot, A2 is a time-domain delay strategy with a delay time of 2 time slots, A3 is a retransmission strategy with a repetition count of 1, A4 is a retransmission strategy with a repetition count of 2, A5 is a hybrid strategy with a delay time of 1 time slot and a repetition count of 1, A6 is a hybrid strategy with a delay time of 1 time slot and a repetition count of 2, A7 is a hybrid strategy with a delay time of 1 time slot and a repetition count of 2, and A8 is a hybrid strategy with a delay time of 2 time slots and a repetition count of 2.

[0097] In one possible implementation, the reward function of the global retransmission strategy model is: when ≤ hour:

[0098] in, The downlink channel block error rate for the 5G HARQ transmission scheme. The preset downlink channel block error rate threshold. This is the reward function.

[0099] when > hour:

[0100]

[0101]

[0102]

[0103]

[0104]

[0105] in, For the basic reward function, The downlink channel block error rate for the current retransmission strategy. This is the reward coefficient; To enhance the reward function, This is the reward function for the average downlink channel quality index. The distance reward function between the transmitting and receiving devices. The reward function is the number of resource blocks. This is the average downlink channel quality metric. The distance between the transceiver devices. This represents the number of resource blocks.

[0106] In one possible implementation, when the power terminal trains the current global retransmission strategy model based on local data, it uses the Q-Learning method for training and employs a classical neural network to fit the current Q value in the Q-Learning method.

[0107] In one possible implementation, the central server updates the current global retransmission strategy model based on the model parameters of the locally trained retransmission strategy model sent by each power terminal, using an attention-based aggregation method. This includes obtaining the aggregation weights of each power terminal using the following formula:

[0108] in, For power terminals No. The aggregation weight for each aggregation period. For viewing window Internal power terminal The total number of requests for all content uploaded. To observe the size of the window, This refers to the number of power terminals.

[0109] Update the current global retransmission strategy model using the following formula:

[0110] in, For the first The model parameters of the global retransmission strategy model after aggregation in each aggregation cycle. For power terminals No. Model parameters of the local training retransmission strategy model for each aggregation cycle.

[0111] All relevant content of each step involved in the aforementioned embodiments of the power terminal retransmission strategy model training method can be referenced from the functional description of the corresponding functional module of the power terminal retransmission strategy model training system in the embodiments of the present invention, and will not be repeated here.

[0112] See Figure 4 In another embodiment of the present invention, a power terminal retransmission strategy determination system is provided, which can be used to implement the above-mentioned power terminal retransmission strategy determination method. Specifically, the power terminal retransmission strategy determination system includes a data acquisition module and a strategy determination module.

[0113] The data acquisition module is used by the power terminal to acquire the downlink channel block error rate, average downlink channel quality index, distance between transceiver devices, and number of resource blocks for the current time period, thereby obtaining the communication status data for the current time period. The strategy determination module is used by the power terminal to obtain the retransmission strategy for the next time period based on the communication status data for the current time period by training a retransmission strategy model. The retransmission strategy model is obtained based on the above-mentioned power terminal retransmission strategy model training method.

[0114] All relevant content of each step involved in the aforementioned embodiments of the power terminal retransmission strategy determination method can be referenced to the functional description of the corresponding functional module of the power terminal retransmission strategy determination system in the embodiments of the present invention, and will not be repeated here.

[0115] The module division in this embodiment of the invention is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0116] In another embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions from the computer storage medium to achieve corresponding method flows or corresponding functions. The processor described in this embodiment of the present invention can be used in the operation of a power terminal retransmission strategy model training method or a power terminal retransmission strategy determination method.

[0117] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the power terminal retransmission strategy model training method or power terminal retransmission strategy determination method in the above embodiments.

[0118] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0119] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0120] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0121] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A training method for a power terminal retransmission strategy model, characterized in that, include: The iterative update steps continue until the preset iteration termination condition is met. The central server then uses the current global retransmission strategy model as the training retransmission strategy model and sends it to each power terminal. The update step includes: The central server sends the current global retransmission strategy model to each power terminal. The power terminal trains the current global retransmission strategy model based on local data, obtains the locally trained retransmission strategy model, and sends the model parameters of the locally trained retransmission strategy model to the central server. The central server updates the current global retransmission strategy model using an attention-based aggregation method, based on the model parameters of the locally trained retransmission strategy model sent by each power terminal.

2. The training method for the power terminal retransmission strategy model according to claim 1, characterized in that, The global retransmission strategy model is constructed based on a reinforcement learning model; The state space of the global retransmission strategy model is: downlink channel block error rate, average downlink channel quality index, distance between transceiver devices, and number of resource blocks. The action space of the global retransmission strategy model is: A0~A8; where A0 is the 5G HARQ transmission scheme, A1 is the time-domain delay strategy with a delay time of 1 time slot, A2 is the time-domain delay strategy with a delay time of 2 time slots, A3 is the retransmission strategy with a repetition count of 1, A4 is the retransmission strategy with a repetition count of 2, A5 is the hybrid strategy with a delay time of 1 time slot and a repetition count of 1, A6 is the hybrid strategy with a delay time of 1 time slot and a repetition count of 2, A7 is the hybrid strategy with a delay time of 1 time slot and a repetition count of 2, and A8 is the hybrid strategy with a delay time of 2 time slots and a repetition count of 2.

3. The training method for the power terminal retransmission strategy model according to claim 2, characterized in that, The reward function of the global retransmission strategy model is: when ≤ hour: in, The downlink channel block error rate for the 5G HARQ transmission scheme. The preset downlink channel block error rate threshold. For the reward function; when > hour: in, For the basic reward function, The downlink channel block error rate for the current retransmission strategy. This is the reward coefficient; To enhance the reward function, This is the reward function for the average downlink channel quality index. The distance reward function between the transmitting and receiving devices. The reward function is the number of resource blocks. This is the average downlink channel quality metric. The distance between transceiver devices This represents the number of resource blocks.

4. The training method for the power terminal retransmission strategy model according to claim 2, characterized in that, When the power terminal trains the current global retransmission strategy model based on local data, it uses the Q-Learning method for training and employs a classical neural network to fit the current Q value in the Q-Learning method.

5. The training method for the power terminal retransmission strategy model according to claim 1, characterized in that, The central server updates the current global retransmission strategy model based on the model parameters of the locally trained retransmission strategy model sent by each power terminal, using an attention-based aggregation method, including: The aggregate weight of each power terminal is obtained by the following formula: in, For power terminals No. The aggregation weight for each aggregation period. For viewing window Internal power terminal The total number of requests for all content uploaded. To observe the size of the window, The number of power terminals; Update the current global retransmission strategy model using the following formula: in, For the first The model parameters of the global retransmission strategy model after aggregation in each aggregation cycle. For power terminals No. Model parameters of the local training retransmission strategy model for each aggregation cycle.

6. A method for determining a power terminal retransmission strategy, characterized in that, include: The power terminal obtains the downlink channel block error rate, average downlink channel quality index, distance between transceiver devices, and number of resource blocks for the current time period to obtain the communication status data for the current time period. The power terminal obtains the retransmission strategy for the next time period by training the retransmission strategy model based on the communication status data of the current time period. The training retransmission strategy model is obtained based on the power terminal retransmission strategy model training method according to any one of claims 1 to 5.

7. A training system for a power terminal retransmission strategy model, characterized in that, It includes a control module, a central server, and several power terminals; The control module is used to control the iterative update steps of the central server and power terminals until the preset iteration termination condition is reached, and to control the central server to use the current global retransmission strategy model as the training retransmission strategy model and send it to each power terminal. The update step includes: The central server sends the current global retransmission strategy model to each power terminal. The power terminal trains the current global retransmission strategy model based on local data, obtains the locally trained retransmission strategy model, and sends the model parameters of the locally trained retransmission strategy model to the central server. The central server updates the current global retransmission strategy model using an attention-based aggregation method, based on the model parameters of the locally trained retransmission strategy model sent by each power terminal.

8. A power terminal retransmission strategy determination system, characterized in that, include: The data acquisition module is used by the power terminal to acquire the downlink channel block error rate, average downlink channel quality index, distance between transceiver devices and the number of resource blocks for the current time period, so as to obtain the communication status data for the current time period. The strategy determination module is used by the power terminal to obtain the retransmission strategy for the next time period by training the retransmission strategy model based on the communication status data of the current time period. The training retransmission strategy model is obtained based on the power terminal retransmission strategy model training method according to any one of claims 1 to 5.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the power terminal retransmission strategy model training method as described in any one of claims 1 to 5, or the steps of the power terminal retransmission strategy determination method as described in claim 6.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the power terminal retransmission strategy model training method as described in any one of claims 1 to 5, or the steps of the power terminal retransmission strategy determination method as described in claim 6.

Images