Sample Determining Method for Wireless AI Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current reinforcement learning for AI neural network models in wireless communication systems is limited by the separation of inference and training functions across different devices, which restricts the ability to perform online training due to computing capability constraints.
Innovation Solution
A method and apparatus that enable the determination of training samples by receiving time-varying parameter information and AI model attribute information, allowing for the training of a target inference model using a network environment, thereby supporting reinforcement learning even when inference and training devices are separated.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If inference and training functions are deployed on different devices, then device computing capability constraints are reduced, but reinforcement learning training cannot be performed
Solution Approach 1:
The patent segments the reinforcement learning training process into two parts: the inference model runs on the terminal device while the training sample generation and policy gradient computation are performed on the network side device. This segmentation allows each component to operate on devices with appropriate computing capabilities, enabling reinforcement learning training to function despite the separation of inference and training devices.
Solution Approach 2:
The patent introduces an intermediary mechanism where the network side device acts as a mediator to generate training samples and compute policy gradients based on feedback from the terminal's inference results. This intermediary approach enables the distributed system to perform reinforcement learning training by having the network side device coordinate the training process using information from the terminal device.
2Ease of operation
If training samples are generated locally, then training can be performed independently, but device computing capabilities are insufficient
Solution Approach 1:
The patent divides the training sample generation task between the terminal device (which provides inference results and feedback) and the network side device (which generates comprehensive training samples including state, action, reward, and next state information). This segmentation enables training sample generation that exceeds the computing capabilities of either device alone while maintaining operational independence through coordinated interaction.
3Measurement precision
If inference model is updated frequently, then communication behavior prediction accuracy improves, but training time and computational resources increase
Solution Approach 1:
The patent implements periodic training updates where the inference model is trained and updated at specific intervals rather than continuously. Training samples are collected over multiple communication behaviors and updated periodically, allowing the system to maintain good prediction accuracy while avoiding the continuous computational overhead of frequent training updates.
Solution Approach 2:
The patent performs preliminary collection of training samples from multiple communication behaviors before initiating model updates. By accumulating sufficient training data in advance and then performing batch updates, the system achieves good prediction accuracy without requiring continuous training computation, thus reducing overall training time and computational resource consumption.
Data Source
AI summary
This application discloses a sample determining method and apparatus, and a device. The sample determining method includes: receiving, by a first device, first information sent by a target device; and determining, by the first device based on an inference result of a target inference model and the time-varying parameter information, a training sample used for training the target inference model. The first information includes time-varying parameter information and Artificial Intelligence (AI) model attribute information. The time-varying parameter information is used for indicating randomness of a communication behavior. The target inference model is an inference model corresponding to the AI model attribute information.