A MIMO mobile phone antenna adaptive tuning method based on reinforcement learning

Abstract

The application discloses a kind of MIMO mobile phone antenna adaptive tuning methods based on reinforcement learning, comprising the following steps: acquisition operating parameter;Operating parameter set is fused and encoded, and the characteristic vector containing historical state and time series information is generated;According to operating parameter set, the action range boundary is calculated, and the out-of-bound detection rule and rollback trigger condition are generated;Characteristic vector and action range boundary are input into improved diffusion strategy generation model, and tuning action is generated using hierarchical policy network;According to the out-of-bound detection rule, detection is carried out and rollback is triggered when out-of-bound, to obtain stable execution state;Stability index is calculated and diversity-stability collaborative reward function is constructed, and hierarchical policy network is updated;Scene label is generated, key-value pair index is formed and stored, and lightweight strategy is constructed and solidified through network distillation method.The application realizes the adaptive tuning of MIMO mobile phone antenna.

Description

Technical Field

[0001] This invention relates to the fields of mobile communication and radio frequency signal processing technology, and in particular to a MIMO mobile phone antenna adaptive tuning method based on reinforcement learning. Background Technology

[0002] With the development of fifth-generation and future mobile communication systems, MIMO (Multiple-Input Multiple-Output) mobile phone antennas are widely used in terminal devices, and their performance directly affects communication capacity, signal coverage, and energy efficiency. Existing MIMO antenna tuning methods mostly employ fixed-parameter design or adaptive algorithms based on simple feedback control. While these methods can achieve parameter correction in some scenarios, they still have the following problems in dynamic and complex environments:

[0003] The acquired antenna port amplitude, phase, and transmission relationship parameters are easily affected by user attitude, environmental noise, and multipath effects, resulting in large parameter fluctuations and low signal-to-noise ratio, thus leading to insufficient tuning accuracy. Existing motion control strategies mostly rely on single boundary settings or rule-based detection mechanisms, which can easily cause tuning failures or prolonged unstable states. In the process of generating tuning strategies, traditional algorithms based on empirical formulas or shallow models cannot effectively integrate historical states and time-series information, resulting in a lack of global optimization capabilities for tuning actions in complex dynamic scenarios. In terms of model updates, existing methods often only consider a single performance index, causing instability in the training process and making it difficult to continuously optimize the policy network. In addition, existing tuning methods lack lightweight policy storage and fast recall mechanisms for multiple scenario conditions, resulting in high computational and storage overhead when running on terminal devices, making it difficult to meet the requirements of real-time performance and low power consumption.

[0004] Therefore, how to provide a reinforcement learning-based adaptive tuning method for MIMO mobile phone antennas is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose an adaptive tuning method for MIMO mobile phone antennas based on reinforcement learning. This invention fully utilizes the improved diffusion strategy generation model and hierarchical strategy network in reinforcement learning, combined with working parameter acquisition, action range boundary constraints, out-of-bounds detection and rollback mechanism, and describes in detail the steps and algorithms for achieving adaptive tuning of mobile phone antennas. It has the advantages of high tuning accuracy, strong dynamic adaptability and high operating efficiency.

[0006] An adaptive tuning method for a MIMO mobile phone antenna based on reinforcement learning according to an embodiment of the present invention includes the following steps:

[0007] Collect the operating parameters of the MIMO mobile phone antenna during operation and form a set of operating parameters with timestamps;

[0008] The set of working parameters is fused and encoded to generate a feature vector containing historical state and time series information;

[0009] Calculate the action range boundary based on the set of working parameters, and generate out-of-bounds detection rules and rollback trigger conditions;

[0010] The feature vector and action range boundary are input into the improved diffusion strategy generation model. The hierarchical strategy network is used to embed the action range boundary during the diffusion sampling process. The high-level strategy network generates the coarse-tuning target, and the low-level strategy network generates the fine-tuning action based on the coarse-tuning target, thus forming the tuning action.

[0011] The tuning action is sent to the tuning circuit and detected according to the boundary detection rules. When the detection result is within the boundary of the action range, the execution continues. When the detection result exceeds the boundary of the action range, the circuit is rolled back to the most recent valid tuning action according to the rollback trigger condition to obtain a stable execution state.

[0012] In a stable execution state, the set of working parameters is collected again, the stability index is calculated, and a diversity-stability collaborative reward function based on the stability index is constructed. The reward is calculated and the hierarchical policy network is updated using the reward results.

[0013] The updated hierarchical policy network is combined with scene conditions to generate scene labels. The scene labels are then indexed and stored as key-value pairs with the optimal tuning action to form a lightweight policy, which is then solidified through network distillation.

[0014] Optionally, the operating parameters specifically include port amplitude, port phase, port transmission relationship, port reflection, radiation direction, and ambient noise. The operating parameters are combined to form an operating parameter set, and a timestamp is added to each data item in the operating parameter set to obtain an operating parameter set with timestamps.

[0015] Optionally, the step of fusing and encoding the set of working parameters to generate a feature vector containing historical state and time series information specifically includes:

[0016] The set of working parameters with timestamps is arranged in chronological order and normalized to obtain a normalized sequence.

[0017] The normalized sequence is encoded. Discrete data is encoded using one-hot encoding, while continuous data is encoded using an affine transformation method that multiplies the weight matrix with the input vector and adds a bias term.

[0018] The encoded normalized sequence is input into the feedforward neural network, and the fused encoded representation is output.

[0019] Timestamp sequence information is added to the fused encoded representation, and a time series representation is formed using positional encoding.

[0020] The time series representation is combined with the feature vector of the previous period, and the combination method is a recurrent neural network structure.

[0021] The output is a feature vector containing historical state information and time series information.

[0022] Optionally, the step of calculating the action range boundary based on the set of working parameters and generating boundary detection rules and rollback trigger conditions specifically includes:

[0023] The set of working parameters is statistically processed within a sliding time window to calculate the weighted mean and weighted extreme values ​​of various parameters, thus forming dynamic initial boundary parameters;

[0024] A covariance matrix is ​​constructed based on the dynamic initial boundary parameters, and the correlation interval between the parameters is calculated using the covariance matrix to obtain the multidimensional joint boundary.

[0025] Based on the multidimensional joint boundary, a confidence threshold is introduced to set the value range of various parameters at the confidence level, thus forming the action range boundary.

[0026] The boundary detection conditions are set according to the action range boundary. The boundary detection conditions are triggered when the parameter value of the tuning action exceeds the value range of the corresponding confidence threshold, thus forming the boundary detection rules.

[0027] Based on the boundary detection rules, rollback conditions are established. The rollback conditions are set so that when the boundary detection is triggered, the rollback will first be rolled back to the most recent tuning action that was completed within the action range boundary. When the tuning action is unavailable, the rollback will be rolled back to the average of all tuning actions that were executed within the action range boundary within the preset time window, thus forming the rollback trigger condition.

[0028] Optionally, the improved diffusion strategy generation model specifically includes:

[0029] The improved diffusion strategy generation model consists of a conditional diffusion branch, a boundary projection branch, a hierarchical coarse adjustment branch, a hierarchical fine adjustment branch, a candidate evaluation branch, and an action synthesis branch.

[0030] The conditional diffusion branch receives the feature vector and the action range boundary, concatenates the feature vector with the timestamp code to form the condition vector, performs noise addition and reverse denoising on the condition vector according to a preset number of steps, the noise intensity is adaptively adjusted according to the timestamp code and feature vector statistics, and sets an in-step correction unit in each denoising step to linearly correct the residual of the previous time step, and outputs an initial sampling representation containing the inter-step index and intermediate samples.

[0031] In each denoising step of the conditional diffusion branch, the boundary projection branch reads intermediate samples, performs feasible region projection item by item according to the amplitude, phase and switching constraints of the action range boundary, adopts the pruning order of amplitude priority and phase postponement, performs consistency verification on port-level coupling items and writes back the consistency flag, and generates a boundary consistent sample sequence.

[0032] The hierarchical coarse adjustment branch inputs the boundary-consistent sample sequence into the high layer of the hierarchical policy network, embeds the port coupling graph representation and frequency band label, and generates a coarse adjustment target set in the action space according to the grid step size table. The grid step size table is adaptively selected according to the width of the action range boundary. The source sample index and boundary consistency mark are recorded for each coarse adjustment target to obtain an indexed coarse adjustment target set.

[0033] The hierarchical fine-tuning branch inputs the indexed set of coarse-tuning targets into the lower layers of the hierarchical policy network. It performs secondary sampling or single-step gradient updates in the local neighborhood of each coarse-tuning target, and uses a port block coordinate update method and a step size decay table to perform linkage correction on the coupling sensitive ports. It outputs a set of fine-tuning action candidates and inherits the corresponding index and consistency flag.

[0034] The candidate evaluation branch combines the fine-tuning action candidate set with the feature vector to construct a dual-score vector of matching degree score and boundary consistency score. The matching degree threshold and consistency threshold are derived from the boundary constraint set and the sliding time window statistics. The most recent effective tuning action is introduced as a warm-start candidate and added to the set. The similarity matrix is ​​calculated for all candidates and deduplication is performed according to the similarity threshold to obtain a simplified candidate set.

[0035] The motion synthesis branch concatenates the simplified candidate set by port dimension, performs numerical normalization and gear mapping according to the action range boundary, generates a hierarchical output action set after completing the port order consistency check, and selects a single output tuning action from the hierarchical output action set.

[0036] Optionally, the step of sending the tuning action to the tuning circuit and detecting it according to the boundary detection rules, maintaining execution when the detection result is within the action range boundary, and rolling back to the most recent valid tuning action according to the rollback trigger condition when the detection result exceeds the action range boundary, to obtain a stable execution state, specifically includes:

[0037] The tuning action is converted into a control command and sent to the tuning circuit to obtain the tuning circuit execution data.

[0038] The data executed by the tuning circuit is judged item by item according to the out-of-bounds detection rules to obtain the detection results;

[0039] When the detection results show that all the data executed by the tuning circuit is within the boundary of the action range, the current tuning action is marked as a valid action and maintained to obtain a continuous execution state;

[0040] When the detection result indicates that the data executed by the tuning circuit exceeds the boundary of the action range, the rollback condition is triggered, and the most recently completed tuning action is called from the marked valid actions to obtain the rollback action.

[0041] The retrace action is converted into a control command and sent to the tuning circuit. The detection process is repeated until the detection result meets the action range boundary, and a stable execution state is obtained.

[0042] Optionally, the step of collecting the set of working parameters again under stable execution state, calculating stability index and constructing a diversity-stability collaborative reward function based on the stability index, calculating reward and using the reward result to update the hierarchical policy network specifically includes:

[0043] The operating data of the tuning circuit is collected under stable execution conditions to form a new set of operating parameters;

[0044] The stability index is calculated based on the new set of working parameters. A diversity-stability collaborative reward function is defined based on the stability index. The diversity-stability collaborative reward function consists of three parts: the first part is a stability reward term that is proportional to the stability index; the second part is a diversity reward term obtained by constructing a kernel matrix for the candidate set embedding vectors and calculating its logarithm; and the third part is an action smoothing constraint term obtained by averaging the differences in the tuning action vectors at consecutive time steps, which is introduced in the form of a penalty.

[0045] The reward value calculated by the diversity-stable collaborative reward function is input into the hierarchical policy network, and the network parameters are updated according to the reward value to form the updated hierarchical policy network.

[0046] Optionally, the step of combining the updated hierarchical policy network with scene conditions to generate scene labels, forming key-value pairs with the optimal tuning action and storing them to form a lightweight policy, and then solidifying it through network distillation, specifically includes:

[0047] Based on the updated hierarchical policy network, scene conditions are input, including user location, environmental noise, and channel state information. Scene feature representation is obtained by combining scene conditions, and scene labels are generated through classification mapping. The scene label is a unique identifier sequence.

[0048] The scene label is paired with the optimal tuning action output by the updated hierarchical policy network in that scene to form a key-value pair index. The key-value pair index uses the scene label as the key and the optimal tuning action as the value. All indexes are stored in a table or hash format to form a mapping structure that can be retrieved quickly.

[0049] A lightweight policy is constructed using an index set. The lightweight policy directly takes the scene label as input and outputs the optimal tuning action. During the construction process, compression and pruning are performed to remove redundant entries and retain only representative mapping relationships, resulting in a compact lightweight policy model.

[0050] The system employs a network distillation method for solidification. During the distillation process, the lightweight policy model is used as the teacher model, and the student network is used as the learning object. The system takes scene labels as input and outputs the corresponding optimal tuning actions. Through knowledge transfer, the student network learns the mapping relationship of the teacher model. After distillation, the student network is saved as the final lightweight policy model.

[0051] The beneficial effects of this invention are:

[0052] This invention introduces a reinforcement learning-based adaptive tuning method for MIMO mobile phone antennas, which significantly improves upon existing tuning methods that rely on fixed parameters or simple feedback mechanisms. First, at the data acquisition level, the set of working parameters covers port amplitude, phase, transmission relationship, port reflection, radiation direction, and environmental noise, and incorporates timestamps to ensure the integrity and temporal sequence of the collected data, thereby improving the reliability of parameter input. Second, at the action generation level, an improved diffusion strategy generation model combined with a hierarchical policy network is used to directly embed the action range boundary into the diffusion sampling process, generating coarse-tuning targets and fine-tuning actions in a layered manner, achieving adaptive tuning in complex dynamic environments. Third, at the tuning execution level, out-of-bounds detection rules and rollback trigger conditions are established. When the detected data exceeds the boundary, it can quickly roll back to the most recent valid tuning action, avoiding the system being in an unstable state for a long time. Furthermore, at the policy update level, a diversity-stability collaborative reward function based on stability indicators is designed, combining stability features, diversity preservation, and action smoothing constraints to effectively improve the robustness and convergence of policy training. Finally, at the lightweight application level, the updated hierarchical policy network is combined with scene conditions to generate scene labels, constructing a key-value pair index of scene labels and optimal tuning actions, and solidifying it through network distillation. This improves the operating efficiency on terminal devices while reducing storage and computational overhead. This invention enables high-precision, low-power, and real-time adaptive antenna tuning under multiple scene conditions. Attached Figure Description

[0053] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0054] Figure 1 This is a flowchart of a reinforcement learning-based adaptive tuning method for MIMO mobile phone antennas proposed in this invention.

[0055] Figure 2 This is a schematic diagram of a reinforcement learning-based adaptive tuning method for MIMO mobile phone antennas proposed in this invention.

[0056] Figure 3 This is a framework diagram of the improved diffusion strategy generation model in the MIMO mobile phone antenna adaptive tuning method based on reinforcement learning proposed in this invention. Detailed Implementation

[0057] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0058] refer to Figure 1-3 An adaptive tuning method for MIMO mobile phone antennas based on reinforcement learning includes the following steps:

[0059] Collect the operating parameters of the MIMO mobile phone antenna during operation and form a set of operating parameters with timestamps;

[0060] The set of working parameters is fused and encoded to generate a feature vector containing historical state and time series information;

[0061] Calculate the action range boundary based on the set of working parameters, and generate out-of-bounds detection rules and rollback trigger conditions;

[0062] The feature vector and action range boundary are input into the improved diffusion strategy generation model. The hierarchical strategy network is used to embed the action range boundary during the diffusion sampling process. The high-level strategy network generates the coarse-tuning target, and the low-level strategy network generates the fine-tuning action based on the coarse-tuning target, thus forming the tuning action.

[0063] The tuning action is sent to the tuning circuit and detected according to the boundary detection rules. When the detection result is within the boundary of the action range, the execution continues. When the detection result exceeds the boundary of the action range, the circuit is rolled back to the most recent valid tuning action according to the rollback trigger condition to obtain a stable execution state.

[0064] In a stable execution state, the set of working parameters is collected again, the stability index is calculated, and a diversity-stability collaborative reward function based on the stability index is constructed. The reward is calculated and the hierarchical policy network is updated using the reward results.

[0065] The updated hierarchical policy network is combined with scene conditions to generate scene labels. The scene labels are then indexed and stored as key-value pairs with the optimal tuning action to form a lightweight policy, which is then solidified through network distillation.

[0066] In this embodiment, the working parameters specifically include port amplitude, port phase, port transmission relationship, port reflection, radiation direction, and environmental noise. The working parameters are combined to form a working parameter set, and a timestamp is added to each data item in the working parameter set to obtain a working parameter set with timestamps.

[0067] In this embodiment, the step of fusing and encoding the set of working parameters to generate a feature vector containing historical state and time series information specifically includes:

[0068] The set of working parameters with timestamps is arranged in chronological order and normalized to obtain a normalized sequence.

[0069] The normalized sequence is encoded. Discrete data is encoded using one-hot encoding, while continuous data is encoded using an affine transformation method that multiplies the weight matrix with the input vector and adds a bias term.

[0070] The encoded normalized sequence is input into the feedforward neural network, and the fused encoded representation is output.

[0071] Timestamp sequence information is added to the fused encoded representation, and a time series representation is formed using positional encoding.

[0072] The time series representation is combined with the feature vector of the previous period, and the combination method is a recurrent neural network structure.

[0073] The output is a feature vector containing historical state information and time series information.

[0074] This implementation method generates feature vectors containing historical states by normalizing, encoding, and time-series modeling the set of working parameters, thereby achieving a fusion representation of multi-source data, ensuring temporal correlation, and effectively enhancing the adaptability of the tuned model to dynamic environments.

[0075] In this embodiment, the step of calculating the action range boundary based on the set of working parameters and generating out-of-bounds detection rules and rollback trigger conditions specifically includes:

[0076] The set of working parameters is statistically processed within a sliding time window to calculate the weighted mean and weighted extreme values ​​of various parameters, thus forming dynamic initial boundary parameters;

[0077] A covariance matrix is ​​constructed based on the dynamic initial boundary parameters, and the correlation interval between the parameters is calculated using the covariance matrix to obtain the multidimensional joint boundary.

[0078] Based on the multidimensional joint boundary, a confidence threshold is introduced to set the value range of various parameters at the confidence level, thus forming the action range boundary.

[0079] The boundary detection conditions are set according to the action range boundary. The boundary detection conditions are triggered when the parameter value of the tuning action exceeds the value range of the corresponding confidence threshold, thus forming the boundary detection rules.

[0080] Based on the boundary detection rules, rollback conditions are established. The rollback conditions are set so that when the boundary detection is triggered, the rollback will first be rolled back to the most recent tuning action that was completed within the action range boundary. When the tuning action is unavailable, the rollback will be rolled back to the average of all tuning actions that were executed within the action range boundary within the preset time window, thus forming the rollback trigger condition.

[0081] This implementation method calculates the boundary of the action range and sets out the boundary detection rules and rollback conditions, thereby achieving real-time constraints and error correction on the tuning action, ensuring the boundary controllability and operational safety during the tuning process, and avoiding entering abnormal states.

[0082] In this embodiment, the improved diffusion strategy generation model specifically includes:

[0083] The improved diffusion strategy generation model consists of a conditional diffusion branch, a boundary projection branch, a hierarchical coarse adjustment branch, a hierarchical fine adjustment branch, a candidate evaluation branch, and an action synthesis branch.

[0084] The conditional diffusion branch receives the feature vector and the action range boundary, concatenates the feature vector with the timestamp code to form the condition vector, performs noise addition and reverse denoising on the condition vector according to a preset number of steps, the noise intensity is adaptively adjusted according to the timestamp code and feature vector statistics, and sets an in-step correction unit in each denoising step to linearly correct the residual of the previous time step, and outputs an initial sampling representation containing the inter-step index and intermediate samples.

[0085] In each denoising step of the conditional diffusion branch, the boundary projection branch reads intermediate samples, performs feasible region projection item by item according to the amplitude, phase and switching constraints of the action range boundary, adopts the pruning order of amplitude priority and phase postponement, performs consistency verification on port-level coupling items and writes back the consistency flag, and generates a boundary consistent sample sequence.

[0086] The hierarchical coarse adjustment branch inputs the boundary-consistent sample sequence into the high layer of the hierarchical policy network, embeds the port coupling graph representation and frequency band label, and generates a coarse adjustment target set in the action space according to the grid step size table. The grid step size table is adaptively selected according to the width of the action range boundary. The source sample index and boundary consistency mark are recorded for each coarse adjustment target to obtain an indexed coarse adjustment target set.

[0087] The hierarchical fine-tuning branch inputs the indexed set of coarse-tuning targets into the lower layers of the hierarchical policy network. It performs secondary sampling or single-step gradient updates in the local neighborhood of each coarse-tuning target, and uses a port block coordinate update method and a step size decay table to perform linkage correction on the coupling sensitive ports. It outputs a set of fine-tuning action candidates and inherits the corresponding index and consistency flag.

[0088] The candidate evaluation branch combines the fine-tuning action candidate set with the feature vector to construct a dual-score vector of matching degree score and boundary consistency score. The matching degree threshold and consistency threshold are derived from the boundary constraint set and the sliding time window. The matching degree threshold is determined by statistically analyzing the similarity distribution between historical tuning actions and feature vectors within the sliding time window, taking the mean plus or minus the standard deviation to obtain the threshold range. The consistency threshold is determined by judging the execution results of historical actions based on the action range boundary, calculating the proportion that satisfies the boundary constraints within the window, and taking the lower percentile of the proportion as the threshold. The most recent effective tuning action is introduced as a warm-start candidate and added to the set. The similarity matrix is ​​calculated for all candidates and deduplication is performed according to the similarity threshold to obtain a simplified candidate set.

[0089] The motion synthesis branch concatenates the simplified candidate set by port dimension, performs numerical normalization and gear mapping according to the action range boundary, generates a hierarchical output action set after completing the port order consistency check, and selects a single output tuning action from the hierarchical output action set.

[0090] This implementation employs an improved diffusion strategy generation model, introducing conditional diffusion and boundary projection branches during the diffusion sampling process. Combined with a hierarchical strategy network, it generates tuning actions layer by layer, achieving a seamless connection between coarse and fine tuning. The high-level strategy network generates coarse-tuning targets with strong global coverage, ensuring the overall directionality of the actions. The low-level strategy network performs fine-tuning within the local neighborhood of the coarse-tuning target, effectively improving the precision and matching accuracy of the actions. The candidate evaluation and action synthesis branches further achieve redundancy removal and consistency correction for candidate actions, enabling the generated tuning actions to maintain diversity while possessing high stability and executability. This model structure overcomes the shortcomings of traditional shallow models, which can only perform local optimization and are prone to getting trapped in suboptimal solutions, significantly improving the adaptive tuning capability of MIMO mobile phone antennas in complex dynamic environments.

[0091] In this embodiment, the step of sending the tuning action to the tuning circuit and detecting it according to the boundary detection rules, maintaining execution when the detection result is within the action range boundary, and rolling back to the most recent valid tuning action according to the rollback trigger condition when the detection result exceeds the action range boundary, to obtain a stable execution state, specifically includes:

[0092] The tuning action is converted into a control command and sent to the tuning circuit to obtain the tuning circuit execution data.

[0093] The data executed by the tuning circuit is judged item by item according to the out-of-bounds detection rules to obtain the detection results;

[0094] When the detection results show that all the data executed by the tuning circuit is within the boundary of the action range, the current tuning action is marked as a valid action and maintained to obtain a continuous execution state;

[0095] When the detection result indicates that the data executed by the tuning circuit exceeds the boundary of the action range, the rollback condition is triggered, and the most recently completed tuning action is called from the marked valid actions to obtain the rollback action.

[0096] The retrace action is converted into a control command and sent to the tuning circuit. The detection process is repeated until the detection result meets the action range boundary, and a stable execution state is obtained.

[0097] This implementation method achieves rapid recovery in the event of tuning failure through tuning action execution, out-of-bounds detection, and rollback triggering mechanisms, ensuring that the antenna always operates in a stable state, effectively reducing communication performance fluctuations, and improving tuning reliability.

[0098] In this embodiment, the step of re-collecting the set of working parameters under stable execution conditions, calculating stability indices, constructing a diversity-stability collaborative reward function based on the stability indices, calculating rewards, and updating the hierarchical policy network using the reward results specifically includes:

[0099] The operating data of the tuning circuit is collected under stable execution conditions to form a new set of operating parameters;

[0100] Stability indices are calculated based on the new set of operating parameters. ,in Indicates at time The numerical values ​​obtained from the number of times the tuning action is executed and the success rate within the sliding time window;

[0101] Based on a stability index, a diversity-stability collaborative reward function is defined. This function consists of three parts: the first part is a stability reward term proportional to the stability index; the second part is a diversity reward term obtained by constructing a kernel matrix from the embedding vectors of the candidate set and calculating its logarithm; and the third part is an action smoothing constraint term obtained by averaging the differences in the tuning action vectors at consecutive time steps, introduced in the form of a penalty. The mathematical expression of the reward function formed by combining these three parts is as follows:

[0102] ;

[0103] in Indicates at time The reward value; , , , which are constants greater than zero, representing the stability weight, diversity weight, and motion smoothness weight, respectively; For at any time The tuning motion vector, The dimension representing the tuning action. Indicates at time The same dimension of tuning action value, For candidate set Constructed kernel matrix, The embedding vector representing the candidate action. To use a similarity kernel function with a cosine kernel, To and A dimensionally consistent identity matrix;

[0104] The reward value calculated by the diversity-stable collaborative reward function is input into the hierarchical policy network, and the network parameters are updated according to the reward value to form the updated hierarchical policy network.

[0105] This implementation calculates a stability index under stable execution conditions and constructs a diversity-stable collaborative reward function, thereby achieving dynamic updates of the policy network, ensuring robustness and convergence during training, and thus obtaining a better tuning policy.

[0106] In this embodiment, the step of combining the updated hierarchical policy network with scene conditions to generate scene labels, forming key-value pairs with the optimal tuning action and storing them to form a lightweight policy, and then solidifying it through network distillation, specifically includes:

[0107] Based on the updated hierarchical policy network, scene conditions are input, including user location, environmental noise, and channel state information. Scene feature representation is obtained by combining scene conditions, and scene labels are generated through classification mapping. The scene label is a unique identifier sequence.

[0108] The scene label is paired with the optimal tuning action output by the updated hierarchical policy network in that scene to form a key-value pair index. The key-value pair index uses the scene label as the key and the optimal tuning action as the value. All indexes are stored in a table or hash format to form a mapping structure that can be retrieved quickly.

[0109] A lightweight policy is constructed using an index set. The lightweight policy directly takes the scene label as input and outputs the optimal tuning action. During the construction process, compression and pruning are performed to remove redundant entries and retain only representative mapping relationships, resulting in a compact lightweight policy model.

[0110] The system employs a network distillation method for solidification. During the distillation process, the lightweight policy model is used as the teacher model, and the student network is used as the learning object. The system takes scene labels as input and outputs the corresponding optimal tuning actions. Through knowledge transfer, the student network learns the mapping relationship of the teacher model. After distillation, the student network is saved as the final lightweight policy model.

[0111] This implementation generates scene tags and constructs a key-value pair index with the optimal tuning action, and then uses network distillation to solidify lightweight strategies, thereby enabling the strategy model to be quickly invoked and run in multiple scenarios, reducing the computing and storage overhead of terminal devices.

[0112] Example 1:

[0113] To verify the feasibility of this invention in practice, it was applied to a dynamic communication environment tuning scenario for a MIMO mobile phone antenna. The experiment was conducted in a semi-open anechoic chamber, measuring 15 meters long, 10 meters wide, and 4 meters high. The environment included a variable attenuator, a controllable interference source, and a multipath reflector to simulate the complex scenarios in real mobile communication. The test equipment was a 5G smartphone prototype equipped with a four-antenna array, operating at 3.5 GHz and supporting 4×4 MIMO mode.

[0114] During testing, scenarios were simulated, including changes in user handheld posture, body obstruction, environmental noise interference, and rapid channel state switching. Under each scenario, the system first collected operating parameters such as port amplitude, port phase, port transmission relationship, port reflection, and radiation direction, and added environmental noise values ​​to form a timestamped set of operating parameters. Subsequently, the fusion coding and improved diffusion strategy generation model proposed in this invention worked together to generate tuning actions and send them to the tuning circuit to complete real-time adaptive antenna tuning.

[0115] Throughout the experiment, 30 different scenario conditions were set, each lasting 10 minutes. In the comparative experiments, traditional fixed-parameter tuning methods, a tuning method based on simple PID feedback, and the adaptive tuning method based on reinforcement learning proposed in this invention were used. Comparative metrics included channel capacity (Gbps), energy efficiency (bits / J), stability (effective tuning action ratio), bit error rate (BER), and average tuning latency (ms). Experimental data are shown in Table 1.

[0116] Table 1. Performance Comparison Data of Different Methods in Multiple Scenarios

[0117]

[0118] As shown in Table 1, the method of this invention outperforms both the traditional fixed-parameter method and the PID feedback method in all test scenarios. Regarding channel capacity, the method of this invention achieves an average improvement of approximately 41.2%, fully utilizing the spatial degrees of freedom of the multi-antenna array. In terms of energy efficiency, the method of this invention achieves an average of 188 bits / J, nearly 80% higher than the traditional fixed-parameter method, indicating effective energy consumption control while maintaining high capacity. Regarding stability, the effective tuning ratio of the method of this invention is consistently above 90%, significantly better than the comparative methods. Regarding the bit error rate, the method of this invention is controlled within the range of 2–3 × 10⁻³, while the traditional method is generally above 5 × 10⁻³, demonstrating the outstanding anti-interference performance of the method of this invention. Regarding the average tuning delay, the method of this invention is controlled at around 30 ms, approximately 40% shorter than the traditional method, ensuring real-time performance during communication.

[0119] Further statistical analysis of the overall average values ​​across 30 scenarios revealed that the fixed-parameter method performed poorly in terms of capacity, energy efficiency, and stability. While the PID feedback method showed improvement, it remained limited by a simple control strategy, resulting in insufficient tuning accuracy and response speed. This invention utilizes an improved diffusion strategy to generate a model and a hierarchical strategy network, achieving adaptive tuning in dynamic environments and maintaining stable communication under multi-source interference and complex channel conditions.

[0120] As can be seen from this embodiment, the reinforcement learning-based MIMO mobile phone antenna adaptive tuning method proposed in this invention can achieve high-precision, low-latency, and high-energy-efficiency antenna tuning in dynamic and complex environments. Compared with existing methods, it significantly improves capacity, stability, and anti-interference capabilities, ensuring the continuous reliability of terminal devices in 5G and future communication systems.

[0121] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for adaptive tuning of MIMO handset antennas based on reinforcement learning, characterized in that, The method comprises the following steps: Collecting working parameters of a MIMO mobile phone antenna during operation to form a working parameter set with a time stamp; Fusing and encoding the working parameter set to generate a feature vector containing historical state and time sequence information; Calculating the action range boundary from the working parameter set, and generating an out-of-boundary detection rule and a rollback trigger condition; Inputting the feature vector and the action range boundary into an improved diffusion strategy generation model, embedding the action range boundary in the diffusion sampling process using a hierarchical strategy network, generating a coarse adjustment target by the high-level strategy network, and generating a fine adjustment action by the low-level strategy network according to the coarse adjustment target to form a tuning action; Downlinking the tuning action to the tuning circuit, detecting according to the out-of-boundary detection rule, maintaining execution when the detection result is within the action range boundary, and rolling back to the last valid tuning action according to the rollback trigger condition when the detection result exceeds the action range boundary to obtain a stable execution state; In the stable execution state, the working parameter set is collected again, a stability index is calculated, a diversity-stability collaborative reward function based on the stability index is constructed, the reward is calculated, and the hierarchical strategy network is updated using the reward result; Combining the updated hierarchical strategy network with the scene condition to generate a scene label, combining the scene label with the optimal tuning action to form a key-value pair index and storing it to form a lightweight strategy and solidify it through network distillation.

2. The MIMO handset antenna adaptive tuning method based on reinforcement learning according to claim 1, characterized in that, The working parameters specifically include port amplitude, port phase, port transmission relationship, port reflection, radiation direction, and environmental noise, the working parameters are combined to form a working parameter set, and a time stamp is added to each data in the working parameter set to obtain a working parameter set with a time stamp.

3. The method of claim 1, wherein the method further comprises: The fusion encoding of the working parameter set to generate a feature vector containing historical state and time sequence information specifically includes: Arranging the working parameter set with a time stamp in chronological order, normalizing to obtain a normalized sequence; Encoding the normalized sequence, using one-hot encoding for discrete data and affine transformation by multiplying the weight matrix with the input vector and adding a bias term for continuous data; Inputting the encoded normalized sequence into a feedforward neural network to output a fused encoding representation; Adding time stamp order information to the fused encoding representation to form a time sequence representation using position encoding; Combining the time sequence representation with the feature vector of the previous period in a recurrent neural network structure; Outputting a feature vector containing historical state information and time sequence information.

4. The method of claim 1, wherein, The calculation of the action range boundary from the working parameter set, and the generation of the out-of-boundary detection rule and the rollback trigger condition specifically include: Statistically processing the working parameter set in a sliding time window to calculate the weighted mean and weighted extreme value of each type of parameter to form dynamic initial boundary parameters; Building a covariance matrix from the dynamic initial boundary parameters, calculating the correlation interval between parameters using the covariance matrix, and obtaining a multi-dimensional joint boundary; Introducing a confidence threshold based on the multi-dimensional joint boundary to set the value range of each type of parameter at the confidence level to form an action range boundary; The out-of-bound detection condition is triggered when a parameter value of the tuning action exceeds a value range corresponding to a confidence threshold, and an out-of-bound detection rule is formed; A rollback condition is established based on the out-of-bound detection rule, and the rollback condition is set to first rollback to a last tuning action that is executed within the action range boundary when the out-of-bound detection is triggered, and rollback to a mean value of all tuning actions that are executed within the action range boundary in a preset time window when the tuning action is unavailable, and a rollback trigger condition is formed.

5. The method of claim 1, wherein, The improved diffusion strategy generation model specifically comprises: The improved diffusion strategy generation model comprises a conditional diffusion branch, a boundary projection branch, a hierarchical coarse adjustment branch, a hierarchical fine adjustment branch, a candidate evaluation branch, and an action synthesis branch; The conditional diffusion branch receives a feature vector and an action range boundary, splices the feature vector and a time stamp to form a condition vector, performs noise addition and reverse denoising on the condition vector according to a preset step number, and adaptively adjusts the noise intensity according to the time stamp code and the feature vector statistics, sets a step correction unit at each denoising step to linearly correct the residual error at the last time, and outputs an initial sampling representation containing an inter-step index and an intermediate sample; The boundary projection branch reads the intermediate sample at each denoising step of the conditional diffusion branch, performs feasible region projection on each item according to the amplitude, phase and switch constraints of the action range boundary, adopts an amplitude-first and phase-posterior clipping order, performs consistency checking on the port-level coupling items and writes back a consistency flag, and generates a boundary-consistent sample sequence; The hierarchical coarse adjustment branch inputs the boundary-consistent sample sequence into the high layer of the hierarchical strategy network, embeds the port coupling graph representation and the frequency band label, generates a coarse adjustment target set on the action space according to a grid step table, adaptively selects the grid step table according to the width of the action range boundary, records the source sample index and the boundary consistency flag for each coarse adjustment target, and obtains an indexed coarse adjustment target set; The hierarchical fine adjustment branch inputs the indexed coarse adjustment target set into the low layer of the hierarchical strategy network, performs secondary sampling or single-step gradient update in the local neighborhood of each coarse adjustment target, adopts a port block coordinate update method and a step decay table to perform linkage correction on the coupling-sensitive ports, outputs a fine adjustment action candidate set and inherits the corresponding index and consistency flag; The candidate evaluation branch combines the fine adjustment action candidate set and the feature vector to construct a double-score vector of matching degree score and boundary consistency score, and the matching degree threshold and the consistency threshold are derived from the boundary constraint set and the sliding time window statistics, and the last effective tuning action is introduced as a warm-up candidate to join the set, a similarity matrix is calculated for all candidates, and a similarity threshold is used to perform deduplication, to obtain a refined candidate set; The action synthesis branch performs port dimension splicing on the refined candidate set, performs numerical normalization and gear mapping according to the action range boundary, and generates a hierarchical output action set after completing port order consistency checking, and selects a single output tuning action from the hierarchical output action set.

6. The method of claim 1, wherein, The tuning action is issued to the tuning circuit, and detection is performed according to the out-of-bound detection rule; when the detection result is within the action range boundary, execution is maintained; when the detection result exceeds the action range boundary, rollback is performed to the nearest valid tuning action according to the rollback trigger condition, so that a stable execution state is obtained, and the specific steps include: The tuning action is converted into a control instruction and issued to the tuning circuit to obtain tuning circuit execution data; The tuning circuit execution data is judged item by item according to the out-of-bound detection rule to obtain a detection result; When the detection result indicates that all the tuning circuit execution data are within the action range boundary, the current tuning action is marked as a valid action and maintained for execution, so that a continuous execution state is obtained; When the detection result indicates that the tuning circuit execution data exceed the action range boundary, a rollback condition is triggered, and the last executed tuning action is called from the marked valid actions to obtain a rollback action; The rollback action is converted into a control instruction and issued to the tuning circuit, and the detection process is repeated until the detection result meets the action range boundary, so that a stable execution state is obtained.

7. The method of claim 1, wherein the method further comprises: In the stable execution state, the working parameter set is collected again, the stability index is calculated, and the diversity-stability collaborative reward function based on the stability index is constructed, the reward is calculated, and the reward result is used to update the hierarchical policy network, and the specific steps include: In the stable execution state, the tuning circuit running data is collected to form a new working parameter set; The stability index is calculated according to the new working parameter set, and the diversity-stability collaborative reward function is defined based on the stability index. The diversity-stability collaborative reward function consists of three parts. The first part is a stability reward item proportional to the stability index. The second part is a diversity reward item obtained by constructing a kernel matrix from the candidate set embedding vector and calculating the logarithm of its determinant. The third part is an action smoothing constraint item obtained by averaging the difference between the tuning action vectors at consecutive time points, which is introduced in the form of punishment. The reward value calculated by the diversity-stability collaborative reward function is input into the hierarchical policy network, and the network parameters are updated according to the reward value to form an updated hierarchical policy network.

8. The method of claim 1, wherein the method further comprises: The updated hierarchical policy network is combined with the scene condition to generate a scene label, and the scene label and the optimal tuning action are combined to form a key-value pair index and stored, forming a lightweight policy and being solidified through network distillation, and the specific steps include: The scene condition is input into the updated hierarchical policy network, the scene condition includes user location, environmental noise and channel state information, the scene feature representation is obtained by combining the scene condition, and the scene label is generated by classification mapping, the scene label is a unique identifier sequence; The scene label is paired with the optimal tuning action output by the updated hierarchical policy network in the scene to form a key-value pair index, the key-value pair index takes the scene label as the key and the optimal tuning action as the value, all indexes are stored in the form of a table or a hash, forming a mapping structure that can be quickly retrieved; A lightweight strategy is constructed using an index set, which directly takes scene tags as input and outputs corresponding optimal tuning actions, and performs compression and pruning during construction to remove redundant entries and only keep representative mapping relationships, resulting in a compact lightweight strategy model; The network distillation method is used for solidification. During the distillation process, the lightweight strategy model is used as the teacher model, and the student network is used as the learning object. The scene tags are input and the corresponding optimal tuning actions are output. Through knowledge transfer, the student network learns the mapping relationship of the teacher model. After distillation, the student network is saved as the final lightweight strategy model.

Images