Multiprotocol wireless communication remote control system based on reinforcement learning

Abstract

The invention discloses a multi-protocol wireless communication remote control system based on reinforcement learning, and the system comprises a historical state data collection module which is used for generating a standardized data set; the improved Autoformer construction module is used for establishing an improved Autoformer model and outputting multi-step prediction of time delay, jitter and packet loss; the Lyapunov reinforcement learning optimization module is used for optimizing a prediction decision based on a composite Lyapunov-barrier function and quadratic programming safety projection; the prediction execution module is used for generating a future prediction result set; the constraint optimization solving module is used for solving cost and outputting optimal control parameters; and the remote control execution module is used for executing protocol establishment, switching and aggregation according to the optimal parameters and returning results. According to the method, the Autoformer model and the Lyapunov reinforcement learning algorithm are improved, so that low-delay and high-reliability control of multi-protocol wireless communication is realized.

Description

Technical Field

[0001] This invention relates to the field of wireless communication technology, and in particular to a multi-protocol wireless communication remote control system based on reinforcement learning. Background Technology

[0002] Existing remote control systems often rely on a single or a few fixed protocols on the wireless side, frequently using static thresholds or heuristic rules for protocol selection, power, and encoding configuration, or employing a cascaded process of "measurement—threshold determination—handover triggering." To reduce latency and jitter, some solutions incorporate simple fragmentation, redundancy, or multi-link backup, but lack the ability to predict future timeframes. They typically rely solely on short-window statistics, making it difficult to promptly identify non-stationary factors such as congestion, interference, and obstruction. Common prediction methods primarily use linear models or generalized recurrent networks, failing to provide structured modeling for communication-control coupled data with both seasonal and trend-based characteristics in long time series, resulting in insufficient characterization of peak latency and tail packet loss. On the decision-making side, which is often composed of separate modules, there is a lack of a unified optimization framework for protocol selection, seamless handover, multi-link aggregation, and physical layer parameter configuration, making it difficult to minimize the overall cost.

[0003] The main shortcomings of existing technologies are as follows: at the prediction level, there is insufficient characterization of long-term dependence and the superposition of seasonality and trends, making it difficult to provide stable multi-step results of latency, jitter and packet loss that can be used for decision-making; at the decision-making level, the coupled variables of protocol selection, switching, aggregation and power, encoding and retransmission are not included in a unified objective and constraint, making it difficult to coordinate the optimal within the energy consumption and cost boundaries; security relies on empirical thresholds and lacks formal guarantees based on Lyapunov functions and barrier functions; at the execution stage, there is a lack of secondary planning projection of candidate actions, which makes it impossible to ensure that each step meets the safety constraints with minimal modification, resulting in the risk of going out of bounds under exploration and abnormal conditions.

[0004] Therefore, how to provide a multi-protocol wireless communication remote control system based on reinforcement learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a multi-protocol wireless communication remote control system based on reinforcement learning. This invention comprehensively utilizes long-sequence time-series prediction modeling, constrained reinforcement learning, and formalized secure execution techniques to construct a closed-loop architecture of "prediction-decision-execution-backhaul." By improving the Autoformer model, it performs multi-step predictions of end-to-end latency, latency jitter, and packet loss rate at the edge, providing forward-looking information for protocol management. Based on a composite Lyapunov-barrier function, it constructs stability descent constraints and safety set invariance constraints, and employs quadratic programming secure projection to map candidate actions with minimal modifications to the set of possible actions, achieving unified decision-making for protocol selection, seamless handover, multi-link aggregation, and parameters such as power, encoding, retransmission, and fragmentation. Constraint optimization is performed under energy consumption and cost thresholds, outputting the optimal combination of control parameters, which is then executed on the multi-protocol wireless communication platform. The results are backhauled to the data acquisition and modeling stages to form a closed loop. This invention's system possesses low latency, high reliability, and provably secure characteristics for remote control, while also offering engineering advantages such as unified decision-making, real-time edge deployment, and auditability.

[0006] A multi-protocol wireless communication remote control system based on reinforcement learning according to an embodiment of the present invention includes the following modules:

[0007] The historical status data acquisition module is used to collect multi-protocol and control data and preprocess it to generate a standardized dataset;

[0008] An improved Autoformer building block is used to build an improved Autoformer model and train it on a standardized dataset to output multi-step prediction results for latency, jitter, and packet loss.

[0009] The Lyapunov reinforcement learning optimization module is used to optimize prediction decisions based on the composite Lyapunov-barrier function and quadratic programming safe projection, and obtain the optimized Autoformer model.

[0010] The prediction execution module is used to call the optimized Autoformer model for retraining and to generate a set of future prediction results.

[0011] The constrained optimization solution module is used to solve for costs under energy consumption and cost constraints and output the optimal combination of control parameters;

[0012] The remote control execution module is used to establish, switch, and aggregate protocols according to optimal parameters and then return the results.

[0013] Optionally, modules can be integrated using the following methods:

[0014] Historical status data is collected at the edge gateway and preprocessed to obtain a standardized dataset;

[0015] An improved Autoformer model is constructed, the historical window length, prediction step size and batch size are set, trend and seasonal decomposition is performed through the learnable decomposition kernel decomposition module, latency aggregation and feature reconstruction are performed based on the robust sparse correlation reconstruction module, and probability modeling is performed using the uncertainty prediction head module to obtain multi-step prediction results of end-to-end latency, latency jitter and packet loss rate.

[0016] The reinforcement learning algorithm based on Lyapunov functions optimizes the multi-step point prediction results. It constructs descent constraints and safety set invariance constraints by combining Lyapunov-barrier functions, and performs minimum modification projection using a quadratic programming safety projection layer to obtain the optimized Autoformer model.

[0017] The standardized dataset is input into the optimized Autoformer model to perform training and multi-step prediction, and a set of prediction results for end-to-end latency, latency jitter and packet loss rate at multiple future time points are obtained.

[0018] Based on the prediction result set, an objective function is established, with the reduction of end-to-end latency, latency jitter and packet loss rate as the optimization objectives. Under the constraint that energy consumption and cost do not exceed the preset threshold, a constraint solution with fixed parameters is performed to calculate the comprehensive cost function and obtain the optimal combination of control parameters.

[0019] Based on the optimal combination of control parameters, the multi-protocol wireless communication platform performs protocol establishment, seamless switching, and multi-link aggregation, and the execution results are sent back to the data acquisition and preprocessing stage to complete remote control.

[0020] Optionally, the historical status data includes link performance indicators of the multi-protocol wireless link and control task parameters of the remote control task. The link performance indicators are received signal strength, round-trip time, delay jitter, packet loss rate and congestion. The control task parameters are instruction priority, allowable delay threshold, actuator feedback error and timeout flag.

[0021] Optionally, the preprocessing includes missing data filling, anomaly removal, time alignment, and numerical normalization.

[0022] Optionally, obtaining the multi-step point prediction results of end-to-end delay, delay jitter, and packet loss rate includes:

[0023] The improved Autoformer model includes a feature construction module, a learnable decomposition kernel module, a robust sparse reconstruction module, an uncertainty prediction head module, and a feature fusion and output module.

[0024] A feature construction module is built to segment the standardized dataset according to the set historical window length, prediction step size and batch size, forming numerical sequence input and time feature input;

[0025] A learnable decomposition kernel module is constructed, a set of multi-base kernel one-dimensional convolution kernels and a gated weight generation unit are set, and each convolution kernel is weighted and summed according to its weight to form an effective decomposition kernel. The effective decomposition kernel is used to perform one-dimensional convolution on the numerical sequence input to obtain the trend sequence, and then the trend sequence is subtracted from the original sequence to obtain the seasonal sequence.

[0026] A robust sparse reconstruction module is constructed, which includes a correlation estimation unit, a threshold filtering unit, a sparse selection unit, and a time delay aggregation unit. The module calculates the correlation scores of the seasonal sequence and the time-shifted sequence and forms a correlation score sequence. Based on a fixed threshold rule, a candidate time delay set with correlation scores not lower than the threshold is retained. A sparse selection method is used to select a preset number of time delay indices with the highest scores from the candidate time delay set. The corresponding time-shifted subsequences are weighted and summed according to their weights to obtain the reconstructed seasonal sequence.

[0027] An uncertainty prediction head module is constructed, and a mean output unit and a variance output unit are set. The reconstructed seasonal series and trend series are connected in the channel dimension and then input into the uncertainty prediction head module. The mean output unit provides the point prediction results of end-to-end delay, delay jitter and packet loss rate for multiple prediction steps, and the variance output unit provides the variance parameter corresponding to the point prediction results.

[0028] A feature fusion and output module is constructed, which adds the trend sequence and the reconstructed seasonal sequence point by point to obtain the layer output features. The features are then passed layer by layer according to the stacked structure of the encoder and decoder, and finally outputs the multi-step point prediction results of end-to-end delay, delay jitter and packet loss rate.

[0029] Optionally, the solution yields the softened variable values ​​and the action vector, including:

[0030] Identify the elements for improving the Autoformer model, including the state set, action set, discount factor, empirical approximation of transition dynamics, fixed cost budget, and fixed decrease factor;

[0031] Define a nonnegative Lyapunov function and set a barrier function obtained by monotonic transformation of the safety set decision function. Construct a composite function as the Lyapunov-barrier function by summing with fixed weights.

[0032] At the current state, a descent inequality constraint is established for the expected increment and cost term of the composite function. In discrete time, a forward invariance inequality constraint is established for the safety set decision function. Local linearization is then used to write the inequality constraint as an affine inequality about the action vector, thus obtaining the affine inequality constraint.

[0033] The original action vector is output from the action candidate unit. A quadratic programming secure projection problem is established. The objective function is the quadratic form of the original action vector and the weighted Euclidean distance. The constraints are affine inequality and upper and lower bounds of the action. The projected action vector that satisfies the constraints is obtained by solving the problem. The upper and lower bounds of the action come from the actual physical and protocol constraints in the communication.

[0034] The projected action vectors interact with the environment, and the reward samples during the interaction process are collected. The policy loss, the Lyapunov consistency loss of the composite function increment and cost term, and the feasibility penalty of the first norm of the softening variable are calculated through the reward samples. The three are summed with fixed weights to form the total loss and update the action candidate unit parameters and Lyapunov function parameters.

[0035] Repeat the interaction, projection, and update until a fixed iteration limit or convergence criterion is reached, and output the optimized Autoformer model that satisfies the descent constraint and the safety set invariance constraint.

[0036] Optionally, the set of predicted results for end-to-end latency, latency jitter, and packet loss rate at multiple future time points includes:

[0037] Call the optimized Autoformer model and load fixed network structure parameters and training hyperparameters, and set the sliding training window length, prediction step size, batch size, learning rate, number of training rounds and number of early stopping rounds;

[0038] A sliding window is used to construct training samples and label samples on a standardized dataset, and standardized feature vector sequences and label sequences are generated according to time index;

[0039] The training loss function is the sum of the weighted absolute errors of end-to-end latency, latency jitter and packet loss rate. The optimized Autoformer model is subjected to batch forward computation, loss calculation and parameter update until the training round limit is reached or the early stopping condition is met.

[0040] The standardized feature vector sequence and label sequence are input in a rolling manner according to the prediction step size. The end-to-end delay point prediction result, delay jitter point prediction result and packet loss rate point prediction result at multiple future time points are output according to the time index, forming a prediction result set corresponding to the time index.

[0041] Optionally, the calculation of the comprehensive cost function and the obtaining of the optimal control parameter combination includes:

[0042] The optimization objective is to minimize the comprehensive cost function. The comprehensive cost function is a linear weighted sum of the time average of the end-to-end delay prediction sequence, the time average of the delay jitter prediction sequence, and the time average of the packet loss rate prediction sequence, with non-negative constant weights. A fixed penalty term for the change in protocol switching and link aggregation method is added. The solution is obtained under the constraint that the unit time average of energy consumption and cost does not exceed their respective thresholds.

[0043] By setting reliability and feasibility constraints, a mixed-integer linear programming problem is established and solved to obtain the combination of control parameters for optimal protocol selection, optimal link aggregation and redundancy, optimal transmit power level, optimal coding error correction level, optimal retransmission limit, and optimal fragmentation strategy.

[0044] Optionally, the step of sending the execution result back to the data acquisition and preprocessing stage to complete remote control includes:

[0045] Based on the combination of control parameters, the protocol establishment, session persistence, seamless handover and multi-link aggregation are performed on the multi-protocol wireless communication platform. The primary protocol, backup or parallel links, transmit power level, coding error correction level, retransmission limit and fragmentation are set in sequence. During the execution process, the link status and control command issuance are synchronously scheduled.

[0046] After execution, end-to-end latency, latency jitter, packet loss rate, energy consumption and cost are collected and recorded in real time. The collected results, along with the corresponding control parameter identifiers, time indexes and link identifiers, are sent back to the data collection and preprocessing stage to update the standardized dataset and form remote control.

[0047] The beneficial effects of this invention are:

[0048] This invention introduces an improved Autoformer prediction model at the edge gateway side to achieve multi-step look-ahead prediction of end-to-end latency, latency jitter, and packet loss rate. By employing a learnable decomposition kernel and robust sparse correlation reconstruction, combined with simplified uncertainty output, it can accurately capture key risk characteristics such as peak latency and tail packet loss in long-series, strong interference, and non-stationary scenarios, significantly improving prediction stability and availability. It provides highly reliable prior information for protocol selection, switching, and aggregation, and reduces lag and mismatch caused by decisions based on short-window statistics.

[0049] This invention introduces a composite Lyapunov-barrier function and quadratic programming safety projection at the decision-making and security levels, forming a unified link of "prediction-constraint-projection-execution". Before each step of execution, the candidate action is mapped to a set of actionable actions that meet the constraints of stability degradation and safety set invariance. This ensures closed-loop stability and provable security during protocol switching and multi-link aggregation. At the same time, energy consumption and cost are included as hard boundaries in the solution, avoiding the overstepping and instability of traditional empirical threshold schemes under exploratory or abnormal conditions.

[0050] This invention constructs an engineering closed loop of prediction-decision-execution-backhaul, supporting protocol establishment, seamless switching, and multi-link aggregation on multi-protocol wireless platforms with optimal control parameter combinations. The execution results are backhauled for continuous data accumulation and model retraining, achieving low-latency, high-reliability, and auditable remote control. The system has advantages such as real-time deployment at the edge, unified decision space, and reproducible fixed parameters. It can stably reduce end-to-end latency and jitter, suppress packet loss, and take into account energy consumption and cost constraints without increasing manual parameter tuning costs, meeting the stability and reliability requirements of industrial-grade remote control applications. Attached Figure Description

[0051] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0052] Figure 1 This is a schematic diagram of the structure of a multi-protocol wireless communication remote control system based on reinforcement learning proposed in this invention;

[0053] Figure 2 This is a flowchart illustrating a multi-protocol wireless communication remote control method based on reinforcement learning proposed in this invention.

[0054] Figure 3 This is a schematic diagram of the functional modules of the improved Autoformer model for a multi-protocol wireless communication remote control method based on reinforcement learning proposed in this invention. Detailed Implementation

[0055] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0056] refer to Figure 1 A multi-protocol wireless communication remote control system based on reinforcement learning includes the following modules:

[0057] The historical status data acquisition module is used to collect multi-protocol and control data and preprocess it to generate a standardized dataset;

[0058] An improved Autoformer building block is used to build an improved Autoformer model and train it on a standardized dataset to output multi-step prediction results for latency, jitter, and packet loss.

[0059] The Lyapunov reinforcement learning optimization module is used to optimize prediction decisions based on the composite Lyapunov-barrier function and quadratic programming safe projection, and obtain the optimized Autoformer model.

[0060] The prediction execution module is used to call the optimized Autoformer model for retraining and to generate a set of future prediction results.

[0061] The constrained optimization solution module is used to solve for costs under energy consumption and cost constraints and output the optimal combination of control parameters;

[0062] The remote control execution module is used to establish, switch, and aggregate protocols according to optimal parameters and then return the results.

[0063] refer to Figure 2 and Figure 3 A multi-protocol wireless communication remote control method based on reinforcement learning, comprising:

[0064] Historical status data is collected at the edge gateway and preprocessed to obtain a standardized dataset;

[0065] An improved Autoformer model is constructed, the historical window length, prediction step size and batch size are set, trend and seasonal decomposition is performed through the learnable decomposition kernel decomposition module, latency aggregation and feature reconstruction are performed based on the robust sparse correlation reconstruction module, and probability modeling is performed using the uncertainty prediction head module to obtain multi-step prediction results of end-to-end latency, latency jitter and packet loss rate.

[0066] The reinforcement learning algorithm based on Lyapunov functions optimizes the multi-step point prediction results. It constructs descent constraints and safety set invariance constraints by combining Lyapunov-barrier functions, and performs minimum modification projection using a quadratic programming safety projection layer to obtain the optimized Autoformer model.

[0067] The standardized dataset is input into the optimized Autoformer model to perform training and multi-step prediction, and a set of prediction results for end-to-end latency, latency jitter and packet loss rate at multiple future time points are obtained.

[0068] Based on the prediction result set, an objective function is established, with the reduction of end-to-end latency, latency jitter and packet loss rate as the optimization objectives. Under the constraint that energy consumption and cost do not exceed the preset threshold, a constraint solution with fixed parameters is performed to calculate the comprehensive cost function and obtain the optimal combination of control parameters.

[0069] Based on the optimal combination of control parameters, the multi-protocol wireless communication platform performs protocol establishment, seamless switching, and multi-link aggregation, and the execution results are sent back to the data acquisition and preprocessing stage to complete remote control.

[0070] In this embodiment, the historical state data includes the link performance indicators of the multi-protocol wireless link and the control task parameters of the remote control task. The link performance indicators are received signal strength, round-trip time, delay jitter, packet loss rate and congestion. The control task parameters are instruction priority, allowable delay threshold, actuator feedback error and timeout flag.

[0071] In this embodiment, the preprocessing includes missing data filling, anomaly removal, time alignment, and numerical normalization.

[0072] In this embodiment, obtaining the multi-step point prediction results of end-to-end delay, delay jitter, and packet loss rate includes:

[0073] The improved Autoformer model includes a feature construction module, a learnable decomposition kernel module, a robust sparse reconstruction module, an uncertainty prediction head module, and a feature fusion and output module.

[0074] A feature construction module is built to segment the standardized dataset according to the set historical window length, prediction step size and batch size, forming numerical sequence input and time feature input;

[0075] A learnable decomposition kernel module is constructed, which sets up a multi-base kernel one-dimensional convolution kernel set and a gated weight generation unit. The convolution kernels are weighted and summed according to their weights to form an effective decomposition kernel. The effective decomposition kernel is used to perform one-dimensional convolution on the numerical sequence input to obtain the trend sequence. Then, the trend sequence is subtracted from the original sequence to obtain the seasonal sequence. Where:

[0076] The setup of the multi-base kernel one-dimensional convolution kernel set and gated weight generation unit is as follows: Several one-dimensional convolution branches are configured for each channel, and each branch corresponds to a base convolution kernel. The length and dilation rate of the kernels can be different. Channel-wise depth convolution is used to perform convolution on each channel. The convolution stride is one, and the padding method is to keep the length unchanged. At the same time, a gated weight generation unit is constructed to perform global averaging on the input in the time dimension to obtain the channel-level description vector. The output is an unnormalized weight with the same number of base convolution kernels through a small feedforward network. The weights are then exponentially normalized to make each weight non-negative and summed to one in the same channel.

[0077] The trend sequence is obtained as follows: For each channel, the output sequence of all convolution branches is first obtained, and then the weighted sum is performed point by point according to the gating weight to obtain the initial trend sequence. In order to suppress excessive smoothing and boundary offset, the trend gating coefficient is introduced to be linearly combined with the exponential moving average sequence with fixed parameters, and the channel bias term is superimposed to obtain the final trend sequence. All calculations are performed while maintaining the same time length and alignment relationship as the input. The dilated convolution branches are also filled symmetrically or by reflection to ensure that the output length is consistent with the input.

[0078] The obtained seasonal sequence is obtained by subtracting point by point, subtracting the trend value at the corresponding position from the original input value at each time position;

[0079] A robust sparse reconstruction module is constructed, comprising a correlation estimation unit, a threshold filtering unit, a sparse selection unit, and a time delay aggregation unit. It calculates correlation scores for seasonal and time-shifted sequences, forming a correlation score sequence. Based on a fixed threshold rule, a candidate time delay set with correlation scores not lower than the threshold is retained. A sparse selection method is used to select a preset number of time delay indices with the highest scores from the candidate time delay set. The corresponding time-shifted subsequences are then weighted and summed according to their weights to obtain the reconstructed seasonal sequence. Where:

[0080] The correlation estimation unit is used to measure the similarity between the seasonal sequence and each time-shifted sequence one by one. The measurement method adopts robust correlation scoring. First, the two sequences are decentered and scaled, and then the product is calculated point by point according to the corresponding time position and the average is calculated. At the same time, a threshold truncation or piecewise linear penalty is applied to outliers to reduce the impact of isolated peaks on the score. All robust correlation scores corresponding to time delays are arranged in the order of time delay index to form a correlation score sequence of the same length as the candidate time delay set.

[0081] The threshold filtering unit performs a one-time filtering on the relevant score sequences based on a fixed threshold rule, retaining only the delays that are not lower than the threshold and recording them as a candidate delay set. If no delay meets the threshold requirement, the highest-scoring delays are retained according to a preset backoff rule to ensure that the process can be executed.

[0082] The sparse selection unit selects a preset number of time delay indices from high to low scores, or breaks the parallel relationship in a fixed order when scores are close, to ensure that the number of selected time delays is consistent with the setting and covers different phase positions;

[0083] The time delay aggregation unit is based on the time delay index obtained in the previous section. It aligns the corresponding time-shifted subsequences with the current seasonal sequence on the time axis and then performs a weighted summation. The weights are composed of fixed non-negative coefficients or obtained by a linear mapping and normalization. It is required that the sum of each weight in the selected time delay set is equal to one. The aggregation process performs the same weighted summation for each time position and outputs a reconstructed seasonal sequence with the same length as the input. To enhance numerical stability, a minimum lower limit can be applied to the weights to avoid the complete failure of a certain time delay. Boundary smoothing is added to the reconstructed sequence to eliminate the edge effect introduced by the filling.

[0084] An uncertainty prediction head module is constructed, with mean and variance output units set up. The reconstructed seasonal and trend series are concatenated along the channel dimension and then input into the uncertainty prediction head module. The mean output unit provides the point prediction results for end-to-end delay, delay jitter, and packet loss rate across multiple prediction steps. The variance output unit provides the variance parameters corresponding to the point prediction results, where:

[0085] The uncertainty prediction head module consists of a shared feature extraction layer, a mean output unit, and a variance output unit. The shared feature extraction layer uses a one-dimensional convolution or feedforward layer to map the concatenated features time-by-time and maintain time alignment. The output of the module corresponds one-to-one with the input on the time axis, ensuring that both mean and variance results can be obtained at each prediction step.

[0086] The mean output unit generates point prediction results for three targets at each future prediction step through one or more linear mappings, corresponding to end-to-end delay, delay jitter, and packet loss rate, respectively. The mapping uses a fixed number of output channels and organizes the output vectors according to the target order. Each time position output vector contains three scalars, which represent the point prediction of delay, the point prediction of jitter, and the point prediction of packet loss rate, respectively. To ensure that the numerical range is reasonable, the point prediction of packet loss rate is subjected to range pruning or bounded mapping at the output end to ensure that it falls within the closed interval of zero to one.

[0087] The variance output unit takes the time-by-time features of the shared feature extraction layer as input, and after linear mapping, it is connected to a non-negative constraint function to generate variance parameters that correspond one-to-one with the point prediction. The non-negative constraint function adopts a monotonically increasing transformation with a non-negative output to ensure that the variance is zero or positive. The variance output unit also outputs three scalars at each future prediction step, representing the time delay variance, jitter variance, and packet loss rate variance in turn. Finally, the mean and variance information are collected in time order to form the mean sequence and variance sequence of multi-step prediction.

[0088] A feature fusion and output module is constructed, which adds the trend sequence and the reconstructed seasonal sequence point by point to obtain the layer output features. The features are then passed layer by layer according to the stacked structure of the encoder and decoder, and finally outputs the multi-step point prediction results of end-to-end delay, delay jitter and packet loss rate.

[0089] In this embodiment, the process of obtaining the softening variable value and the action vector includes:

[0090] Identify the elements for improving the Autoformer model, including the state set, action set, discount factor, empirical approximation of transition dynamics, fixed cost budget, and fixed decrease factor;

[0091] Define a nonnegative Lyapunov function and set a barrier function obtained by monotonically transforming the safety set decision function. Construct a composite function, the Lyapunov-barrier function, using a fixed-weight summation, where:

[0092] The definition of the non-negative Lyapunov function is, after determining the composition of the state space and the state vector, selecting a function that is non-negative everywhere, zero at the safe operating point, and monotonically increasing away from the safe operating point as the Lyapunov function. The Lyapunov function is continuously differentiable with respect to the state vector and provides a calculable gradient for the change of each component.

[0093] The safe set decision function is defined as follows: a positive value indicates that the safe set is inside the safe set, a zero value indicates that the safe boundary is on the safe boundary, and a negative value indicates that the boundary is exceeded. The safe set decision function is continuously differentiable. The input is the current state and the output is a scalar. It is used to describe the joint conditions such as end-to-end delay not exceeding a threshold, packet loss rate not exceeding a threshold, and control error not exceeding a threshold. It can be obtained by combining each individual constraint into a scalar form by minimizing the individual constraints or by linearly combining them.

[0094] The construction of the barrier function from the safety set decision function is specifically as follows:

[0095] Choose a monotonic transformation that is strictly monotonically increasing when the independent variable is positive, tends to positive infinity when the independent variable approaches zero, and has a strong penalty property in the safety set. Use the safety set decision function as the independent variable to perform a monotonic transformation to obtain the barrier function.

[0096] The monotonic transformation is selected from logarithmic, reciprocal, or differentiable transformations that satisfy non-negative output for positive numbers, and a strictly positive lower bound is set for the input in the implementation to prevent the value from diverging when it is close to zero.

[0097] The barrier function takes a finite positive value inside the safe set, increases rapidly when approaching the safe boundary, and is not called outside the safe set. To ensure realizability, the gradient calculation method of the barrier function with respect to the state is recorded, so that it can be directly called during action linearization and constraint matrix construction.

[0098] At the current state, a descent inequality constraint is established for the expected increment and cost term of the composite function. In discrete time, a forward invariance inequality constraint is established for the safety set decision function. Local linearization is then used to rewrite the inequality constraints as affine inequalities about the action vector, resulting in the affine inequality constraints, where:

[0099] The process of establishing descent inequality constraints involves using known discrete-time system dynamics to obtain the conditional distribution of the state at the next time step, calculating the expected value of the composite Lyapunov-barrier function under the conditional distribution, subtracting the value of the composite function under the current state from the expected value to obtain the expected increment, calculating the difference between the cost item related to the current action and the fixed cost budget, scaling the difference proportionally by a pre-calibrated non-negative constant, adding the expected increment to the scaled cost excess item to form a single scalar measure, and requiring the scalar to be no greater than zero, which serves as the descent inequality constraint.

[0100] The establishment of forward invariance inequality constraints involves obtaining the conditional distribution of the state at the next time step using the same transition model under the same current state and candidate actions, calculating the conditional expectation or deterministic approximation of the safety set decision function at the next time step, and comparing it with the value of the safety set decision function at the current time step. According to the standard form of discrete-time invariance, a fixed contraction coefficient or a fixed safety margin is set, and the target value of the decision function at the next time step is compared with the target value of the decision function at the current time step after adjustment by the fixed coefficient or fixed margin. The former is required to be no less than the latter. The inequality is denoted as forward invariance inequality constraint.

[0101] The inequality constraint is written as an affine inequality with respect to the action vector, specifically as follows:

[0102] Around the current state and nominal action, a first-order Taylor expansion approximation is performed on the state update mapping, the composite Lyapunov-barrier function, and the safe set decision function, respectively. The function approximation value at the next moment is represented by the function value at the current point plus the product of the first-order partial derivatives with respect to the state and action and the increment. The expected increment and cost terms in the descent inequality constraint are replaced by a first approximation, and all terms related to the action are collected on the left side, while constant terms unrelated to the action are moved to the right side, forming an affine inequality of matrix multiplication by the action vector plus constant terms not greater than zero.

[0103] The forward invariance inequality constraint is processed in the same way to obtain the affine inequality that is matrix multiplied by action vector plus constant term not less than zero. The two types of affine inequalities and the upper and lower bound constraints of the action are used as the linear constraint input for the quadratic programming safe projection.

[0104] The original action vector is output from the action candidate unit. A quadratic programming secure projection problem is established, with the quadratic form of the original action vector and the weighted Euclidean distance as the objective function. Affine inequality constraints and upper and lower bounds of the actions are used as constraints. The projected action vector that satisfies the constraints is obtained by solving the problem. The upper and lower bounds of the actions are derived from the actual physical and protocol constraints in the communication. The establishment of the quadratic programming secure projection problem is specifically as follows:

[0105] Given the state information at the current moment, the action candidate unit outputs a set of original action vectors based on the state and multi-step prediction results. Each component of the original action vector corresponds to the control variables of protocol selection, link aggregation and redundancy mode, transmit power level, coding error correction level, retransmission limit and fragmentation, respectively. In order to restrict the original action vector to the set of possible actions that satisfy the descent inequality constraint and the forward invariance inequality constraint, a quadratic programming secure projection problem is constructed. The original action vector is regarded as the reference point to be projected, and the decision variable is defined as the projected action vector to be solved. The dimension and component meaning of the projected action vector are consistent with those of the original action vector.

[0106] In constructing the objective function, the square of the weighted Euclidean distance between the original action vector and the projected action vector is used as the optimization objective. The difference between the original action vector and the projected action vector at each component is multiplied by a pre-set non-negative weight. The squares of all component differences multiplied by the weights are calculated and summed. The sum is used as the objective function to be minimized. The weights are used to reflect the relative importance of different control variables when deviating from the original value and are kept as fixed constants during construction. In constructing the constraints, all the descent inequality constraints and forward invariance inequality constraints obtained by local linearization in the previous step are written as linear inequalities about the projected action vector. The range of values ​​of each action component is added to the same set of linear constraints in the form of lower bound constraints and upper bound constraints, forming a set of affine inequalities containing several linear functions that are not greater than zero or not less than zero, and a set of upper and lower bound inequalities for action components.

[0107] After determining the objective function and constraints, the objective function with weighted Euclidean distance squared and all linear inequality constraints are combined to form a standard quadratic programming problem. The original action vector is treated as the parameter input, and the projected action vector is considered the sole decision variable in the quadratic programming problem. The solution process is executed by calling a pre-selected quadratic programming solver with fixed parameters. Under the premise of satisfying all descent constraints, forward invariance constraints, and upper and lower bound constraints on actions, a projected action vector that minimizes the weighted Euclidean distance squared is obtained. This projected action vector is the optimal solution to the quadratic programming safe projection problem and serves as the actual execution action for interaction with the environment and for distribution to the multi-protocol wireless communication platform.

[0108] The projected action vectors interact with the environment, and reward samples are collected during the interaction. The policy loss, the Lyapunov consistency loss of the composite function increment and cost term, and the feasibility penalty of the L1 norm of the softening variable are calculated based on these reward samples. These three factors are then summed with fixed weights to form the total loss, which is used to update the action candidate unit parameters and the Lyapunov function parameters.

[0109] The calculation of the policy loss involves fixing the current policy parameters and Lyapunov correlation parameters, using the projected action vector obtained in the previous step to interact with the environment, collecting one or more sample sequences of states, actions, immediate rewards, and termination markers in chronological order, and then weighting and accumulating the immediate rewards on each trajectory from back to front according to a preset discount factor to obtain the discounted cumulative reward for each time step. The difference between the discounted cumulative reward and the value estimate, or the advantage signal processed by the advantage estimation method, is used to construct the policy gradient class loss. By averaging the samples within a batch, the policy loss with the goal of "increasing the probability of high-reward actions and reducing the probability of low-reward actions" is obtained.

[0110] The calculation of the composite function increment and the Lyapunov consistency loss of the cost term involves using the current parameters to calculate the function value of the composite Lyapunov-barrier function at each time step in the current state, and then approximating the function value in the next state based on environmental dynamics or experience. The difference between the two is the "composite function increment". Simultaneously, based on a predefined cost function, the cost value corresponding to the current state and the projected action is calculated at each time step, and the difference is calculated with a fixed cost budget. The difference is scaled according to a pre-set descent coefficient and then added to the composite function increment to obtain a scalar measure representing "whether the descent requirement is met". For each time step, the scalar is compared with zero, and the part greater than zero is taken as the degree of violation of the descent constraint. The square or absolute value of the violation degree is averaged within a batch to obtain the Lyapunov consistency loss.

[0111] The calculation of the feasibility penalty for the first norm of the softened variable is carried out when constructing a quadratic programming safe projection problem. In order to ensure that there is still a solution when the constraints are tight or the linearization approximation error is large, non-negative softened variables are usually introduced into the descent inequality constraints and forward invariance inequality constraints. The original linear inequality that must be satisfied is relaxed to the form of "the linear function plus the softened variable is not greater than zero" or "not less than zero". After the quadratic programming is solved, each sample will get a set of softened variable values ​​at each time step. The absolute values ​​of all softened variables in the same batch are summed or averaged over time and by the sample, and then multiplied by a pre-set penalty weight to obtain the feasibility penalty term.

[0112] Repeat the interaction, projection, and update until a fixed iteration limit or convergence criterion is reached, and output the optimized Autoformer model that satisfies the descent constraint and the safety set invariance constraint.

[0113] In this embodiment, obtaining the prediction result set of end-to-end delay, delay jitter, and packet loss rate at multiple future time points includes:

[0114] Call the optimized Autoformer model and load fixed network structure parameters and training hyperparameters, and set the sliding training window length, prediction step size, batch size, learning rate, number of training rounds and number of early stopping rounds;

[0115] A sliding window is used to construct training samples and label samples on a standardized dataset, and standardized feature vector sequences and label sequences are generated according to time index;

[0116] The training loss function is the sum of the weighted absolute errors of end-to-end latency, latency jitter and packet loss rate. The optimized Autoformer model is subjected to batch forward computation, loss calculation and parameter update until the training round limit is reached or the early stopping condition is met.

[0117] The standardized feature vector sequence and label sequence are input in a rolling manner according to the prediction step size. The end-to-end delay point prediction result, delay jitter point prediction result and packet loss rate point prediction result at multiple future time points are output according to the time index, forming a prediction result set corresponding to the time index.

[0118] In this embodiment, calculating the comprehensive cost function and obtaining the optimal combination of control parameters includes:

[0119] The optimization objective is to minimize the comprehensive cost function. The comprehensive cost function is a linear weighted sum of the time average of the end-to-end delay prediction sequence, the time average of the delay jitter prediction sequence, and the time average of the packet loss rate prediction sequence, with non-negative constant weights. A fixed penalty term for the change in protocol switching and link aggregation method is added. The solution is obtained under the constraint that the unit time average of energy consumption and cost does not exceed their respective thresholds.

[0120] By setting reliability and feasibility constraints, a mixed-integer linear programming problem is established and solved to obtain the combination of control parameters for optimal protocol selection, optimal link aggregation and redundancy, optimal transmit power level, optimal coding error correction level, optimal retransmission limit, and optimal fragmentation strategy.

[0121] In this embodiment, the step of sending the execution result back to the data acquisition and preprocessing stage to complete remote control includes:

[0122] Based on the combination of control parameters, the protocol establishment, session persistence, seamless handover and multi-link aggregation are performed on the multi-protocol wireless communication platform. The primary protocol, backup or parallel links, transmit power level, coding error correction level, retransmission limit and fragmentation are set in sequence. During the execution process, the link status and control command issuance are synchronously scheduled.

[0123] After execution, end-to-end latency, latency jitter, packet loss rate, energy consumption and cost are collected and recorded in real time. The collected results, along with the corresponding control parameter identifiers, time indexes and link identifiers, are sent back to the data collection and preprocessing stage to update the standardized dataset and form remote control.

[0124] Example 1:

[0125] To verify the feasibility of this invention in practice, it was applied to the remote crane control system of a hot rolling production line in a coastal steel enterprise, illustrating the application process and effects of the system. Crane control, operating remotely from the central control room, handles the hoisting of hot billets under conditions of high temperature, strong electromagnetic interference, metal obstruction, and multipath reflection. The existing site had one Wi-Fi 6 private network, one 5G industrial private network, and several industrial Bluetooth relays, employing a rule-based strategy of Wi-Fi priority, low-switching to 5G, and Bluetooth backup after timeout. An edge gateway was deployed at the crane control layer, directly connected to the Wi-Fi AP, 5G CPE, and Bluetooth gateway, and connected to the crane PLC and control center. Link indicators such as received signal strength, round-trip time, latency jitter, packet loss rate, and congestion, as well as control task parameters such as instruction priority, allowable latency threshold, actuator feedback error, and timeout flags, were continuously collected at 20-millisecond granularity. After missing data filling, anomaly removal, time alignment, and normalization, a standardized historical dataset was constructed.

[0126] An improved Autoformer model is built on the edge gateway side. Using the multi-channel link and control feature sequence of the past ten to twenty seconds as the input window, it performs multi-step predictions of end-to-end latency, latency jitter, and packet loss rate for the next one to two seconds. The model performs trend and seasonal decomposition through a learnable decomposition kernel module, aggregates the seasonal components for latency through a robust sparse reconstruction module, and outputs the point prediction results and variance quantization for each prediction step from an uncertainty prediction head to characterize latency peaks and tail packet loss risks. These prediction outputs, along with the current link and control states, are used as reinforcement learning states to drive a Lyapunov function-based decision logic to output actions such as protocol selection, multi-link aggregation and redundancy, transmit power level, coding error correction level, retransmission limit, and fragmentation strategy in each control cycle.

[0127] To ensure closed-loop stability and communication security, a composite Lyapunov-barrier function is introduced to construct stability descent constraints and safety set invariance constraints. Local linearization is then performed near the current state, and the result is rewritten as a linear inequality concerning the action. A quadratic programming safety projection is then established, projecting the original actions given by the action candidate units onto the set of feasible actions that satisfy the constraints using the minimum weighted Euclidean distance. The projected control parameters are executed in real-time on a multi-protocol wireless platform, enabling protocol establishment, seamless switching, and multi-link aggregation for vehicle communication. To quantify the beneficial effects of this invention, from May to June 2024, three key overhead cranes in a steel plant were selected. Comparative tests were conducted under three typical operating conditions: normal daytime production, nighttime maintenance, and peak shift changes. The tests were performed using both the original rule-based scheme and the system of this invention, with each mode running continuously for no less than 72 hours.

[0128] Table 1 Performance comparison between rule-based scheme and the system of the present invention

[0129]

[0130] As shown in Table 1, under normal daytime production conditions, the system of this invention significantly improves end-to-end transmission performance compared to the original regularized scheme. The original scheme had an average latency of 28.7 milliseconds and a 95th percentile latency of 65.3 milliseconds, while the system of this invention reduces the average latency to 19.5 milliseconds and the 95th percentile latency to 42.1 milliseconds, shifting the overall latency distribution forward and significantly compressing the latency at the tail end. The packet loss rate decreased from 0.92% to 0.37%, and the average latency jitter decreased from 11.8 milliseconds to 6.3 milliseconds. This indicates that under daily production loads, the improved Autoformer prediction and reinforcement learning decision-making based on Lyapunov functions can more stably select appropriate protocols and parameters, significantly improving link quality and providing a smoother response for remote control operations.

[0131] The advantages of this invention's system become even more pronounced under conditions of greater interference or more sudden business disruptions, such as nighttime maintenance and peak shift changes. In nighttime maintenance scenarios, this invention's system reduces the average latency from 31.2 milliseconds to 21.0 milliseconds, the 95th percentile latency from 71.8 milliseconds to 46.5 milliseconds, the packet loss rate from 1.05% to 0.41%, and the latency jitter from 13.4 milliseconds to 7.1 milliseconds. In peak shift change scenarios, the original solution's 95th percentile latency reached 89.7 milliseconds, while this invention's system controls it at 54.2 milliseconds, reducing the packet loss rate from 1.38% to 0.52%, and the average jitter from 16.9 milliseconds to 8.9 milliseconds. This demonstrates that during periods of frequent multi-protocol switching and intense link contention, this invention, through unified decision-making and secondary planning for secure projection, effectively avoids sudden increases in latency and worsening packet loss caused by blind or excessive switching, maintaining the stability and reliability of the remote control link.

[0132] For long-term, high-pressure scenarios such as continuous lifting of heavy loads, the system of this invention also exhibits excellent steady-state performance. Under the original regular scheme, the average latency was 34.5 milliseconds, the 95th percentile latency was 83.1 milliseconds, the packet loss rate was 1.21%, and the average latency jitter was 15.2 milliseconds. After adopting this invention, the average latency decreased to 22.4 milliseconds, the 95th percentile latency decreased to 51.7 milliseconds, the packet loss rate decreased to 0.48%, and the latency jitter decreased to 8.0 milliseconds, with the overall latency curve becoming significantly lower and narrower. A comprehensive analysis of various operating conditions shows that this invention can stably reduce the average latency and tail latency under different load and interference conditions, significantly reducing packet loss and jitter. It demonstrates the beneficial effects of end-to-end low-latency, high-reliability communication for remote control, and ensures closed-loop stability through unified decision-making on protocol selection and parameter configuration, providing more reliable technical support for industrial-grade remote control scenarios.

[0133] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A multi-protocol wireless communication remote control system based on reinforcement learning, characterized in that, Includes the following modules: The historical status data acquisition module is used to collect multi-protocol and control data and preprocess it to generate a standardized dataset; An improved Autoformer building block is used to build an improved Autoformer model and train it on a standardized dataset to output multi-step prediction results for latency, jitter, and packet loss. The Lyapunov reinforcement learning optimization module is used to optimize prediction decisions based on the composite Lyapunov-barrier function and quadratic programming safe projection, and obtain the optimized Autoformer model. The prediction execution module is used to call the optimized Autoformer model for retraining and to generate a set of future prediction results. The constrained optimization solution module is used to solve for costs under energy consumption and cost constraints and output the optimal combination of control parameters; The remote control execution module is used to establish, switch, and aggregate protocols according to optimal parameters and then return the results.

2. A multi-protocol wireless communication remote control method based on reinforcement learning, applied to the multi-protocol wireless communication remote control system based on reinforcement learning as described in claim 1, characterized in that, include: Historical status data is collected at the edge gateway and preprocessed to obtain a standardized dataset; An improved Autoformer model is constructed, the historical window length, prediction step size and batch size are set, trend and seasonal decomposition is performed through the learnable decomposition kernel decomposition module, latency aggregation and feature reconstruction are performed based on the robust sparse correlation reconstruction module, and probability modeling is performed using the uncertainty prediction head module to obtain multi-step prediction results of end-to-end latency, latency jitter and packet loss rate. The reinforcement learning algorithm based on Lyapunov functions optimizes the multi-step point prediction results. It constructs descent constraints and safety set invariance constraints by combining Lyapunov-barrier functions, and performs minimum modification projection using a quadratic programming safety projection layer to obtain the optimized Autoformer model. The standardized dataset is input into the optimized Autoformer model to perform training and multi-step prediction, and a set of prediction results for end-to-end latency, latency jitter and packet loss rate at multiple future time points are obtained. Based on the prediction result set, an objective function is established, with the reduction of end-to-end latency, latency jitter and packet loss rate as the optimization objectives. Under the constraint that energy consumption and cost do not exceed the preset threshold, a constraint solution with fixed parameters is performed to calculate the comprehensive cost function and obtain the optimal combination of control parameters. Based on the optimal combination of control parameters, the multi-protocol wireless communication platform performs protocol establishment, seamless switching, and multi-link aggregation, and the execution results are sent back to the data acquisition and preprocessing stage to complete remote control.

3. The multi-protocol wireless communication remote control method based on reinforcement learning according to claim 2, characterized in that, The historical status data includes link performance indicators of multi-protocol wireless links and control task parameters of remote control tasks. The link performance indicators are received signal strength, round-trip time, delay jitter, packet loss rate and congestion. The control task parameters are instruction priority, allowable delay threshold, actuator feedback error and timeout flag.

4. The multi-protocol wireless communication remote control method based on reinforcement learning according to claim 2, characterized in that, The preprocessing includes missing data filling, anomaly removal, time alignment, and numerical normalization.

5. The multi-protocol wireless communication remote control method based on reinforcement learning according to claim 2, characterized in that, The multi-step point prediction results for obtaining end-to-end delay, delay jitter, and packet loss rate include: The improved Autoformer model includes a feature construction module, a learnable decomposition kernel module, a robust sparse reconstruction module, an uncertainty prediction head module, and a feature fusion and output module. A feature construction module is built to segment the standardized dataset according to the set historical window length, prediction step size and batch size, forming numerical sequence input and time feature input; A learnable decomposition kernel module is constructed, a set of multi-base kernel one-dimensional convolution kernels and a gated weight generation unit are set, and each convolution kernel is weighted and summed according to its weight to form an effective decomposition kernel. The effective decomposition kernel is used to perform one-dimensional convolution on the numerical sequence input to obtain the trend sequence, and then the trend sequence is subtracted from the original sequence to obtain the seasonal sequence. A robust sparse reconstruction module is constructed, which includes a correlation estimation unit, a threshold filtering unit, a sparse selection unit, and a time delay aggregation unit. The module calculates the correlation scores of the seasonal sequence and the time-shifted sequence and forms a correlation score sequence. Based on a fixed threshold rule, a candidate time delay set with correlation scores not lower than the threshold is retained. A sparse selection method is used to select a preset number of time delay indices with the highest scores from the candidate time delay set. The corresponding time-shifted subsequences are weighted and summed according to their weights to obtain the reconstructed seasonal sequence. An uncertainty prediction head module is constructed, and a mean output unit and a variance output unit are set. The reconstructed seasonal series and trend series are connected in the channel dimension and then input into the uncertainty prediction head module. The mean output unit provides the point prediction results of end-to-end delay, delay jitter and packet loss rate for multiple prediction steps, and the variance output unit provides the variance parameter corresponding to the point prediction results. A feature fusion and output module is constructed, which adds the trend sequence and the reconstructed seasonal sequence point by point to obtain the layer output features. The features are then passed layer by layer according to the stacked structure of the encoder and decoder, and finally outputs the multi-step point prediction results of end-to-end delay, delay jitter and packet loss rate.

6. The multi-protocol wireless communication remote control method based on reinforcement learning according to claim 2, characterized in that, The solution yields softened variable values ​​and action vectors, including: Identify the elements for improving the Autoformer model, including the state set, action set, discount factor, empirical approximation of transition dynamics, fixed cost budget, and fixed decrease factor; Define a nonnegative Lyapunov function and set a barrier function obtained by monotonic transformation of the safety set decision function. Construct a composite function as the Lyapunov-barrier function by summing with fixed weights. At the current state, a descent inequality constraint is established for the expected increment and cost term of the composite function. In discrete time, a forward invariance inequality constraint is established for the safety set decision function. Local linearization is then used to write the inequality constraint as an affine inequality about the action vector, thus obtaining the affine inequality constraint. The prediction result of the previous moment is parametrically linearly mapped to the current state input to obtain the original action vector. A quadratic programming safe projection problem is established, with the quadratic form of the original action vector and the weighted Euclidean distance as the objective function and the affine inequality constraint and the upper and lower bounds of the action as the constraint conditions. The projected action vector that satisfies the constraints is obtained by solving the problem. The upper and lower bounds of the action come from the actual physical and protocol constraints in the communication. The projected action vectors interact with the environment, and the reward samples during the interaction process are collected. The policy loss, the Lyapunov consistency loss of the composite function increment and cost term, and the feasibility penalty of the first norm of the softening variable are calculated through the reward samples. The three are summed with fixed weights to form the total loss and update the action candidate unit parameters and Lyapunov function parameters. Repeat the interaction, projection, and update until a fixed iteration limit or convergence criterion is reached, and output the optimized Autoformer model that satisfies the descent constraint and the safety set invariance constraint.

7. The multi-protocol wireless communication remote control method based on reinforcement learning according to claim 2, characterized in that, The set of predicted results for end-to-end latency, latency jitter, and packet loss rate at multiple future time points includes: Call the optimized Autoformer model and load fixed network structure parameters and training hyperparameters, and set the sliding training window length, prediction step size, batch size, learning rate, number of training rounds and number of early stopping rounds; A sliding window is used to construct training samples and label samples on a standardized dataset, and standardized feature vector sequences and label sequences are generated according to time index; The training loss function is the sum of the weighted absolute errors of end-to-end latency, latency jitter and packet loss rate. The optimized Autoformer model is subjected to batch forward computation, loss calculation and parameter update until the training round limit is reached or the early stopping condition is met. The standardized feature vector sequence and label sequence are input in a rolling manner according to the prediction step size. The end-to-end delay point prediction result, delay jitter point prediction result and packet loss rate point prediction result at multiple future time points are output according to the time index, forming a prediction result set corresponding to the time index.

8. The multi-protocol wireless communication remote control method based on reinforcement learning according to claim 2, characterized in that, The calculation of the comprehensive cost function and the obtaining of the optimal control parameter combination include: The optimization objective is to minimize the comprehensive cost function. The comprehensive cost function is a linear weighted sum of the time average of the end-to-end delay prediction sequence, the time average of the delay jitter prediction sequence, and the time average of the packet loss rate prediction sequence, with non-negative constant weights. A fixed penalty term for the change in protocol switching and link aggregation method is added. The solution is obtained under the constraint that the unit time average of energy consumption and cost does not exceed their respective thresholds. By setting reliability and feasibility constraints, a mixed-integer linear programming problem is established and solved to obtain the combination of control parameters for optimal protocol selection, optimal link aggregation and redundancy, optimal transmit power level, optimal coding error correction level, optimal retransmission limit, and optimal fragmentation strategy.

9. A multi-protocol wireless communication remote control method based on reinforcement learning according to claim 2, characterized in that, The step of sending the execution results back to the data acquisition and preprocessing stage to complete remote control includes: Based on the combination of control parameters, the protocol establishment, session persistence, seamless handover and multi-link aggregation are performed on the multi-protocol wireless communication platform. The primary protocol, backup or parallel links, transmit power level, coding error correction level, retransmission limit and fragmentation are set in sequence. During the execution process, the link status and control command issuance are synchronously scheduled. After execution, end-to-end latency, latency jitter, packet loss rate, energy consumption and cost are collected and recorded in real time. The collected results, along with the corresponding control parameter identifiers, time indexes and link identifiers, are sent back to the data collection and preprocessing stage to update the standardized dataset and form remote control.

Images