Network game acceleration strategy optimization system based on reinforcement learning

Abstract

The invention discloses an online game acceleration strategy optimization system based on reinforcement learning, and the system comprises a network state sensing module which is used for collecting the network state information of a network path and carrying out the normalization processing, and generating a network state vector; the strategy generation module is used for generating probability distribution of network acceleration actions; the strategy execution module is used for constructing a network acceleration strategy instruction set and accelerating the forwarding process of the game data in the network; the return feedback module is used for collecting communication performance feedback data after the acceleration strategy is executed and calculating a return value; and the strategy optimization module is used for performing optimization iteration on the strategy network and the state value function network. According to the method, a near-end strategy optimization algorithm is combined, intelligent modeling of real-time network performance and dynamic acceleration strategy generation are achieved, the method has the advantages of being high in adaptability and strategy stability, and the consistency of the transmission stability, the response speed and the player experience of an online game in a complex environment is remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of computer network technology, and in particular to a network game acceleration strategy optimization system based on reinforcement learning. Background Technology

[0002] With the continuous expansion of the online game user base and the increasing complexity of game scenarios, network transmission performance has a more and more significant impact on player experience. Especially in mobile networks, unstable broadband environments, or cross-regional access scenarios, network problems such as high latency, severe packet loss, and frequent jitter often occur, directly leading to slow game operation response, screen stuttering, and synchronization imbalance, seriously affecting players' immersion and competitive fairness. To alleviate the above problems, existing technologies have proposed a variety of network acceleration methods, commonly including traffic scheduling mechanisms based on fixed routing strategies, CDN acceleration, TCP / UDP protocol optimization, dedicated line relay, and edge node forwarding. These solutions have improved the connection performance of some users to a certain extent.

[0003] Chinese invention patent application CN107277071A, published on October 20, 2017, discloses a method for accelerating online games. This method involves dividing and encapsulating TCP data packets sent by the client into sub-packets at the IP layer, using an LSP client to separate game-related traffic and send it to an acceleration server. When the client and game server are on the same network, direct transmission via NAP nodes is used. If they are in a different network environment, multi-line configuration is used to efficiently forward the sub-packets to the target game server, thus achieving game acceleration across network environments.

[0004] In the above invention applications, the acceleration methods mainly rely on preset rules or static path strategies, which cannot dynamically perceive real-time changes in the network environment and lack the ability to model the complex nonlinear relationship between network status and acceleration strategies. When faced with problems such as sudden network congestion, unstable links, and regional access differences, these static strategies often respond slowly and cannot achieve intelligent and personalized adjustment of acceleration paths and parameters, resulting in unstable acceleration effects or even the opposite effect.

[0005] This invention addresses the lack of dynamism, adaptability, and intelligence in existing online game acceleration strategies by proposing a method for optimizing online game acceleration strategies based on a near-end policy optimization algorithm. By constructing a reinforcement learning agent, the method collects network state data and generates executable acceleration strategies. After strategy execution, a reward signal is constructed based on multi-dimensional performance feedback, guiding continuous iterative updates of the strategy. This achieves joint optimization of multi-dimensional parameters such as path selection, node switching, and protocol adjustment during online game transmission, significantly improving the stability and personalization of acceleration effects and overcoming the shortcomings of existing technologies, such as policy rigidity, slow response, and poor generalization. Summary of the Invention

[0006] One objective of this invention is to propose a network game acceleration strategy optimization system based on reinforcement learning. This invention constructs a closed-loop reinforcement learning system consisting of a network state awareness module, a strategy generation module, a strategy execution module, a reward feedback module, and a strategy optimization module. It employs a multi-dimensional network performance data acquisition and state vector construction mechanism, combined with a reinforcement learning structure based on a near-end policy optimization algorithm, to dynamically decide on acceleration strategies such as path selection, relay node configuration, protocol switching, redundancy control, and traffic adjustment, generating the optimal network transmission strategy. After executing the acceleration strategy, the system collects transmission performance feedback data in real time, constructs a multi-objective reward function, and guides the iterative optimization of the policy network and value function network by pruning the objective function, achieving continuous adaptation to complex network environments. This method effectively solves the problems of static strategies, slow response, and rigid optimization rules in existing network acceleration technologies, significantly improving transmission stability, latency optimization capabilities, and personalized acceleration effects in network game scenarios. It is suitable for intelligent acceleration applications with multiple network architectures and multi-region game connections.

[0007] A network game acceleration strategy optimization system based on reinforcement learning according to an embodiment of the present invention includes: The network state awareness module is used to collect network state information of the network path between the user terminal and the game server and perform normalization processing to generate a network state vector. The policy generation module is used to input the network state vector into the policy network and generate the probability distribution of the network's accelerated actions; The strategy execution module is used to construct a network acceleration strategy instruction set based on the selected action, and accelerate the forwarding process of game data in the network based on the strategy instruction set; The feedback module is used to collect communication performance feedback data after the acceleration strategy is executed, and to calculate the reward value based on the communication performance feedback data. The policy optimization module is used to construct the pruning loss function and the mean squared error loss function. It uses mini-batch sample training to optimize and iterate the policy network and the state value function network respectively, thereby improving the expected return of policy selection and enhancing training convergence.

[0008] Optionally, the steps for collecting network state information of the optional network path between the user terminal and the game server, performing normalization processing, and generating a network state vector are as follows: S11. Perform parallel probing of the N network paths between the user terminal and the game server, and obtain the network performance data of each network path at the current moment; S12. Normalize the collected network performance data. S13. Construct a network state vector from the normalized network performance data. : ; in, For a moment The network state vector below, Let i be the round-trip delay of the i-th network path. Let be the packet loss rate of the i-th network path. Let be the instantaneous jitter value of the i-th network path. Let be the bandwidth utilization of the i-th network path. Let be the real-time load value of the i-th network path, and N be the total number of network paths.

[0009] Optionally, the network performance data for each network path at the current moment includes round-trip delay, packet loss rate, instantaneous jitter value, bandwidth utilization, and real-time load value.

[0010] Optionally, the steps for inputting the network state vector into the policy network to generate the probability distribution of the network's accelerated actions are as follows: S21. Transfer the network state vector Input to policy network The probability of the current action is calculated through forward propagation. ,in, express The specific action selected at each moment, i.e., the choice of network acceleration strategy, and the probability of the old strategy are recorded. The policy network consists of an input layer, multiple hidden layers, and an output layer. The input layer receives the network state vector, each hidden layer is a fully connected layer, and the output layer outputs each action in the action space through the Softmax function. The probability of; S22, Transfer the state vector Input state value function network Output the current state estimate The State Value Function Network (SFCN) has a similar structure to the Policy Network, consisting of an input layer, multiple hidden layers, and an output layer. The input layer shares the same input dimension as the Policy Network, and the output layer is a single node that represents the estimated value of the current state.

[0011] Optionally, the steps for constructing a network acceleration strategy instruction set based on the selected action and accelerating the forwarding process of game data in the network based on the strategy instruction set are as follows: S31, Based on the current action probability The action at the current moment is determined using the maximum probability selection method. ,in, A represents the action space, which contains multiple network acceleration strategy options; S32, Action Mapped to network acceleration policy instruction set The instruction set is used to control data flow scheduling strategies, including specifying data transmission paths, configuring relay node parameters, selecting communication protocols, and adjusting data transmission behavior. S33, According to the policy instruction set Configure network parameters; S34. Accelerate the forwarding of game data based on the optimized data transmission channel.

[0012] Optionally, the policy instruction set Including acceleration path number Relay node number Protocol type identifier Redundant coding switch status Data packet sending interval parameter ,in, K is the total number of available paths. M represents the number of deployable nodes. , This indicates whether to enable the redundancy error correction mechanism. Used to control transmission frequency and flow control behavior.

[0013] Optionally, the step of using the policy instruction set... The steps to configure network parameters are as follows: S331, According to the acceleration path number Send a path configuration command to the network routing control module to change the game data transmission path to... ; S332, Based on the relay node number Schedule the designated relay server node to establish an end-to-end forwarding channel, which supports multi-hop routing and parallel link aggregation modes; S333, Identifier based on protocol type Switching the data transmission protocol stack controls how data packets are transmitted via the selected protocol. Transmission is performed to adapt to the connection stability and latency requirements under different network scenarios; S334, When the redundancy coding switch is in state When the forward error correction mechanism is activated, the original game data packets are subjected to redundant encoding processing to generate a set of redundant data packets; when the redundant encoding switch is active... If this occurs, the forward error correction mechanism will be disabled; S335, According to the data packet transmission interval parameter The system limits the rate at which game data packets are sent to ensure that the network load is within an acceptable range. At the same time, it adjusts the distribution characteristics of the data flow in the path to mitigate the risk of congestion caused by sudden traffic surges.

[0014] Optionally, the steps for collecting communication performance feedback data after the acceleration strategy is implemented and calculating the reward value based on the feedback data are as follows: S41. Real-time collection of communication performance feedback data between the user terminal and the game server, the communication performance feedback data including actual round-trip latency. Packet loss rate jitter value Game control command response time Data flow stability score The stability index represents the uniformity and orderliness of data packet arrival times; S42. Calculate the return value by comparing the current feedback data with historical benchmark performance data. : ; in, The weighting factors for each indicator satisfy the following conditions: , For historical round-trip time delay, Historical packet loss rate This refers to the response time for historical game control commands.

[0015] Optionally, the steps for constructing the pruning loss function and the mean squared error loss function, and optimizing and iterating the policy network and the state value function network using mini-batch training, are as follows: S51, Based on Return Value Valuation based on current state Calculate the advantage function: ; in, It indicates the superiority of the current action relative to the baseline state value, and is used to evaluate the performance of the current strategy decision; S52, By trimming the objective function The gradient ascent method is used to optimize and iterate the policy network, updating it to improve the expected return of the action selection policy and pruning the objective function. The calculation method is as follows: ; in, The probability ratio of the strategies. , The probability of the old strategy The probability of the current policy. For hyperparameters, The `clip()` function controls the magnitude of policy updates and limits the range of policy changes to prevent excessive updates from causing training instability. The `clip()` function is calculated as follows: ; S53, By minimizing the mean square error loss function Optimize the state-value function network parameters, the mean square error loss function Defined as: ; in, Estimating the current state, This is the return value.

[0016] The beneficial effects of this invention are: 1. This invention introduces a reinforcement learning architecture based on a near-end policy optimization algorithm to construct an intelligent network game acceleration system with perception, decision-making, execution, feedback, and optimization capabilities. It realizes dynamic and adaptive adjustment of acceleration strategies in complex and variable network environments. Through high-dimensional feature extraction of network states and construction of state vectors, the system can perceive key performance indicators such as network latency, packet loss rate, jitter, bandwidth utilization, and node load in real time.

[0017] 2. This invention combines policy network and state-value function network collaborative modeling to achieve joint optimization of multiple acceleration actions such as path, relay node, protocol parameters, and transmission control. By using the designed advantage function and pruning objective function to update the policy network, the convergence speed and stability of the policy are effectively improved, and the generalization ability and robustness under non-stationary network conditions are significantly enhanced.

[0018] 3. Compared with traditional rule-based or static policy-based acceleration methods, this invention has significant advantages in intelligence and adaptability. It can automatically select the optimal acceleration policy according to real-time changes in the network environment, avoiding performance degradation caused by fixed paths or manual configuration. At the same time, by integrating multiple evaluation indicators such as latency, packet loss, response time and flow stability through the reward function, the system can accurately reflect the policy execution effect and promote the evolution of policies towards personalization and high performance. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a block diagram of the network game acceleration strategy optimization system based on reinforcement learning proposed in this invention. Figure 2This is a flowchart illustrating the configuration of network parameters based on a policy instruction set, as proposed in this invention. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0021] refer to Figure 1 and Figure 2 A network game acceleration strategy optimization system based on reinforcement learning includes: The network state awareness module is used to collect network state information of the network path between the user terminal and the game server and perform normalization processing to generate a network state vector. The policy generation module is used to input the network state vector into the policy network and generate the probability distribution of the network's accelerated actions; The strategy execution module is used to construct a network acceleration strategy instruction set based on the selected action, and accelerate the forwarding process of game data in the network based on the strategy instruction set; The feedback module is used to collect communication performance feedback data after the acceleration strategy is executed, and to calculate the reward value based on the communication performance feedback data. The policy optimization module is used to construct the pruning loss function and the mean squared error loss function. It uses mini-batch sample training to optimize and iterate the policy network and the state value function network respectively, thereby improving the expected return of policy selection and enhancing training convergence.

[0022] In this embodiment, the steps for collecting network state information of the selectable network path between the user terminal and the game server, performing normalization processing, and generating a network state vector are as follows: S11. Perform parallel probing of the N network paths between the user terminal and the game server, and obtain the network performance data of each network path at the current moment; S12. Normalize the collected network performance data. S13. Construct a network state vector from the normalized network performance data. : ; in, For a moment The network state vector below, Let i be the round-trip delay of the i-th network path. Let be the packet loss rate of the i-th network path. Let be the instantaneous jitter value of the i-th network path. Let be the bandwidth utilization of the i-th network path. Let be the real-time load value of the i-th network path, and N be the total number of network paths.

[0023] In this embodiment, the network performance data of each network path at the current moment includes round-trip latency, packet loss rate, instantaneous jitter value, bandwidth utilization, and real-time load value.

[0024] In this embodiment, the steps for inputting the network state vector into the policy network and generating the probability distribution of network acceleration actions are as follows: S21. Transfer the network state vector Input to policy network The probability of the current action is calculated through forward propagation. ,in, express The specific action selected at each moment, i.e., the choice of network acceleration strategy, and the probability of the old strategy are recorded. The policy network consists of an input layer, multiple hidden layers, and an output layer. The input layer receives the network state vector, each hidden layer is a fully connected layer, and the output layer outputs each action in the action space through the Softmax function. The probability of; S22, Transfer the state vector Input state value function network Output the current state estimate The State Value Function Network (SFCN) has a similar structure to the Policy Network, consisting of an input layer, multiple hidden layers, and an output layer. The input layer shares the same input dimension as the Policy Network, and the output layer is a single node that represents the estimated value of the current state.

[0025] In this embodiment, the steps for constructing a network acceleration strategy instruction set based on the selected action and accelerating the forwarding process of game data in the network based on the strategy instruction set are as follows: S31, Based on the current action probability The action at the current moment is determined using the maximum probability selection method. ,in, A represents the action space, which contains multiple network acceleration strategy options; S32, Action Mapped to network acceleration policy instruction set The instruction set is used to control data flow scheduling strategies, including specifying data transmission paths, configuring relay node parameters, selecting communication protocols, and adjusting data transmission behavior. S33, According to the policy instruction set Configure network parameters; S34. Accelerate the forwarding of game data based on the optimized data transmission channel.

[0026] In this embodiment, the strategy instruction set Including acceleration path number Relay node number Protocol type identifier Redundant coding switch status Data packet sending interval parameter ,in, K is the total number of available paths. M represents the number of deployable nodes. , This indicates whether to enable the redundancy error correction mechanism. Used to control transmission frequency and flow control behavior.

[0027] In this embodiment, according to the policy instruction set The steps to configure network parameters are as follows: S331, According to the acceleration path number Send a path configuration command to the network routing control module to change the game data transmission path to... ; S332, Based on the relay node number Schedule the designated relay server node to establish an end-to-end forwarding channel, which supports multi-hop routing and parallel link aggregation modes; S333, Identifier based on protocol type Switching the data transmission protocol stack controls how data packets are transmitted via the selected protocol. Transmission is performed to adapt to the connection stability and latency requirements under different network scenarios; S334, When the redundancy coding switch is in state When the forward error correction mechanism is activated, the original game data packets are subjected to redundant encoding processing to generate a set of redundant data packets; when the redundant encoding switch is active... If this occurs, the forward error correction mechanism will be disabled; S335, According to the data packet transmission interval parameter The system limits the rate at which game data packets are sent to ensure that the network load is within an acceptable range. At the same time, it adjusts the distribution characteristics of the data flow in the path to mitigate the risk of congestion caused by sudden traffic surges.

[0028] In this embodiment, the steps for collecting communication performance feedback data after the acceleration strategy is executed and calculating the reward value based on the feedback data are as follows: S41. Real-time collection of communication performance feedback data between the user terminal and the game server, the communication performance feedback data including actual round-trip latency. Packet loss rate jitter value Game control command response time Data flow stability score The stability index represents the uniformity and orderliness of data packet arrival times; S42. Calculate the return value by comparing the current feedback data with historical benchmark performance data. : ; in, The weighting factors for each indicator satisfy the following conditions: , For historical round-trip time delay, Historical packet loss rate This refers to the response time for historical game control commands.

[0029] In this embodiment, the steps for constructing the pruning loss function and the mean squared error loss function, and optimizing and iterating the policy network and the state value function network using mini-batch training are as follows: S51, Based on Return Value Valuation based on current state Calculate the advantage function: ; in, It indicates the superiority of the current action relative to the baseline state value, and is used to evaluate the performance of the current strategy decision; S52, By trimming the objective function Gradient ascent is used to iteratively optimize the policy network, updating it to improve the expected return of the action selection policy and pruning the objective function. The calculation method is as follows: ; in, The probability ratio of the strategies. , The probability of the old strategy The probability of the current policy. For hyperparameters, The `clip()` function controls the magnitude of policy updates and limits the range of policy changes to prevent excessive updates from causing training instability. The `clip()` function is calculated as follows: ; S53, By minimizing the mean square error loss function Optimize the state-value function network parameters, the mean square error loss function Defined as: ; in, Estimating the current state, This is the return value.

[0030] Example In this embodiment, the mobile version of a large-scale cross-regional MMORPG was selected as the experimental scenario. This game has a large number of real-time interactive behaviors, including dungeon battles, world bosses, guild wars, and multi-player voice collaboration. Players need to frequently synchronize data and commands, and are extremely sensitive to network latency. The experiment was conducted from July 1 to July 10, 2025 in Beijing, Shanghai, Chengdu, and Guangzhou, covering the 4G / 5G network environments of the three major operators: China Telecom, China Unicom, and China Mobile. The experiment was deployed and tested with real users, with a total of 2,000 participants.

[0031] In this scenario, the network game acceleration strategy optimization system based on the near-end policy optimization algorithm proposed in this invention was deployed. The system is integrated into the underlying network module of the game client and runs as an intelligent acceleration plugin. The system first collects the network path data currently available on the device in real time at the start of each game session, extracts key state feature parameters (including path latency, jitter, packet loss rate, bandwidth, node load, etc.), constructs a state vector, and inputs it into the trained policy network for inference calculation. The system generates policy actions including node selection, path configuration, protocol type, redundancy coding, and transmission control, automatically executes the policy and opens the data acceleration channel. During the session, the system continuously collects network performance feedback and updates the policy model to improve its adaptability to dynamic network environments.

[0032] In this experiment, the experimental group of users enabled the intelligent acceleration system of this invention, while the control group of users used a traditional VPN acceleration solution based on static path rules. The network game experience indicators of the two groups of users were compared through the following dimensions: average round-trip latency, packet loss rate, game operation response time, stuttering rate (frame interruption frequency), path switching number, etc. At the same time, the user satisfaction rating was recorded to reflect the subjective experience improvement brought about by system optimization.

[0033] Experimental data shows that in complex network environments, traditional VPN solutions suffer from problems such as delayed policy response, frequent path switching, and abnormal data forwarding when network quality deteriorates. In contrast, the system of this invention can autonomously optimize policies based on real-time network conditions, significantly improving stability and smoothness. For example, in Guangzhou, the average latency of players using the acceleration system of this invention decreased from 83ms to 52ms, the packet loss rate decreased from 1.4% to 0.4%, and the game lag rate decreased by about 65%. In Chengdu, the system's protocol switching mechanism automatically switched some links from UDP to QUIC, successfully addressing the high packet loss problem and stabilizing the latency within 60ms, while player operation response time decreased from an average of 420ms to 260ms.

[0034] Furthermore, during the 10-day testing period, the experimental group players switched paths an average of 2.1 times per hour, while the control group switched paths an average of 6.3 times per hour. This fully demonstrates that the strategy generation mechanism of the present invention has stronger strategy stability and environmental adaptability, and reduces performance fluctuations caused by frequent switching. In terms of user satisfaction ratings, the experimental group scored an average of 4.4 / 5.0, while the control group scored 3.1 / 5.0.

[0035] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A network game acceleration strategy optimization system based on reinforcement learning, characterized in that, include: The network state awareness module is used to collect network state information of the network path between the user terminal and the game server and perform normalization processing to generate a network state vector. The policy generation module is used to input the network state vector into the policy network and generate the probability distribution of the network's accelerated actions; The strategy execution module is used to construct a network acceleration strategy instruction set based on the selected action, and accelerate the forwarding process of game data in the network based on the strategy instruction set; The feedback module is used to collect communication performance feedback data after the acceleration strategy is executed, and to calculate the reward value based on the communication performance feedback data. The policy optimization module is used to construct the pruning loss function and the mean squared error loss function, and to optimize and iterate the policy network and the state value function network respectively using a mini-batch sample training method.

2. The network game acceleration strategy optimization system based on reinforcement learning according to claim 1, characterized in that, The steps for collecting network state information of the selectable network path between the user terminal and the game server, performing normalization processing, and generating a network state vector are as follows: S11. Perform parallel probing of the N network paths between the user terminal and the game server, and obtain the network performance data of each network path at the current moment; S12. Normalize the collected network performance data. S13. Construct a network state vector from the normalized network performance data. : ； in, For a moment The network state vector below, Let i be the round-trip delay of the i-th network path. Let be the packet loss rate of the i-th network path. Let be the instantaneous jitter value of the i-th network path. Let be the bandwidth utilization of the i-th network path. Let be the real-time load value of the i-th network path, and N be the total number of network paths.

3. The network game acceleration strategy optimization system based on reinforcement learning according to claim 2, characterized in that, The network performance data for each network path at the current moment includes round-trip delay, packet loss rate, instantaneous jitter, bandwidth utilization, and real-time load.

4. The network game acceleration strategy optimization system based on reinforcement learning according to claim 1, characterized in that, The steps for inputting the network state vector into the policy network to generate the probability distribution of network acceleration actions are as follows: S21. Transfer the network state vector Input to policy network The probability of the current action is calculated through forward propagation. ,in, express The specific action selected at each moment, and the probability of the old strategy recorded. ; S22, Transfer the state vector Input state value function network Output the current state estimate .

5. The network game acceleration strategy optimization system based on reinforcement learning according to claim 1, characterized in that, The steps for constructing a network acceleration strategy instruction set based on the selected action and accelerating the forwarding process of game data in the network based on the strategy instruction set are as follows: S31, Based on the current action probability The action at the current moment is determined using the maximum probability selection method. ,in, A represents the action space, which contains multiple network acceleration strategy options; S32, Action Mapped to network acceleration policy instruction set ; S33, According to the policy instruction set Configure network parameters; S34. Accelerate the forwarding of game data based on the optimized data transmission channel.

6. The network game acceleration strategy optimization system based on reinforcement learning according to claim 5, characterized in that, The policy instruction set Including acceleration path number Relay node number Protocol type identifier Redundant coding switch status Data packet sending interval parameter .

7. The network game acceleration strategy optimization system based on reinforcement learning according to claim 5, characterized in that, According to the policy instruction set The steps to configure network parameters are as follows: S331, According to the acceleration path number Send a path configuration command to the network routing control module to change the game data transmission path to... ; S332, Based on the relay node number Schedule the designated relay server node to establish an end-to-end forwarding channel; S333, Identifier based on protocol type Switching the data transmission protocol stack controls how data packets are transmitted via the selected protocol. Transmit; S334, When the redundancy coding switch is in state When this happens, the forward error correction mechanism is activated to perform redundant encoding on the original game data packets, generating a set of redundant data packets; When the redundancy coding switch is in state If this occurs, the forward error correction mechanism will be disabled; S335, According to the data packet transmission interval parameter It limits the rate at which game data packets are sent and adjusts the distribution characteristics of the data stream along the path.

8. The network game acceleration strategy optimization system based on reinforcement learning according to claim 1, characterized in that, The steps for collecting communication performance feedback data after the acceleration strategy is implemented and calculating the reward value based on the feedback data are as follows: S41. Real-time collection of communication performance feedback data between the user terminal and the game server, the communication performance feedback data including actual round-trip latency. Packet loss rate jitter value Game control command response time Data flow stability score ; S42. Calculate the return value by comparing the current feedback data with historical benchmark performance data. : ； in, The weighting factors for each indicator satisfy the following conditions: , For historical round-trip time delay, Historical packet loss rate This refers to the response time for historical game control commands.

9. The network game acceleration strategy optimization system based on reinforcement learning according to claim 1, characterized in that, The steps for constructing the pruning loss function and the mean squared error loss function, and optimizing and iterating the policy network and the state value function network using mini-batch training are as follows: S51, Based on Return Value Valuation based on current state Calculate the advantage function: ； in, It indicates the superiority of the current action relative to the baseline state. S52, By trimming the objective function The gradient ascent method is used to iteratively optimize the policy network and prune the objective function. The calculation method is as follows: ； in, The probability ratio of the strategies. , For the old strategy probability, The probability of the current policy. As a hyperparameter, the `clip()` function is used to limit the range of policy changes, preventing excessive updates from causing training instability. The `clip()` function is calculated as follows: ； S53, By minimizing the mean square error loss function Optimize the state-value function network parameters, the mean square error loss function Defined as: ； in, Estimating the current state, This is the return value.

Images