High-speed environment network switching optimization system and method based on meta-policy gradient

Abstract

The invention discloses a high-speed environment network switching optimization system and method based on meta-policy gradient, and relates to the technical field of wireless communication, and the system comprises a data collection module which is responsible for collecting the network environment and equipment state information of user equipment in real time, carrying out the preprocessing and filtering operation, and outputting the information; the model training module is used for training a switching strategy model on a plurality of simulation tasks by using historical data by adopting a double-layer optimization architecture based on a meta-strategy gradient, so that the model can quickly adapt to new switching requirements in different environments; the online decision-making module is responsible for receiving the output data of the data collection module, calling a model to generate an optimal switching strategy, then executing the strategy and monitoring a switching result; and the self-correction module is responsible for receiving the switching result of the online decision-making module and performing short-term fine adjustment and periodic updating on the model so as to ensure the long-term adaptability of the model in different environments. The method is used for fast network switching in the high-speed mobile environment.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technologies, and more specifically, to a high-speed environment network handover optimization system and method based on meta-policy gradient. Background Art

[0002] With the rapid development of mobile communication technologies and the increasing demand for high-speed data communication from users, the network connection stability of mobile devices in high-speed moving environments such as high-speed trains and subways has become an urgent problem to be solved. In such scenarios, due to high-speed movement, user equipment needs to frequently switch base stations, resulting in unstable connections, problems such as call interruptions and data transmission interruptions, which greatly affect the user experience.

[0003] Traditional network handover mechanisms usually rely on fixed signal strength thresholds or preset handover strategies, but these methods are difficult to effectively handle the frequently changing network conditions in high-speed moving environments. In environments such as high-speed trains and subways, the device handover rate is high, and the rapid attenuation of network signals, the existence of coverage blind spots, and fluctuations in base station loads may all lead to connection interruptions or handover failures. Especially in scenarios with a high density of people, base station resources are contended by a large number of user devices, further exacerbating the problems of signal congestion and degraded connection quality.

[0004] In addition, in recent years, machine learning and reinforcement learning technologies have been applied to optimize network handover strategies, but most methods require a large amount of environmental data and a long learning time, and it is difficult to adapt to frequent environmental changes in real time in high-speed scenarios. In this situation, there is an urgent need for a handover optimization method with fast adaptability and self-correction ability to address the challenges in high-speed movement and complex network environments. Summary of the Invention

[0005] Aiming at the problem that user equipment frequently switches base stations in high-speed moving environments such as high-speed trains and subways, resulting in connection interruptions, the present invention provides a high-speed environment network handover optimization system and method based on meta-policy gradient.

[0006] In a first aspect, the present invention provides a high-speed environment network handover optimization system based on meta-policy gradient, and the technical solution adopted to solve the above technical problems is as follows:

[0007] A high-speed environment network handover optimization system based on meta-policy gradient, comprising:

[0008] A data collection module, responsible for real-time monitoring and collection of the network environment and device status information of user equipment, and generating data suitable for model training and optimization after data preprocessing and filtering operations;

[0009] The model training module adopts a two-layer optimization architecture based on meta-policy gradients and uses historical data to train the switching strategy model on multiple simulation tasks, so that the switching strategy model can quickly adapt to new switching requirements in different environments;

[0010] The online decision-making module is responsible for receiving the real-time data output by the data collection module, calling the trained switching strategy model to generate the optimal switching strategy based on the received real-time data, then executing the optimal switching strategy, monitoring the switching results, and feeding back to the self-correction module;

[0011] The self-correction module is responsible for receiving the switching results fed back by the online decision-making module, and performing short-term fine-tuning and periodic updates on the switching strategy model to ensure the long-term adaptability of the model in different environments.

[0012] Optionally, the data collection modules involved specifically include:

[0013] The signal measurement unit is used to monitor the strength of different base station signals using wireless signal measurement tools, and record the signal strength of the current base station and the signal conditions of each neighboring base station in real time;

[0014] A load monitoring unit, used to monitor the load status of the current base station;

[0015] An information acquisition unit, used to acquire the position and speed information of the user equipment through a GPS module and an inertial sensor;

[0016] The preprocessing unit is used to clean and normalize the signal strength, base station load, and location information collected by the above-mentioned units, remove noise and outliers, and generate data suitable for model training and optimization.

[0017] Optionally, the model training modules involved specifically include:

[0018] Constructing a generation unit, which is used to construct a variety of simulated network environments to generate simulation tasks similar to the actual environment;

[0019] The inner layer optimization unit is used to perform inner layer optimization using the policy gradient algorithm on each simulation task and adjust the policy parameters of the switching policy model to maximize the reward function of the switching policy model;

[0020] The outer optimization unit is used to calculate the overall meta-loss function using the task results after the inner optimization, and to update the initial policy parameters of the switching policy model using the meta-policy gradient on multiple tasks, so that the switching policy model can quickly adapt to the new task, thereby outputting the optimal switching policy in the new task environment;

[0021] A repeating execution unit, which is used to continuously repeat the dual iteration process of the inner optimization unit and the outer optimization unit until the overall meta-loss converges, completing the training of the handover policy model;

[0022] A model deployment unit, which is used to deploy the trained handover policy model to user devices or edge nodes for online use.

[0023] Further optionally, the inner optimization unit specifically performs the following operations: according to the handover decision in the current state, assign reward values to each action; through the backpropagation algorithm, calculate the policy gradient for each task, and update the policy parameters of the handover policy model to adapt to the current task; repeat the above update steps multiple times until the handover policy model converges in this task environment;

[0024] The outer optimization unit specifically performs the following operations: on all tasks, compare and analyze the initial policy with the adjusted policy results, and construct a meta-loss function; use the meta-policy gradient optimization technique to adjust the initial policy of the handover policy model so that the handover policy model can quickly find the optimal policy with the fewest parameter update steps when facing a new task environment.

[0025] Further optionally, the self-correction module specifically includes:

[0026] A result receiving unit, which is used to receive the handover results fed back by the online decision module, and use the incremental learning method to fine-tune the local policy parameters of the handover policy model to continuously provide the optimal handover policy in the current network environment;

[0027] A timing update unit, which is used to receive the real-time data output by the data collection module, add the received real-time data to the training set at regular intervals, use the data in the training set as input, and call the model training module to periodically optimize the policy parameters of the handover policy model to maintain the adaptability of the handover policy model to the new environment;

[0028] An anomaly simulation module, which is used to receive the real-time data output by the data collection module, simulate and generate data specifically for the anomaly scenario when a certain monitoring data is abnormal, and then call the model training module to specifically optimize the policy parameters of the handover policy model to ensure the handover success rate and stability in this anomaly scenario.

[0029] In a second aspect, the present invention provides a high-speed environment network handover optimization method based on meta-policy gradient. The technical solution adopted to solve the above technical problems is as follows:

[0030] A high-speed environment network handover optimization method based on meta-policy gradient, which includes the following steps:

[0031] S1. Monitor and collect the network environment and device status information of the user device in real time. After performing data preprocessing and filtering operations, generate data suitable for model training and optimization.

[0032] S2. Adopt a two-layer optimization architecture based on meta-policy gradient. Use historical data to train the handover policy model on multiple simulation tasks, so that the handover policy model can quickly adapt to new handover requirements in different environments.

[0033] S3. Based on the real-time data output in step S1, the trained handover policy model generates the optimal handover policy. Subsequently, execute the optimal handover policy, monitor the handover result, and feedback it to step S4.

[0034] S4. Receive the handover result feedback from step S3 and the real-time data output in step S1, and perform short-term fine-tuning and periodic update on the handover policy model to ensure the long-term adaptability of the model in different environments.

[0035] Optionally, step S1 specifically includes:

[0036] Use a wireless signal measurement tool to monitor the strength of signals from different base stations, and record the signal strength of the current base station and the signal conditions of each neighboring base station in real time.

[0037] Monitor the load status of the current base station.

[0038] Obtain the location and speed information of the user device through the GPS module and inertial sensors.

[0039] Clean and normalize the collected signal strength, base station load, and location information, remove noise and outliers, and generate data suitable for model training and optimization.

[0040] Further optionally, step S2 specifically includes:

[0041] Construct multiple simulated network environments to generate simulation tasks similar to the actual environment.

[0042] Perform inner-layer optimization on each simulation task using the policy gradient algorithm, adjust the policy parameters of the handover policy model to maximize the reward function of the handover policy model.

[0043] For outer-layer optimization, use the task results after inner-layer optimization, calculate the overall meta-loss function, and update the initial policy parameters of the handover policy model using meta-policy gradient on multiple tasks, so that the handover policy model can quickly adapt to new tasks, and thus output the optimal handover policy in the new task environment.

[0044] Continuously repeat the double-iteration process of inner-layer optimization and outer-layer optimization until the overall meta-loss converges, and complete the training of the handover policy model.

[0045] Deploy the trained handover policy model on the user device or edge node for online use.

[0046] Further optionally, the specific inner-layer optimization process is as follows: According to the handover decision in the current state, assign a reward value to each action; Through the backpropagation algorithm, calculate the policy gradient for each task, and update the policy parameters of the handover policy model to adapt to the current task; Repeat the above update steps multiple times until the handover policy model converges in this task environment;

[0047] The specific outer-layer optimization process is as follows: On all tasks, compare and analyze the initial policy and the adjusted policy results to construct a meta-loss function; Use the meta-policy gradient optimization technique to adjust the initial policy of the handover policy model so that the handover policy model can quickly find the optimal policy with the fewest parameter update steps when facing a new task environment.

[0048] Further optionally, step S4 involved specifically includes:

[0049] Receive the handover result feedback from step S3, and use the incremental learning method to fine-tune the local policy parameters of the handover policy model to continuously provide the optimal handover policy in the current network environment;

[0050] Receive the real-time data output from step S1, add the received real-time data to the training set at regular intervals, use the data in the training set as input, and call the model training module to periodically optimize the policy parameters of the handover policy model to maintain the adaptability of the handover policy model to the new environment;

[0051] Receive the real-time data output from step S1, simulate and generate data specifically for the abnormal scenario when a certain monitoring data is abnormal, and then call the model training module to specifically optimize the policy parameters of the handover policy model to ensure the handover success rate and stability in this abnormal scenario.

[0052] A high-speed environment network handover optimization system and method based on meta-policy gradient according to the present invention has the beneficial effects compared with the prior art:

[0053] 1. The present invention introduces a meta-policy gradient algorithm to construct a switching strategy model with rapid learning and adaptation capabilities, then quickly adjusts the strategy parameters of the switching strategy model through inner layer optimization, and performs self-correction and optimization upgrade of the switching strategy model through outer layer optimization, so that the switching strategy model can quickly converge to the optimal strategy in different network environments, which can effectively reduce network interruptions during the switching process, improve communication quality and user experience, and has high applicability and promotion value, and solves the problem of frequent switching of base stations by user equipment in high-speed mobile environments such as high-speed rail and subway, resulting in connection interruptions;

[0054] 2. The present invention uses the meta-policy gradient algorithm to perform outer optimization on the switching strategy model based on the inner optimization, so that the initial strategy parameters of the switching strategy model can quickly adapt to the new environment. This two-layer optimization structure significantly reduces the training time and required update steps of the switching strategy model under new tasks, ensuring that the switching strategy model quickly converges to the optimal switching strategy in a high-speed mobile environment;

[0055] 3. The present invention generates an optimal switching strategy through real-time analysis of multiple factors such as signal strength, base station load, and user location. The switching strategy model can switch to a better base station in time when the connection quality deteriorates, thereby greatly reducing network interruptions and disconnections and improving the stability of network connections;

[0056] 4. The present invention performs short-term fine-tuning and periodic updates on the switching strategy model based on feedback results and real-time data, which can ensure the long-term adaptability of the model in different environments; especially in abnormal load or high-density crowd environments, the switching strategy model maintains a high level of switching performance through periodic updates and adaptive optimization, which can improve the robustness and long-term adaptability of the switching strategy model. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Attached Figure 1 is a module connection block diagram of the first embodiment of the present invention;

[0058] Attached Figure 2 It is a schematic diagram of the method flow of embodiment 2 of the present invention. DETAILED DESCRIPTION

[0059] In order to make the technical solution, the technical problem solved and the technical effect of the present invention more clearly understood, the technical solution of the present invention is clearly and completely described below in conjunction with specific embodiments.

[0060] Embodiment 1:

[0061] Reference Figure 1 , this embodiment proposes a high-speed environment network switching optimization system based on meta-strategy gradient, which includes:

[0062] The data collection module is responsible for real-time monitoring and collecting the network environment and device status information of the user device, and generating data suitable for model training and optimization after data preprocessing and filtering operations;

[0063] The model training module adopts a two-layer optimization architecture based on meta-policy gradients, and uses historical data to train the handover policy model on multiple simulation tasks, enabling the handover policy model to quickly adapt to new handover requirements in different environments;

[0064] The online decision-making module is responsible for receiving the real-time data output by the data collection module, calling the trained handover policy model to generate the optimal handover policy based on the received real-time data, then executing the optimal handover policy, monitoring the handover result, and feeding it back to the self-correction module; for example, the policy may include maintaining the current connection, switching to a neighboring base station, or adjusting power control, etc. If the generated optimal handover policy is to perform a handover operation, the handover request is immediately executed to connect the user device to the optimal base station. During the handover process, the connection status is continuously monitored, and the success rate, latency, and number of network interruptions of the handover are recorded;

[0065] The self-correction module is responsible for receiving the handover result feedback from the online decision-making module, performing short-term fine-tuning and periodic updates on the handover policy model to ensure the long-term adaptability of the model in different environments.

[0066] In this embodiment, the data collection module specifically includes:

[0067] The signal measurement unit is used to monitor the strength of signals from different base stations using wireless signal measurement tools (such as RSRP, RSSI, SINR, etc.), and record the signal strength of the current base station and the signal conditions of each neighboring base station in real time;

[0068] The load monitoring unit is used to monitor the load status of the current base station (such as the number of access users, data throughput, etc.) and judge the load level of the base station;

[0069] The information acquisition unit is used to obtain the location and speed information of the user device through the GPS module and inertial sensors; at this time, in order to ensure the real-time calculation, low-latency location information acquisition can be implemented on the device, and the moving trajectory is smoothed through algorithms such as Kalman filtering to improve the accuracy of location estimation;

[0070] The preprocessing unit is used to clean and normalize the signal strength, base station load, and location information collected by the foregoing units, remove noise and outliers, and generate data suitable for model training and optimization.

[0071] In this embodiment, the model training module specifically includes:

[0072] A construction generation unit for constructing a variety of simulated network environments to generate simulated tasks similar to the actual environment;

[0073] An inner layer optimization unit for performing inner layer optimization on each simulated task using a policy gradient algorithm (such as REINFORCE or PPO), adjusting the policy parameters of the handover policy model to maximize the reward function of the handover policy model (e.g., reducing interruptions and delays). The specific implementation operations of the inner layer optimization unit include: assigning a reward value to each action according to the handover decision in the current state; calculating the policy gradient for each task through the backpropagation algorithm, and updating the policy parameters of the handover policy model to adapt to the current task; repeating the above update steps multiple times until the handover policy model converges in this task environment;

[0074] An outer layer optimization unit for calculating the overall meta-loss function using the task results optimized by the inner layer, and updating the initial policy parameters of the handover policy model using the meta-policy gradient on multiple tasks, so that the handover policy model can quickly adapt in new tasks, thereby outputting the optimal handover policy in the new task environment. The specific implementation operations of the outer layer optimization unit include: comparing and analyzing the initial policy with the adjusted policy results on all tasks to construct a meta-loss function; using the meta-policy gradient optimization technique to adjust the initial policy of the handover policy model so that the handover policy model can quickly find the best policy with the fewest parameter update steps when facing a new task environment;

[0075] A repeated execution unit for continuously repeating the double iteration process of the inner layer optimization unit and the outer layer optimization unit until the overall meta-loss converges, completing the training of the handover policy model; specifically, after each inner layer optimization is completed, its results are passed as input to the outer layer optimization. After the outer layer optimization adjusts the initial parameters, it provides a new starting point for the next round of inner layer optimization. This process repeats continuously, continuously improving the policy parameters of the handover policy model until the overall meta-loss converges. At this time, the performance of the handover policy model in new tasks reaches a stable and satisfactory state, with good generalization ability and fast adaptation ability;

[0076] A model deployment unit for deploying the trained handover policy model to user devices or edge nodes for online use.

[0077] In this embodiment, the self-correction module specifically includes:

[0078] A result receiving unit for receiving the handover results feedback by the online decision module, and fine-tuning the local policy parameters of the handover policy model using an incremental learning method to continuously provide the optimal handover policy in the current network environment;

[0079] A timing update unit, which is used to receive real-time data output by the data collection module (such as the current base station signal strength, the load of neighboring base stations, the device location and speed, etc.), add the received real-time data to the training set at regular intervals, use the data in the training set as input, and call the model training module to periodically optimize the policy parameters of the handover policy model to maintain the adaptability of the handover policy model to the new environment;

[0080] An anomaly simulation module, which is used to receive real-time data output by the data collection module (such as the current base station signal strength, the load of neighboring base stations, the device location and speed, etc.), simulate and generate data specifically for an abnormal scenario (such as the overload of the base station load during peak hours) when a certain monitoring data is abnormal (such as the overload of the base station load during peak hours), and then call the model training module to specifically optimize the policy parameters of the handover policy model to ensure the handover success rate and stability in this abnormal scenario (such as the overload of the base station load during peak hours).

[0081] Embodiment 2:

[0082] Combined with the attached Figure 2 In this embodiment, a high-speed environment network handover optimization method based on meta-policy gradient is proposed, which includes the following steps:

[0083] S1. Real-time monitor and collect the network environment and device status information of the user equipment, and generate data suitable for model training and optimization after data preprocessing and filtering operations, specifically including:

[0084] Use wireless signal measurement tools (such as RSRP, RSSI, SINR, etc.) to monitor the strength of different base station signals, and record the signal strength of the current base station and the signal conditions of each neighboring base station in real time;

[0085] Monitor the load status of the current base station (such as the number of access users, data throughput, etc.) and judge the load level of the base station;

[0086] Obtain the location and speed information of the user equipment through the GPS module and inertial sensors; in order to ensure the real-time calculation, low-latency location information acquisition can be implemented on the device, and the moving trajectory can be smoothed through algorithms such as Kalman filtering to improve the accuracy of location estimation;

[0087] Clean and normalize the collected signal strength, base station load, and location information, remove noise and outliers, and generate data suitable for model training and optimization.

[0088] S2. Adopt a double-layer optimization architecture based on meta-policy gradient, and use historical data to train the handover policy model on multiple simulation tasks, so that the handover policy model can quickly adapt to new handover requirements in different environments, specifically including:

[0089] S2.1. Construct multiple simulated network environments to generate simulated tasks similar to the actual environment; the parameters of the simulated network environment include base station density, signal interference level, user speed distribution, base station load, etc. In the simulated tasks, different handover requirements will be reflected by simulating scenarios such as different user trajectories, network load fluctuations, and signal strength attenuation.

[0090] S2.2. On each simulated task, use a policy gradient algorithm (such as REINFORCE or PPO) for inner optimization. The goal of inner optimization is to adjust the policy parameters of the handover policy model to maximize the reward function of the handover policy model (such as reducing interruptions and delays). Specifically, the inner optimization process is as follows: assign a reward value to each action according to the handover decision in the current state; calculate the policy gradient for each task through the backpropagation algorithm, and update the policy parameters of the handover policy model to adapt to the current task; repeat the above update steps multiple times until the handover policy model converges in this task environment.

[0091] S2.3. The outer optimization uses the task results after inner optimization, calculates the overall meta-loss function, and updates the initial policy parameters of the handover policy model using meta-policy gradients on multiple tasks, so that the handover policy model can quickly adapt in new tasks, thereby outputting the optimal handover policy in the new task environment. Specifically, the outer optimization process is as follows: on all tasks, compare and analyze the initial policy with the adjusted policy results to construct a meta-loss function; use meta-policy gradient optimization techniques to adjust the initial policy of the handover policy model, so that the handover policy model can quickly find the best policy with the fewest parameter update steps when facing a new task environment.

[0092] S2.4. Continuously repeat the double-iteration process of the inner optimization in step S2.2 and the outer optimization in step S2.3 until the overall meta-loss converges, and complete the training of the handover policy model. Specifically, after each inner optimization is completed, its results are passed as input to the outer optimization. After the outer optimization adjusts the initial parameters, it provides a new starting point for the next round of inner optimization. This process repeats continuously to continuously improve the model parameters. Until the overall meta-loss converges, at this time, the performance of the model in new tasks reaches a stable and satisfactory state, with good generalization ability and fast adaptation ability.

[0093] S2.5. Deploy the trained handover policy model on user equipment or edge nodes for online use.

[0094] S3. Based on the real-time data output in step S1 (such as the current base station signal strength, neighboring base station load, device location and speed, etc.), the trained handover policy model generates the optimal handover policy, then executes the optimal handover policy, monitors the handover results, and feeds back to step S4.

[0095] The strategies may include maintaining the current connection, switching to a neighboring base station, or adjusting power control, etc. If the generated optimal handover strategy is to perform a handover operation, the handover request is immediately executed to connect the user equipment to the optimal base station. During the handover process, the connection status is continuously monitored, and the handover success rate, latency, and number of network interruptions are recorded.

[0096] S4. Receive the handover result feedback from step S3 and the real-time data output by step S1, and perform short-term fine-tuning and periodic updates on the handover strategy model to ensure the long-term adaptability of the model in different environments. Specifically, it includes:

[0097] S4.1. Receive the handover result feedback from step S3, and use the incremental learning method to fine-tune the local policy parameters of the handover strategy model to continuously provide the optimal handover strategy in the current network environment;

[0098] S4.2. Receive the real-time data output by step S1. Every set time interval (such as every day or every week), add the received real-time data to the training set. Use the data in the training set as input, and call the model training module to periodically optimize the policy parameters of the handover strategy model to maintain the adaptability of the handover strategy model to the new environment;

[0099] S4.3. Receive the real-time data output by step S1. When a certain monitored data is abnormal (such as the base station load being overloaded during peak hours), simulate and generate data specifically for this abnormal scenario (such as the base station load being overloaded during peak hours). Subsequently, call the model training module to specifically optimize the policy parameters of the handover strategy model to ensure the handover success rate and stability in this abnormal scenario (such as the base station load being overloaded during peak hours).

[0100] In summary, by using the high-speed environment network handover optimization system and method based on meta-policy gradient of the present invention, through the optimization method based on meta-policy gradient, it aims to improve the handover efficiency and stability of the mobile network in high-speed scenarios, reduce network interruptions, and improve communication quality, thereby providing users with a more stable and high-quality network service experience.

[0101] The above applications have elaborated in detail the principles and implementation manners of the present invention through specific individual cases. These embodiments are only used to help understand the core technical content of the present invention. Based on the above specific embodiments of the present invention, those skilled in the art of the present technology, without departing from the principle of the present invention, any improvements and modifications made to the present invention shall fall within the scope of patent protection of the present invention.

Claims

1. A high-speed environment network switching optimization system based on meta-policy gradient, characterized in that: It includes: The data collection module is responsible for real-time monitoring and collection of network environment and device status information of user devices, and generates data suitable for model training and optimization after data preprocessing and filtering operations; The model training module adopts a two-layer optimization architecture based on meta-policy gradients and uses historical data to train the switching strategy model on multiple simulation tasks, so that the switching strategy model can quickly adapt to new switching requirements in different environments; The online decision-making module is responsible for receiving the real-time data output by the data collection module, calling the trained switching strategy model to generate the optimal switching strategy based on the received real-time data, then executing the optimal switching strategy, monitoring the switching results, and feeding back to the self-correction module; The self-correction module is responsible for receiving the switching results fed back by the online decision-making module, and performing short-term fine-tuning and periodic updates on the switching strategy model to ensure the long-term adaptability of the model in different environments.

2. According to claim 1, a high-speed environment network switching optimization system based on meta-policy gradient is characterized in that: The data collection module specifically includes: The signal measurement unit is used to monitor the strength of different base station signals using wireless signal measurement tools, and record the signal strength of the current base station and the signal conditions of each neighboring base station in real time; A load monitoring unit, used to monitor the load status of the current base station; An information acquisition unit, used to acquire the position and speed information of the user equipment through a GPS module and an inertial sensor; The preprocessing unit is used to clean and normalize the signal strength, base station load, and location information collected by the above-mentioned units, remove noise and outliers, and generate data suitable for model training and optimization.

3. According to claim 2, a high-speed environment network switching optimization system based on meta-policy gradient is characterized in that: The model training module specifically includes: Constructing a generation unit, which is used to construct a variety of simulated network environments to generate simulation tasks similar to the actual environment; The inner layer optimization unit is used to perform inner layer optimization using the policy gradient algorithm on each simulation task and adjust the policy parameters of the switching policy model to maximize the reward function of the switching policy model; The outer optimization unit is used to calculate the overall meta-loss function using the task results after the inner optimization, and to update the initial policy parameters of the switching policy model using the meta-policy gradient on multiple tasks, so that the switching policy model can quickly adapt to the new task, thereby outputting the optimal switching policy in the new task environment; A repeated execution unit is used to repeatedly execute the dual iterative process of the inner optimization unit and the outer optimization unit until the overall meta-loss converges and the training of the switching strategy model is completed; The model deployment unit is used to deploy the trained switching strategy model to the user device or edge node for online use.

4. A high-speed environment network switching optimization system based on meta-strategy gradient according to claim 3, characterized in that: The inner optimization unit specifically performs the following operations: assigning a reward value to each action according to the switching decision under the current state; calculating the policy gradient for each task through the back propagation algorithm, and updating the policy parameters of the switching policy model to adapt to the current task; repeating the above updating steps multiple times until the switching policy model reaches convergence in the task environment. The outer optimization unit specifically performs the following operations: on all tasks, the initial strategy is compared and analyzed with the adjusted strategy results to construct a meta-loss function; the initial strategy of the switching strategy model is adjusted using the meta-strategy gradient optimization technique so that the switching strategy model can quickly find the optimal strategy with the least parameter update steps when facing a new task environment.

5. A high-speed environment network switching optimization system based on meta-strategy gradient according to claim 4, characterized in that: The self-correction module specifically includes: A result receiving unit is used to receive the switching result fed back by the online decision module, and fine-tune the local policy parameters of the switching policy model using an incremental learning method to continuously provide the optimal switching policy under the current network environment; A timing update unit is used to receive the real-time data output by the data collection module, add the received real-time data to the training set at a set interval, use the data in the training set as input, and call the model training module to periodically optimize the policy parameters of the switching policy model to maintain the adaptability of the switching policy model to the new environment; The abnormal simulation module is used to receive the real-time data output by the data collection module. When a certain monitoring data is abnormal, it simulates and generates data specifically for the abnormal scenario. Then it calls the model training module to optimize the strategy parameters of the switching strategy model to ensure the switching success rate and stability in the abnormal scenario.

6. A high-speed environment network switching optimization method based on meta-strategy gradient, characterized in that: The steps include: S1, real-time monitoring and collection of user equipment network environment and equipment status information, and data preprocessing and filtering operations to generate data suitable for model training and optimization; S2, adopts a two-layer optimization architecture based on meta-policy gradient, uses historical data to train the switching strategy model on multiple simulation tasks, so that the switching strategy model can quickly adapt to new switching requirements in different environments; S3, based on the real-time data outputted in step S1, the trained switching strategy model generates an optimal switching strategy, then executes the optimal switching strategy, monitors the switching results, and feeds back to step S4; S4, receiving the switching result fed back by step S3 and the real-time data output by step S1, and performing short-term fine-tuning and periodic updating on the switching strategy model to ensure the long-term adaptability of the model in different environments.

7. A high-speed environment network switching optimization method based on meta-strategy gradient according to claim 6, characterized in that: The step S1 specifically includes: Use wireless signal measurement tools to monitor the strength of different base station signals, and record the signal strength of the current base station and the signal conditions of neighboring base stations in real time; Monitor the load status of the current base station; Obtain the location and speed information of the user's device through the GPS module and inertial sensor; The collected signal strength, base station load, and location information are cleaned and normalized to remove noise and outliers and generate data suitable for model training and optimization.

8. The high-speed environment network switching optimization method based on meta-strategy gradient according to claim 7 is characterized in that: The step S2 specifically includes: Construct a variety of simulated network environments to generate simulation tasks similar to the actual environment; Use the policy gradient algorithm to perform inner optimization on each simulated task and adjust the policy parameters of the switching policy model to maximize the reward function of the switching policy model. The outer optimization uses the task results after the inner optimization to calculate the overall meta-loss function, and uses the meta-policy gradient on multiple tasks to update the initial policy parameters of the switching policy model, so that the switching policy model can quickly adapt to the new task, thereby outputting the optimal switching policy in the new task environment; The dual iterative process of inner optimization and outer optimization is repeated until the overall meta-loss converges and the training of the switching strategy model is completed; Deploy the trained switching policy model to user devices or edge nodes for online use.

9. The high-speed environment network switching optimization method based on meta-strategy gradient according to claim 8 is characterized in that: The inner optimization process is as follows: assign a reward value to each action based on the switching decision in the current state; calculate the policy gradient for each task through the back-propagation algorithm, and update the policy parameters of the switching policy model to adapt to the current task; repeat the above update steps many times until the switching policy model reaches convergence in the task environment. The outer optimization process is as follows: on all tasks, the initial strategy and the adjusted strategy results are compared and analyzed to construct a meta-loss function; the meta-strategy gradient optimization technology is used to adjust the initial strategy of the switching strategy model so that the switching strategy model can quickly find the optimal strategy with the least parameter update steps when facing a new task environment.

10. The high-speed environment network switching optimization method based on meta-strategy gradient according to claim 9, characterized in that: The step S4 specifically includes: Receive the switching result fed back in step S3, and use an incremental learning method to fine-tune the local policy parameters of the switching policy model to continuously provide the optimal switching policy under the current network environment; Receive the real-time data outputted in step S1, add the received real-time data to the training set at set intervals, use the data in the training set as input, and call the model training module to periodically optimize the policy parameters of the switching policy model to maintain the adaptability of the switching policy model to the new environment; Receive the real-time data outputted in step S1, and when a certain monitoring data is abnormal, simulate and generate data specifically for the abnormal scenario, then call the model training module to optimize the strategy parameters of the switching strategy model specifically to ensure the switching success rate and stability in the abnormal scenario.

Images