Quic protocol multi-path transmission method based on network environment self-adaption

Through data flow memory pool management, deep reinforcement learning and principal component analysis optimization modules, adaptive adjustment of QUIC protocol multipath transmission is realized, solving the problems of low transmission efficiency and insufficient response capabilities in the existing technology, and improving the Transmission efficiency and robustness.

CN119094444APending Publication Date: 2024-12-06ARTIFICIAL INTELLIGENCE RES INST OF HEFEI COMPREHENSIVE NAT SCI CENT (ANHUI ARTIFICIAL INTELLIGENCE LAB)
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411122604.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-15
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

In the case of network fluctuations and interruptions in the existing MP_QUIC protocol, the data buffering mechanism of multi-path transmission tasks leads to inefficient transmission efficiency and cannot effectively respond to changes in the network environment, affecting the transmission tasks of subsequent paths.

Method used

Adopting data flow memory pool management module, path data flow task scheduling module and principal component analysis and optimization module, through distributed processing of memory pools, deep reinforcement learning and graph neural network, the adaptive adjustment of multi-path transmission architecture and network topology diagram are realized. Optimize key data transmission components.

Benefits of technology

It improves the transmission efficiency of multi-path QUIC protocol in network fluctuation environment, enhances the robustness of network connections, adapts to different extension and extended path transmission protocols, and improves the stability and adaptability of transmission.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119094444A_ABST
    Figure CN119094444A_ABST
Patent Text Reader

Abstract

The invention discloses a self-adaption quic protocol multi-path transmission method based on a network environment. The method comprises the following steps of: realizing distributed processing of transmission multi-path and managing data flow in multi-path transmission by adopting a memory pool method based on a data flow memory pool management module; based on a deep reinforcement learning method, a path data flow task scheduling module is utilized to enable a transmission architecture under multiple paths to realize self-adaptive dynamic adjustment according to a real-time network environment; and the transmission data is analyzed based on the principal component analysis optimization module, key data transmission components are identified and optimized, and network environment adaptive quic protocol multi-path transmission is optimized. According to the invention, the memory pool is constructed according to different transmission paths to realize a distributed processing part of multi-path transmission, and adaptive adjustment is realized according to a network environment based on a deep reinforcement learning method, so that the problem that the multi-path transmission efficiency is blocked due to network fluctuation is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of computer networks, and in particular to a QUIC protocol multi-path transmission method based on network environment adaptation. Background Art

[0002] QUIC is an Internet data transmission protocol proposed by Google to replace TCP. It introduces many new features, which theoretically has better performance than TCP. For example, it solves the head-of-line blocking problem through multi-channel transmission, reduces the transport layer handshake delay through 0GRTT handshake, and provides better support for mobility through connection migration. However, the network environment and terminal devices in real life are diverse, and there are various attacks on the Internet, so the performance of QUIC in the actual network may not be as expected. Therefore, it is very important to explore the impact of QUIC on existing network services. First, the development history and main features of QUIC are introduced, and the two most widely used application scenarios at present - web browsing and video transmission.

[0003] QUIC aims to improve the speed and reliability of network connections to replace the Transmission Control Protocol (TCP) widely used in the current Internet infrastructure. QUIC provides higher security and faster data transmission through encryption and multiplexing technology. It supports sending multiple data streams in parallel on a single connection, thereby reducing latency and increasing throughput. QUIC also has mechanisms such as congestion control and flow control to cope with network congestion and ensure the stability of data transmission. The Internet Engineering Task Force (IETF) has completed the standardization of QUIC, and mainstream web browsers and servers are gradually adopting it. Compared with TCP, QUIC can significantly improve the loading speed of web pages and reduce connection interruptions in high-latency and unstable network environments, such as mobile networks, making the network experience smoother.

[0004] Wireless networks have grown significantly over the past few decades due to the proliferation of mobile devices such as smartphones, tablets, etc. Typically, these devices are often equipped with several wireless interfaces such as Wi-Fi, 4G (LTE), and 5G (i.e., multi-homed devices). When a user moves, one wireless connection may be disconnected, leading to the need for seamless switching between wireless networks. Multipath Transport Protocols (MTP) such as MPQUIC and MPTCP allow for better utilization of available network resources on multi-homed devices. Built on top of UDP, the relatively new QUIC protocol is the benchmark for modern network protocol stacks. In order to enable rapid updates and feature iterations, most QUIC implementations are implemented as userspace libraries, resulting in a large and decentralized ecosystem.

[0005] The disadvantage of the existing technology is that the existing MP_QUIC allows transmission tasks from multiple transmission paths. The protocol fragments and transmits files in a "stream" manner, and mounts each "stream" to different paths to achieve multi-path transmission. When performing fragmentation and "stream" mounting tasks, most of the current research results are to mount fragments in an interlaced manner on multiple paths. While a batch of "streams" are being transmitted on one path, the next batch of "streams" are mounted on another path. When this batch of "streams" starts to be transmitted, the "streams" of the delayed batch are mounted on the previous path to achieve data buffering of the transmission task and reduce the time consumed by waiting for IO read and write operations; however, once the "stream" is mounted to the transmission queue of the current path, it will no longer change. If the network of the transmission path fluctuates and interrupts, it will affect the subsequent transmission tasks of other paths. The network architecture based on the quic protocol is more favored by Internet companies because it performs better than the traditional FTP protocol in a weak network environment, so it is necessary to consider the impact of network fluctuations on MP_QUIC. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the prior art. To achieve the above purpose, a QUIC protocol multi-path transmission method based on network environment adaptation is adopted to solve the problems raised in the above background technology.

[0007] A QUIC protocol multi-path transmission method based on network environment adaptation, the multi-path transmission includes three groups of modules: a data stream memory pool management module, a path data stream task scheduling module, and a principal component analysis optimization module. The steps of the method include:

[0008] Step S1, using a memory pool method based on a data stream memory pool management module to implement distributed processing of multi-path transmission and manage data streams in multi-path transmission;

[0009] Step S2: Based on the deep reinforcement learning method, the path data flow task scheduling module is used to enable the transmission architecture under the multi-path to achieve adaptive dynamic adjustment according to the real-time network environment;

[0010] Step S3: Analyze the transmission data based on the principal component analysis optimization module, identify and optimize key data transmission components, and optimize the QUIC protocol multi-path transmission that is adaptive to the network environment.

[0011] As a further solution of the present invention: the specific steps in step S1 include:

[0012] Set N different network ports of the host device as network transmission paths, and build N different memory pools as buffers for reading the transferred files. In the process of building the memory pool, set the capacity of a single memory pool Ci Set to:

[0013]

[0014] Where i is the average bandwidth utilization under the i-th path, C total The total bandwidth capacity of the network card under the network port corresponding to the path;

[0015] When reading files and storing them in different memory pools, the files are divided into shards in descending order of memory pool capacity and stored in the memory pools. If the file capacity is larger than the total capacity of the memory pools, an equal number of shards will be stored after a certain memory pool changes.

[0016] The capacity of the memory pool is not constant. The network environment within a certain period of time is predicted based on the current network status, and the memory pool capacity is adjusted in advance for the upcoming network environment.

[0017] By inputting the current network topology graph X, it is first input into the trained multi-layer convolutional neural network, and then a network topology graph X' with corresponding path feature values ​​is obtained. After reinforcement learning training, the agent will decide to increase or decrease the capacity of the memory pool corresponding to the path based on the feature value information of the nodes and edges on the path.

[0018] As a further solution of the present invention: the specific steps in step S2 include:

[0019] Get the data in the memory pool and encapsulate it based on the rules of the protocol under this path;

[0020] After transmission based on multi-path, different "stream" transmission queues need to be built according to different paths;

[0021] The data stream memory pool management module optimizes the impact of IO and sending asynchrony under multi-path transmission in an unstable network environment by reasonably allocating memory pools, and operates the unencapsulated buffer content;

[0022] When the file content is encapsulated into a "stream" and mounted on the sending queue of each path, the data on the transmission queue is sent instantly based on the Q-learning algorithm of reinforcement learning.

[0023] As a further solution of the present invention: the specific steps in step S3 include:

[0024] Build an intelligent agent trained by the principal component analysis optimization module to simulate and cope with real network application scenarios;

[0025] Set the training data in the closed network environment as the source domain D s,The real network environment dataset is set as the target domain Dt;

[0026] Among them, D s The network topology set X used for simulation training s ={x 1s ,x 2s ,…,x ns}, ns is the number of topological graphs used for training, the set is used as the data set for transfer learning, and the trained agent model GCN_RLmodel after deep reinforcement learning;

[0027] For X s Each set of elements x in i The label of the action performed i ; Label set composition Y s ={y 1s ,y 2s ,…,y ns} as the label set of the source domain; D t The topology structure data set X under the real network transmission environment t ={x 1t ,x 2t ,…,x nt}, nt is the number of real network topology graphs used for transfer learning, and the total variance objective function f is defined PCA for:

[0028]

[0029] Where T represents the matrix transpose transformation, tr(·) represents the matrix trace operation, I represents the identity matrix, and A t A=I means that the constraint condition is that the feature mapping matrix is ​​an orthogonal matrix. This definition indicates that the maximum value of the following expression is obtained under the premise of satisfying the constraint condition;

[0030] Solving for f PcA The obtained A is the feature mapping matrix. Subsequently, the adjacency matrix x of the topological graph can be mapped to the new dimensional space through A, and then handed over to the intelligent agent model GCN_RLmodel to perform corresponding operations.

[0031] Compared with the prior art, the present invention has the following technical effects:

[0032] The above-mentioned technical solution is adopted by setting up a data flow memory pool management module, a multi-path data flow task scheduling module, and an adaptive optimization module of the network topology map. The memory pool method is used to realize the distributed processing of multi-path transmission. Based on the deep reinforcement learning method, the transmission architecture under multi-path can be adaptively adjusted dynamically according to the real-time network environment. The three groups of modules of "data flow memory pool management + path data flow task scheduling + principal component analysis optimization" are designed to realize the network environment adaptive QUIC protocol multi-path transmission method, improve the transmission efficiency of the multi-path QUIC protocol in the network fluctuation environment, enhance the robustness of network connection transmission for application scenarios with network congestion under a certain path, and can extend and expand the application of this method based on different extensions and expansions of host support and adaptation path transmission protocols. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings:

[0034] Figure 1 A schematic diagram of the steps of the multi-path transmission method according to an embodiment disclosed in the present application;

[0035] Figure 2 A schematic diagram of the structure of a QUIC protocol multi-path transmission module based on network environment adaptation according to an embodiment disclosed in the present application;

[0036] Figure 3 A schematic diagram of a flow chart of a self-adaptive optimization module of a network topology diagram according to an embodiment disclosed in the present application;

[0037] Figure 4 This is a diagram of the encapsulation structure of the file read into the memory pool according to the embodiment disclosed in this application;

[0038] Figure 5 A model of a memory pool for deep reinforcement learning processing according to an embodiment disclosed in this application;

[0039] Figure 6 A schematic diagram of multipath transmission blocking phenomenon according to an embodiment disclosed in the present application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] Please refer to Figure 1 and Figure 2,In an embodiment of the present invention, a QUIC protocol multi-path transmission method based on network environment adaptation, the multi-path transmission includes three groups of modules including a data stream memory pool management module, a path data stream task scheduling module, and a principal component analysis optimization module;

[0042] like Figure 2 As shown in the figure, it is a structural diagram of the multi-path transmission module of the quic protocol based on network environment adaptation; among them, the memory pool management module of the data flow needs to complete the function of building its memory pool for different sending ports respectively, and realize the bandwidth utilization prediction function of different paths according to the actual network topology structure diagram based on the graph neural network and reinforcement learning method, and adjust the size of its memory pool based on the prediction result; the multi-path data flow task scheduling module needs to encapsulate the data in the memory pool into the sending unit "flow" in the quic protocol and mount it on the transmission queue of the corresponding sending port, and add a transmission monitoring module to feedback the current network transmission status in real time, and then adjust the task scheduling of the "flow" on different transmission queues in real time based on the feedback information based on the deep reinforcement learning method; the adaptive optimization module of the network topology diagram improves the generalization of the first two modules, and through the feature mapping characteristics of transfer learning, the trained intelligent agent can adapt to the changing network structure to a certain extent, so that the industrial Internet transmission system can generally maintain a stability, such as Figure 3 As shown, the figure is a schematic diagram of the adaptive optimization module process of the network topology diagram;

[0043] The steps of the multipath transmission method include:

[0044] Step S1: Based on the data stream memory pool management module, a memory pool method is used to implement distributed processing of transmission multipaths and manage data streams in multipath transmission. The specific steps include:

[0045] Specifically, the data stream memory pool management module is optimized by deep reinforcement learning. The data stream memory pool management module builds a memory pool for distributed processing. The memory pool size still needs to be dynamically adjusted through network adaptive processing in the future. It mainly processes data that has not yet been encapsulated to save memory usage.

[0046] Set N different network ports of the host device as network transmission paths, and build N different memory pools as buffers for reading the transferred files. In the process of building the memory pool, set the capacity of a single memory pool C i Set to:

[0047]

[0048] Where i is the average bandwidth utilization under the i-th path, C totalIt is the total bandwidth capacity of the network card under the network port corresponding to the path; this ensures that there will not be too much waste of memory resources during the transmission process.

[0049] When reading files and storing them in different memory pools, the files are divided into pieces and stored in the memory pools in descending order of memory pool capacity. If the file capacity is larger than the total capacity of the memory pools, an equal number of pieces are stored after a certain memory pool changes. For example, Figure 4 As shown, the diagram shows the encapsulation structure of file information in the memory pool;

[0050] The capacity of the memory pool is not constant. In actual network transmission scenarios, the network connection status is different every minute and every second. Therefore, if the capacity of the memory pool is a fixed value, it will cause serious problems in unstable network environments. In addition, since this stage only reads the file into the buffer, there is still some time before the file content is transmitted. Therefore, the task of adjusting the memory pool capacity requires a certain degree of foresight, that is, based on the current network status, the network environment within a certain period of time is predicted, and the memory pool capacity is adjusted one step ahead of the upcoming network environment;

[0051] By inputting the current network topology graph X, it is first input into the trained multi-layer convolutional neural network, and then a network topology graph X' with corresponding path feature values ​​is obtained. After reinforcement learning training, the agent will decide to increase or decrease the capacity of the memory pool corresponding to the path based on the feature value information of the nodes and edges on the path.

[0052] like Figure 5 As shown in the figure, the model of deep reinforcement learning processing memory pool is shown. The characteristic values ​​of network nodes include throughput, packet loss rate, transmission delay, etc. under various transmission path protocols;

[0053] The unit of capacity increased or decreased at a time is M bytes, where M is the lowest unit of "stream" of the quic protocol of the current network path. For example, in the xquic code base open sourced by Alibaba, the capacity of a single stream is 32MB, and the maximum capacity of the current memory pool is the bandwidth value of the path, and the minimum is p×M bytes, where p is the number of transmission threads enabled based on the quic protocol under the current path.

[0054] Step S2: Based on the deep reinforcement learning method, the path data flow task scheduling module is used to enable the transmission architecture under the multi-path to achieve adaptive dynamic adjustment according to the real-time network environment. The specific steps include:

[0055] Specifically, the path data flow task scheduling module is optimized by deep reinforcement learning. The path data flow task scheduling module processes the data flow encapsulated as stream and determines whether the data in the queue is confirmed to be sent or canceled and reassembled.

[0056] Get the data in the memory pool and encapsulate it based on the rules of the protocol under the path;

[0057] After transmission based on multi-path, different "stream" transmission queues need to be built according to different paths;

[0058] The data stream memory pool management module optimizes the impact of IO and sending asynchrony under multi-path transmission in an unstable network environment by reasonably allocating memory pools, and operates the unencapsulated buffer content;

[0059] When the file content is encapsulated into a "stream" and mounted on the sending queue of each path, the data on the transmission queue is sent instantly based on the Q-learning algorithm of reinforcement learning.

[0060] In this embodiment, the task of the path data flow task scheduling module is to take out the data in the memory pool and encapsulate it based on the rules of the protocol under the path. Different network operators have different encapsulation methods for "streams". Therefore, when we choose to use multi-path transmission, we need to build different "stream" transmission queues according to different paths. The data flow memory pool management module in the previous article describes in detail how to optimize the impact of IO and transmission asynchrony under multi-path transmission in an unstable network environment by reasonably allocating memory pools, but these operations are limited to unencapsulated buffer contents. When the file content is encapsulated into a "stream" and then mounted on the sending queue of each path, if the network is unstable, the sending of all "streams" on the queue will be blocked, and the reception of the file will be as follows. Figure 6 Also forced to delay.

[0061] Unlike the adjustment of memory pool capacity, which requires certain prejudgment operations, the sending of data on the transmission queue requires absolute immediacy. Therefore, the Q-learning algorithm of reinforcement learning can be directly used (using the Q-table to store the Q value built based on State and Action, and then selecting the action that can obtain the maximum benefit based on the Q value). In this module, State, Action and Reward are composed of the following elements.

[0062] State: Transmission queue T for each path queue (represented by an array, each number in the queue represents the number of the "flow");

[0063] The network transmission rate s under this path;

[0064] The network transmission packet loss rate d under this path;

[0065] The network transmission delay t along this path;

[0066] Action: T i --->T j (It means taking down the "flow" on the transmission queue of path i and mounting it on the transmission queue of path j);

[0067] Reward: 3-m (the reward increases by 3-m for each "stream" received by the receiving end, where m represents the time it takes for the receiving end to reply and confirm the receipt of the "stream" with a total of m RTTs).

[0068] Step S3: Analyze the transmission data based on the principal component analysis optimization module, identify and optimize the key data transmission components, and optimize the QUIC protocol multi-path transmission that is adaptive to the network environment. The specific steps include:

[0069] Build an intelligent agent trained by the principal component analysis optimization module to simulate and cope with real network application scenarios;

[0070] Specifically, in deep reinforcement learning based on graph neural networks, the training process has very strict requirements on the input graph structure, the trained model has poor generalization, and the application of the original data model cannot effectively achieve the expected training performance. At present, the research results of MP_QUIC based on deep reinforcement learning are basically limited to closed and single network environments. In actual networked environments, even in relatively uniform network conditions in industrial production environments, there are still differences in the distribution of the network training environment and the real environment. For the memory pool management module, many current graph representation learning algorithms are based on the ideal case of independent and identically distributed graphs, that is, the sample data distribution of graph data obeys independent and identically distributed, but this is usually difficult to meet in actual application scenarios. There is often a distribution migration between real samples. Therefore, building a set of adaptive optimization modules based on principal component analysis PCA can help trained intelligent agents to simulate and cope with real network application scenarios.

[0071] Set the training data in the closed network environment as the source domain D s ,The real network environment dataset is set as the target domain Dt;

[0072] Among them, D s The network topology set X used for simulation training s ={x 1s ,x 2s ,…,x ns}, ns is the number of topological graphs used for training, the set is used as the data set for transfer learning, and the trained agent model GCN_RLmodel after deep reinforcement learning;

[0073] For X s Each set of elements x in i The label of the action performedi , (y∈R 3×D , D represents the number of dimensions of the topology map node, 3 represents the action performed - memory pool capacity increased / decreased / unchanged); the label set consists of Y s ={y 1s ,y 2s ,…,y ns} as the label set of the source domain; D t The topology structure data set X under the real network transmission environment t ={x 1t ,x 2t ,…,x nt}, nt is the number of real network topology graphs used for transfer learning, and the total variance objective function f is defined PCA for:

[0074]

[0075] Where T represents the matrix transpose transformation, tr(·) represents the matrix trace operation, I represents the identity matrix, and A T A=I means that the constraint condition is that the feature mapping matrix is ​​an orthogonal matrix. This definition indicates that the maximum value of the following expression is obtained under the premise of satisfying the constraint condition;

[0076] Solving for f PCA The obtained A is the feature mapping matrix. Subsequently, the adjacency matrix x of the topological graph can be mapped to the new dimensional space through A, and then handed over to the intelligent agent model GCN_RLmodel to perform corresponding operations.

[0077] In this embodiment, the multi-path transmission method of the quic protocol based on network environment adaptation can realize distributed reading and transmission preprocessing of transmission files by building a memory pool, and realize intelligent adjustment of critical resources based on real-time network environment by combining graph neural network and reinforcement learning, and complete the network optimization of the multi-path quic protocol in the transmission application scenario with network fluctuations. Since enterprise-level network devices are mostly confidential, the network structure diagram of a host in its local area network must be visible to the enterprise and the information is clear;

[0078] Therefore, the test is implemented based on the following five steps:

[0079] 1) Allocate hardware resources for the multipath protocol supported by the transmission host;

[0080] 2) Set the configuration parameters of each transmission path;

[0081] 3) Obtain the network topology diagram of the current transmission host in the entire local area network in real time;

[0082] 4) Conduct training and principal component analysis of graph neural network reinforcement learning;

[0083] 5) Apply the trained module to multipath QUIC transmission.

[0084] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents, and all should be included in the scope of protection of the present invention.

Claims

1. A QUIC protocol multi-path transmission method based on network environment adaptation, characterized in that The multi-path transmission includes three groups of modules: a data flow memory pool management module, a path data flow task scheduling module, and a principal component analysis optimization module. The steps of the method include: Step S1, using a memory pool method based on a data stream memory pool management module to implement distributed processing of multi-path transmission and manage data streams in multi-path transmission; Step S2: Based on the deep reinforcement learning method, the path data flow task scheduling module is used to enable the transmission architecture under the multi-path to achieve adaptive dynamic adjustment according to the real-time network environment; Step S3: Analyze the transmission data based on the principal component analysis optimization module, identify and optimize key data transmission components, and optimize the QUIC protocol multi-path transmission that is adaptive to the network environment.

2. According to claim 1, a QUIC protocol multi-path transmission method based on network environment adaptation is characterized in that: The specific steps in step S1 include: Set N different network ports of the host device as network transmission paths, and build N different memory pools as buffers for reading the transferred files. In the process of building the memory pool, set the capacity of a single memory pool C i Set to: in, is the average bandwidth utilization under the i-th path, C total The total bandwidth capacity of the network card under the network port corresponding to the path; When reading files and storing them in different memory pools, the files are divided into shards in descending order of memory pool capacity and stored in the memory pools. If the file capacity is larger than the total capacity of the memory pools, an equal number of shards will be stored after a certain memory pool changes. The capacity of the memory pool is not constant. The network environment within a certain period of time is predicted based on the current network status, and the memory pool capacity is adjusted in advance for the upcoming network environment. By inputting the current network topology graph X, it is first input into the trained multi-layer convolutional neural network, and then a network topology graph X' with corresponding path feature values ​​is obtained. After reinforcement learning training, the agent will decide to increase or decrease the capacity of the memory pool corresponding to the path based on the feature value information of the nodes and edges on the path.

3. According to claim 1, a QUIC protocol multi-path transmission method based on network environment adaptation is characterized in that: The specific steps in step S2 include: Get the data in the memory pool and encapsulate it based on the rules of the protocol under this path; After transmission based on multi-path, different "stream" transmission queues need to be built according to different paths; The data stream memory pool management module optimizes the impact of IO and sending asynchrony under multi-path transmission in an unstable network environment by reasonably allocating memory pools, and operates the unencapsulated buffer content; When the file content is encapsulated into a "stream" and mounted on the sending queue of each path, the data on the transmission queue is sent instantly based on the Q-learning algorithm of reinforcement learning.

4. According to claim 1, a QUIC protocol multi-path transmission method based on network environment adaptation is characterized in that: The specific steps in step S3 include: Build an intelligent agent trained by the principal component analysis optimization module to simulate and cope with real network application scenarios; Set the training data in the closed network environment as the source domain D s ,The real network environment dataset is set as the target domain Dt; Among them, D s The network topology set X used for simulation training s ={x 1s , x 2s , …, x ns }, ns is the number of topological graphs used for training, the set is used as the data set for transfer learning, and the trained agent model GCN_RLmodel after deep reinforcement learning; For X s Each set of elements x in i The label of the action performed i ; Label set composition Y s ={y 1s ,y 2s , …, y ns } as the label set of the source domain; D t The topology structure data set X under the real network transmission environment t ={x 1t , x 2t , …, x nt }, nt is the number of real network topology graphs used for transfer learning, and the total variance objective function f is defined PCA for: Where T represents the matrix transpose transformation, tr(·) represents the matrix trace operation, I represents the identity matrix, and A T A=I means that the constraint condition is that the feature mapping matrix is ​​an orthogonal matrix. This definition indicates that the maximum value of the following expression is obtained under the premise of satisfying the constraint condition; Solving for f PCA The obtained A is the feature mapping matrix. Subsequently, the adjacency matrix x of the topological graph can be mapped to the new dimensional space through A, and then handed over to the intelligent agent model GCN_RLmodel to perform corresponding operations.