A TSN switch queue length prediction method based on Transformer and GRU

Abstract

The present invention discloses a TSN switch queue length prediction method based on Transformer and GRU, comprising: firstly building a centrally managed network topology based on TSN switches, collecting feature data of the network topology, extracting independent feature data, and then building a data set of time series with several preset step lengths and their corresponding queue lengths, and then building a neural network based on Transformer and GRU: N Encoder modules are sequentially connected to the first Linear layer and the second Linear layer, with the time series of the preset step length as input and the corresponding TSN switch queue length as output, training the neural network, obtaining a TSN switch queue length prediction model, and finally inputting the time series into the model to obtain the TSN switch queue length. The present invention combines a variety of feature data to help the model better understand the data and improve the model performance. Based on the two methods of Transformer and GRU, the global and local information of the feature data can be learned to improve the model performance.

Description

Technical Field

[0001] The present invention belongs to the field of industrial Internet, and specifically relates to a TSN switch queue length prediction method based on Transformer and GRU. Background Art

[0002] TSN (Time-sensitive networking) is a new generation of deterministic network technology, which is mainly aimed at safety-critical fields such as industrial Internet of Things, industrial automation, vehicle networks, and aerospace electronic system networks. It consists of a cluster of protocols working at the media access control layer. With the support of these protocols, on the one hand, TSN can ensure the data transmission delay in the switching network, and on the other hand, time-sensitive data streams and non-time-sensitive data streams can be mixed and transmitted in the same network, and the parallel transmission of non-critical loads will not affect the transmission delay of critical loads.

[0003] The priority inversion problem in standard Ethernet will cause frame transmission disorder at the switch output port and all subsequent output ports on the path, increasing network delay. To alleviate this problem, TSN proposes a guard band mechanism, that is, to make a judgment before the gate of each real-time queue is opened, to prevent non-real-time frames that will interfere with the transmission of real-time frames, and to prevent the occurrence of priority inversion. The guard band mechanism can prevent non-real-time traffic from interfering with real-time traffic, but at the same time it will introduce bandwidth waste.

[0004] In order to solve the problem of bandwidth waste, one solution is to optimize the scheduling problem based on the SDN (Software Defined Network) system and reduce the number of guard bands. Among them, accurately predicting the queue length of TSN switches and ensuring scheduling optimization are the problems that need to be solved urgently. Summary of the invention

[0005] The technical problem to be solved by the present invention is: in view of the problem that the prior art cannot accurately predict the queue length of a TSN switch and ensure scheduling optimization, a TSN switch queue length prediction method based on Transformer and GRU is provided.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: a TSN switch queue length prediction method based on Transformer and GRU, comprising the following steps:

[0007] S1. Build a centrally managed network topology based on TSN switches and collect characteristic data of the network topology;

[0008] S2, extracting independent feature data from the feature data;

[0009] S3. construct a data set T based on the independent feature data, where the data set T includes: a time series of several preset step lengths and their corresponding queue lengths; and divide the data set T into a training set and a test set;

[0010] S4. Construct a neural network based on Transformer and GRU, take the time series with preset step length in the training set as input and the corresponding TSN switch queue length as output, train the neural network, and obtain the TSN switch queue length prediction model M. init ;

[0011] S5. Input the time series into the TSN switch queue length prediction model M init Get the TSN switch queue length.

[0012] Furthermore, in the aforementioned step S1, the characteristic data includes: sending time, receiving time, the difference between sending time and receiving time, sending packet size, switch time, the difference between sending time and switch time, the difference between receiving time and switch time, sending node, node priority, time slot, traffic pattern, throughput, and queue length.

[0013] Furthermore, the aforementioned step S2 includes the following sub-steps:

[0014] S201, calculating the Spearman correlation coefficient for the feature data, removing the feature data whose absolute value of the correlation coefficient is greater than or equal to a threshold, and obtaining updated feature data;

[0015] S202: Perform multicollinearity analysis on the updated feature data, remove feature data with variance inflation factors greater than or equal to a threshold, and obtain independent feature data.

[0016] Furthermore, in the aforementioned step S4, the neural network based on Transformer and GRU includes: N sequentially connected Encoder modules, a first Linear layer, and a second Linear layer; the first Encoder module serves as the input end of the neural network of Transformer and GRU, and the last Encoder module is connected to the first Linear layer and the second Linear layer in sequence, and the second Linear layer serves as the output end of the neural network of Transformer and GRU.

[0017] Furthermore, the aforementioned Encoder module includes: a first Feed Forward layer, a second Feed Forward layer, a Multi-Head Attention layer, a GRU layer, a first Add&Norm layer, a second Add&Norm layer, a third Add&Norm layer, and an addition operation module;

[0018] The input of the first Feed Forward layer, the input of the GRU layer, and the input of the first Add&Norm layer are used as the input of the Encoder module. The output of the first Feed Forward layer is connected to the input of the Multi-Head Attention layer and the input of the second Add&Norm layer respectively.

[0019] The output of the Multi-Head Attention layer is connected to the input of the second Add&Norm layer;

[0020] The output of the GRU layer is connected to the input of the first Add&Norm layer;

[0021] The output ends of the first Add&Norm layer and the second Add&Norm layer are respectively connected to the input ends of the addition operation module;

[0022] The output end of the addition operation module is connected to the input end of the third Add&Norm layer and the input end of the second Feed Forward layer respectively;

[0023] The output of the second Feed Forward layer is connected to the input of the third Add&Norm layer;

[0024] The output of the third Add&Norm layer serves as the output of the Encoder module.

[0025] Furthermore, the aforementioned TSN switch queue length prediction method based on Transformer and GRU also includes evaluating the TSN switch queue length prediction model M based on the MSE evaluation index. init Performance evaluation is performed on the test set, MSE is the mean square error:

[0026]

[0027] Where n is the number of samples, y i is the true value of the i-th sample, is the predicted value of the i-th sample.

[0028] Furthermore, in the aforementioned step S201, the two sets of feature data X={x i |0<i≤n} and Y={y i |0<i≤n} to calculate the Spearman correlation coefficient as follows:

[0029]

[0030] Where n is the number of samples, d iis the level difference between the samples corresponding to the two sets of feature data, r∈[-1, 1].

[0031] Furthermore, in the aforementioned step S202, the calculation formula of the variance inflation factor is as follows:

[0032]

[0033] Among them, R i is the negative correlation coefficient of the regression analysis of the i-th characteristic data on the remaining characteristic data.

[0034] Furthermore, in the aforementioned step S3, the preset step size is 10.

[0035] Furthermore, in the aforementioned step S201, feature data with a correlation coefficient greater than or equal to 0.7 is removed to obtain updated feature data;

[0036] Furthermore, in the aforementioned step S202, feature data with a variance inflation factor greater than or equal to 10 is removed to obtain independent feature data.

[0037] Compared with the prior art, the beneficial technical effects of the present invention using the above technical solution are as follows:

[0038] 1. The Multi-Head Attention layer in the Encoder module can focus on information at different positions in the feature data at the same time, and on the other hand, it can better capture long-distance dependencies, thereby learning rich feature data information to improve model performance.

[0039] 2. Combine the characteristic data related to the TSN switch queue, such as sending time, priority, time slot, etc., to provide more information for the model, help the model better understand the data, and improve the performance of the prediction model.

[0040] 3. Transformer processes long-distance dependencies in feature data to improve the model's ability to capture global information, while GRU processes short-distance dependencies in feature data to improve the model's ability to capture local information. Based on the two methods of Transformer and GRU, the global and local information of feature data can be learned to improve the performance of the prediction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a model structure diagram of the present invention.

[0042] Figure 2 It is a network topology diagram. DETAILED DESCRIPTION

[0043] In order to better understand the technical content of the present invention, specific embodiments are given and described as follows in conjunction with the accompanying drawings.

[0044] Various aspects of the invention are described herein with reference to the accompanying drawings, in which many illustrative embodiments are shown. The embodiments of the invention are not limited to those described in the accompanying drawings. It should be understood that the invention is implemented by any of the various concepts and embodiments described above, as well as the concepts and embodiments described in detail below, because the concepts and embodiments disclosed in the invention are not limited to any implementation. In addition, some aspects disclosed in the invention may be used alone or in any appropriate combination with other aspects disclosed in the invention.

[0045] The present invention provides a TSN switch queue length prediction method based on Transformer and GRU, comprising the following steps:

[0046] S1. Build a centrally managed network topology based on TSN switches and collect characteristic data of the network topology. Figure 2 The characteristic data includes: sending time send, receiving time arrival, sending time and receiving time difference send2arrivalL, sending packet size size, switch time queue time, sending time and switch time difference send2queueL, receiving time and switch time difference queue2arrivalL, sending node node, node priority priority, time slot interval, traffic mode traffic, throughput throughput, queue length y;

[0047] S2, extracting independent feature data from the feature data. Specifically including sub-steps S201-S202:

[0048] S201, calculate the Spearman correlation coefficient for the feature data, remove the feature data whose absolute value of the correlation coefficient is greater than or equal to 0.7, and obtain updated feature data; X = {x i |0<i≤n} and Y={y i |0<i≤n} are two sets of characteristic data, and the Spearman correlation coefficient calculation formula is:

[0049]

[0050] Where n is the number of samples, d i is the level difference between the samples corresponding to the two sets of feature data, r∈[-1, 1].

[0051] In this embodiment, Table 1 is a correlation coefficient table of various characteristic data, and data with an absolute value of the correlation coefficient greater than or equal to 0.7 are displayed in bold:

[0052] Table 1

[0053]

[0054] The feature data with the absolute value of the correlation coefficient greater than or equal to 0.7 are removed, and the updated feature data are: sending time send, node priority priority, time slot interval, traffic mode traffic, and queue length y.

[0055] S202: Based on the updated feature data, perform multicollinearity analysis, remove feature data with variance inflation factor greater than or equal to 10, and obtain independent feature data. The calculation formula of variance inflation factor is as follows:

[0056]

[0057] Among them, R i The negative correlation coefficient of the regression analysis of the i-th feature data on the other feature data. The larger the variance inflation factor, the greater the possibility of collinearity between the independent variables. Generally speaking, if the variance inflation factor exceeds 10, the regression model has serious multicollinearity.

[0058] In this embodiment, in order to ensure that there is no strong correlation between the feature data, multicollinearity analysis is performed on the updated features to obtain the variance inflation factor result table of each feature data in Table 2. It can be seen that the updated feature data does not have a variance inflation factor greater than or equal to 10. Therefore, in this embodiment, the independent feature data is equal to the updated feature data, namely: sending time send, node priority priority, time slot interval, traffic mode traffic, and queue length y.

[0059] Table 2

[0060] Feature data Variance Inflation Factor send 3.066 priority 3.345 interval 1.625 traffic 3.292 y 8.279

[0061] S3. Construct a data set T based on the independent feature data, where the data set T includes: a number of time series with preset step lengths and their corresponding queue lengths; and divide the data set T into a training set and a test set.

[0062] Based on independent feature data, a time series with a step size of 10 is used to construct a data set T. The first 80% of the data in data set T is used as the training set T train , the last 20% of the data is used as the test set T test Specifically, the data of the 10 steps before the moment is used as training data to predict the queue length at the moment. Table 3 is a data feature table of data set T.

[0063] Table 3

[0064]

[0065] S4. Construct a neural network based on Transformer and GRU, take the time series with preset step length in the training set as input and the corresponding TSN switch queue length as output, train the neural network, and obtain the TSN switch queue length prediction model M. init ;

[0066] refer to Figure 1 , the neural network based on Transformer and GRU includes: N sequentially connected Encoder modules, the first Linear layer, and the second Linear layer; the first Encoder module serves as the input end of the neural network of Transformer and GRU, and the last Encoder module is sequentially connected to the first Linear layer and the second Linear layer, and the second Linear layer serves as the output end of the neural network of Transformer and GRU. The Encoder module includes: the first Feed Forward layer, the second Feed Forward layer, the Multi-Head Attention layer, the GRU layer, the first Add&Norm layer, the second Add&Norm layer, the third Add&Norm layer, and the addition operation module; in the present invention, the first Feed Forward layer and the second Feed Forward layer are feedforward neural networks, the Multi-Head Attention layer is multi-head attention, and the GRU layer (Gated recurrent unit) is a gated recurrent unit. The first Add&Norm layer, the second Add&Norm layer, and the third Add&Norm layer all use batch normalization.

[0067] The input of the first Feed Forward layer, the input of the GRU layer, and the input of the first Add&Norm layer are used as the input of the Encoder module. The output of the first Feed Forward layer is connected to the input of the Multi-Head Attention layer and the input of the second Add&Norm layer, respectively.

[0068] The output end of the Multi-Head Attention layer is connected to the input end of the second Add&Norm layer. In this embodiment, Multi-Head=8.

[0069] The output of the GRU layer is connected to the input of the first Add&Norm layer;

[0070] The output ends of the first Add&Norm layer and the second Add&Norm layer are respectively connected to the input ends of the addition operation module;

[0071] The output end of the addition operation module is connected to the input end of the third Add&Norm layer and the input end of the second Feed Forward layer respectively;

[0072] The output of the second Feed Forward layer is connected to the input of the third Add&Norm layer;

[0073] The output of the third Add&Norm layer serves as the output of the Encoder module.

[0074] In step S4, the neural network is trained with the time series of preset step lengths in the training set as input and the corresponding TSN switch queue length as output to obtain the TSN switch queue length prediction model M init ; During the training process, MSE is used as the loss function for training, and MSE is the mean square error:

[0075]

[0076] Where n is the number of samples, y i is the true value of the i-th sample, is the predicted value of the i-th sample. The smaller the MSE value, the smaller the prediction error.

[0077] S5. Input the time series into the TSN switch queue length prediction model M init Based on the MSE evaluation index, the TSN switch queue length prediction model M init On the test set T test Perform performance evaluation on.

[0078] In this embodiment, two methods, LSTM (Long short-term memory) and BPNN (Backpropagation neural network) in the art are selected as benchmarks to predict the queue length of a queue in a TSN switch. Table 4 is a table of prediction performance of each model. It can be found that the present invention can further improve the performance of the TSN switch queue length prediction model.

[0079] Table 4

[0080]

[0081] Although the present invention has been described above with preferred embodiments, it is not intended to limit the present invention. A person skilled in the art of the present invention may make various modifications and improvements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the definition of the claims.

Claims

1. A TSN switch queue length prediction method based on Transformer and GRU, characterized in that: The steps include: S1. Build a centrally managed network topology based on TSN switches and collect characteristic data of the network topology; the characteristic data includes: sending time, receiving time, difference between sending time and receiving time, sending packet size, switch time, difference between sending time and switch time, difference between receiving time and switch time, sending node, node priority, time slot, traffic pattern, throughput, and queue length; S2, extracting independent feature data from the feature data; specifically comprising the following sub-steps: S201, calculate the Spearman correlation coefficient for the feature data, remove the feature data whose absolute value of the correlation coefficient is greater than or equal to the threshold, and obtain updated feature data; specifically: calculate the Spearman correlation coefficient for the two sets of feature data X={x i |0<i≤n} and Y={y i |0<i≤n} to calculate the Spearman correlation coefficient as follows: Where n is the number of samples, d i is the level difference between the samples corresponding to the two sets of feature data, r∈[-1,1]; S202, performing multicollinearity analysis on the updated feature data, removing feature data with variance inflation factors greater than or equal to a threshold, and obtaining independent feature data; the calculation formula of the variance inflation factor is as follows: Among them, R i is the negative correlation coefficient of the regression analysis of the i-th characteristic data on the other characteristic data; S3. constructing a data set T based on the independent feature data, wherein the data set T includes: a time series of several preset step lengths and their corresponding queue lengths; dividing the data set T into a training set and a test set; S4. Construct a neural network based on Transformer and GRU, take the time series with preset step length in the training set as input and the corresponding TSN switch queue length as output, train the neural network, and obtain the TSN switch queue length prediction model M. init ; S5. Input the time series into the TSN switch queue length prediction model M init Get the TSN switch queue length.

2. According to claim 1, a TSN switch queue length prediction method based on Transformer and GRU is characterized in that: In step S4, the neural network based on Transformer and GRU includes: N sequentially connected Encoder modules, a first Linear layer, and a second Linear layer; the first Encoder module serves as the input end of the neural network of Transformer and GRU, and the last Encoder module is connected to the first Linear layer and the second Linear layer in sequence, and the second Linear layer serves as the output end of the neural network of Transformer and GRU.

3. According to a TSN switch queue length prediction method based on Transformer and GRU according to claim 1, it is characterized in that: The Encoder module includes: the first Feed Forward layer, the second Feed Forward layer, the Multi-Head Attention layer, the GRU layer, the first Add&Norm layer, the second Add&Norm layer, the third Add&Norm layer, and the addition operation module; The input of the first Feed Forward layer, the input of the GRU layer, and the input of the first Add&Norm layer are used as the input of the Encoder module. The output of the first Feed Forward layer is connected to the input of the Multi-Head Attention layer and the input of the second Add&Norm layer respectively. The output of the Multi-Head Attention layer is connected to the input of the second Add&Norm layer; The output of the GRU layer is connected to the input of the first Add&Norm layer; The output ends of the first Add&Norm layer and the second Add&Norm layer are respectively connected to the input ends of the addition operation module; The output end of the addition operation module is connected to the input end of the third Add&Norm layer and the input end of the second Feed Forward layer respectively; The output of the second Feed Forward layer is connected to the input of the third Add&Norm layer; The output of the third Add&Norm layer is used as the output of the Encoder module.

4. According to a TSN switch queue length prediction method based on Transformer and GRU according to claim 1, it is characterized in that: It also includes the TSN switch queue length prediction model M based on the MSE evaluation index. init Performance evaluation is performed on the test set, MSE is the mean square error: Where n is the number of samples, y i is the true value of the i-th sample, y i is the predicted value of the i-th sample.

5. According to a method for predicting queue length of a TSN switch based on Transformer and GRU according to claim 1, it is characterized in that: In step S3, the preset step length is 10.

6. A TSN switch queue length prediction method based on Transformer and GRU according to claim 1 The detection method is characterized in that In step S201, feature data with correlation coefficient greater than or equal to 0.7 is removed. Get updated feature data; In step S202, feature data with a variance inflation factor greater than or equal to 10 is removed to obtain independent feature data.

Images