Traffic detection method, traffic detection device, electronic equipment and storage medium

Abstract

This disclosure provides a traffic detection method, a traffic detection device, an electronic device, and a storage medium, relating to the field of communication technology. The method includes: acquiring traffic data to be detected and obtaining a word embedding matrix based on the traffic data; performing principal component analysis on the word embedding matrix to obtain multiple principal components; determining a target number of target principal components from the multiple principal components, and setting the number of attention heads in the multi-head attention module of the converter model to the target number; for each attention head, determining the output matrix of the attention head based on the key weight matrix, value weight matrix, and query weight matrix of the attention head, as well as the projection of the word embedding matrix onto the target principal component corresponding to the attention head; determining the output matrix of the attention module based on the output matrices of each attention head; and processing the output matrix of the attention module through the converter model to obtain the traffic category corresponding to the traffic data to be detected.

Description

Technical Field

[0001] This disclosure relates to the field of communication technology, and in particular to a flow detection method, flow detection device, electronic device and storage medium. Background Technology

[0002] The rapid development of the Internet of Things (IoT) has brought many potential dangers to mobile communication networks. Network attacks can originate from any IoT connection, making abnormal traffic detection increasingly important.

[0003] In related technologies, traffic data is usually detected by a pre-trained neural network model to obtain the traffic category. However, this does not take into account the characteristics of different input data, resulting in poor model adaptability.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this disclosure is to provide a flow detection method, flow detection device, electronic device, and storage medium, which can improve the accuracy of flow detection in a converter model and enhance the model's flexibility and adaptability.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0007] This disclosure provides a traffic detection method, which includes: acquiring traffic data to be detected and obtaining a word embedding matrix based on the traffic data to be detected; performing principal component analysis based on the word embedding matrix to obtain multiple principal components; determining a target number of target principal components from the multiple principal components, and setting the number of attention heads in the multi-head attention module of the converter model to the target number; for each attention head, determining the output matrix of the attention head based on the key weight matrix, value weight matrix, and query weight matrix of the attention head, and the projection of the word embedding matrix onto the target principal component corresponding to the attention head; determining the output matrix of the attention module based on the output matrices of each attention head; and processing the output matrix of the attention module through the converter model to obtain the traffic category corresponding to the traffic data to be detected.

[0008] In an exemplary embodiment, principal component analysis is performed based on the word embedding matrix to obtain multiple principal components, including: centering and standardizing the word embedding matrix to obtain an intermediate word embedding matrix; determining the covariance matrix of the intermediate word embedding matrix; performing eigenvalue decomposition on the covariance matrix to obtain multiple eigenvectors and eigenvalues ​​corresponding to each eigenvector, and using the multiple eigenvectors as the multiple principal components.

[0009] In an exemplary embodiment, determining a target number of target principal components from a plurality of principal components includes: sorting the plurality of feature vectors according to the feature values ​​corresponding to each feature vector; and determining the top target number of principal components as target principal components.

[0010] In an exemplary embodiment, determining the output matrix of the attention head based on the key weight matrix, value weight matrix, and query weight matrix of the attention head, and the projection of the word embedding matrix onto the target principal component corresponding to the attention head, includes: determining the key matrix of the attention head based on the projection of the key weight matrix and the word embedding matrix onto the target principal component corresponding to the attention head; determining the value matrix of the attention head based on the projection of the value weight matrix and the word embedding matrix onto the target principal component corresponding to the attention head; determining the query matrix of the attention head based on the projection of the query weight matrix and the word embedding matrix onto the target principal component corresponding to the attention head; and determining the output matrix of the attention head based on the key matrix, the value matrix, and the query matrix.

[0011] In an exemplary embodiment, determining the attention module output matrix based on the output matrices of each attention head includes: concatenating the output matrices of each attention head and then performing a linear transformation to obtain the attention module output matrix.

[0012] In an exemplary embodiment, the loss function corresponding to the converter model is determined based on the difference between the attention weight matrix and the principal component matrix, wherein the attention weight matrix is ​​determined based on the key weight matrix, the value weight matrix, and the query weight matrix, and the principal component matrix is ​​determined based on the plurality of principal components.

[0013] In an exemplary embodiment, the key weight matrix, value weight matrix, and query weight matrix of the attention head are obtained based on the initial key weight matrix, the initial value weight matrix, and the initial query weight matrix, which are initialized based on the principal components of the training data.

[0014] This disclosure provides a traffic detection device, comprising: an acquisition module for acquiring traffic data to be detected and obtaining a word embedding matrix based on the traffic data to be detected; an analysis module for performing principal component analysis based on the word embedding matrix to obtain multiple principal components; a determination module for determining a target number of target principal components from the multiple principal components and setting the number of attention heads in the multi-head attention module of the converter model to the target number; the determination module is further configured to, for each attention head, determine the output matrix of the attention head based on the key weight matrix, value weight matrix, and query weight matrix of the attention head, and the projection of the word embedding matrix onto the target principal component corresponding to the attention head; the determination module is further configured to determine the output matrix of the attention module based on the output matrices of each attention head; and the acquisition module is further configured to process the output matrix of the attention module through the converter model to obtain the traffic category corresponding to the traffic data to be detected.

[0015] This disclosure provides an electronic device, including: at least one processor; and a storage terminal device for storing at least one program, which, when executed by the at least one processor, causes the at least one processor to implement any of the above-described traffic detection methods.

[0016] This disclosure provides a computer-readable storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements any of the above-described traffic detection methods.

[0017] The traffic detection method provided in this disclosure performs principal component analysis on the word embedding matrix of the traffic data to be detected, sets the attention heads of the converter model based on the principal component analysis results, and determines the output matrix of each attention head based on the principal component analysis results. This method can set and adjust the converter model accordingly based on the principal components that can reflect the complexity of the data, that is, dynamically adjust the number of attention heads according to the complexity of the input data, so that the converter model can better adapt to different input data and tasks, improve the accuracy of traffic detection of the converter model, and improve the flexibility and adaptability of the model.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0020] Figure 1 This is a flowchart illustrating a traffic detection method according to an exemplary embodiment.

[0021] Figure 2 This is a schematic diagram illustrating word embedding in traffic data to be detected, as shown in an example.

[0022] Figure 3 This is a schematic diagram based on the word vector weight matrix training shown in the example.

[0023] Figure 4 This is a schematic diagram based on an example of a continuous jump character model.

[0024] Figure 5 This is a schematic diagram illustrating the construction of a converter model based on principal component analysis, as shown in an example.

[0025] Figure 6 This is a schematic diagram of a converter model illustrated in an example.

[0026] Figure 7 This is a block diagram of a flow detection device in an exemplary embodiment of the present disclosure.

[0027] Figure 8 This is a schematic diagram of the structure of an electronic device according to an exemplary embodiment. Detailed Implementation

[0028] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0029] The features, structures, or characteristics described in this disclosure can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more specific details omitted, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0030] The accompanying drawings are merely illustrative of this disclosure, and the same reference numerals in the drawings denote the same or similar parts, thus omitting repeated descriptions of them. Some block diagrams shown in the drawings do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in at least one hardware module or integrated circuit, or in different network and / or processor devices and / or microcontroller devices.

[0031] The flowchart shown in the accompanying drawings is merely illustrative and does not necessarily include all content and steps, nor does it require execution in the described order. For example, some steps may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0032] Furthermore, in the description of this disclosure, the terms “a,” “one,” “the,” “the,” and “at least one” are used to indicate the presence of at least one element or component; the terms “comprising,” “including,” and “having” are used to indicate an open-ended inclusion and to mean that there may be other elements or components in addition to the listed elements or components; the terms “first,” “second,” and “third,” etc., are used only as labels and are not a limitation on the number of objects.

[0033] The exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0034] Figure 1 This is a flowchart illustrating a traffic detection method according to an exemplary embodiment. The traffic detection method provided in this disclosure can be executed by any electronic device, and this disclosure does not limit it.

[0035] like Figure 1 As shown, the traffic detection method provided in this embodiment may include the following steps.

[0036] In step S102, the traffic data to be detected is obtained, and the word embedding matrix is ​​obtained based on the traffic data to be detected.

[0037] In this embodiment of the disclosure, the traffic data to be detected can be network traffic data in text form. By detecting the traffic data to be detected, the traffic category of the traffic data to be detected can be obtained. The traffic category can include normal traffic category and abnormal traffic category.

[0038] In this embodiment, the flow data to be detected can be data from an Internet of Things (IoT) connection. IoT connection data can be categorized into sensor data, operational data, maintenance record data, environmental condition data, and equipment characteristic data. Sensor data includes real-time data on various equipment parameters collected by sensors, such as temperature, pressure, vibration, current, and voltage. This data can be used to monitor the equipment's operating status and performance indicators, as well as detect any anomalies. Operational data includes the equipment's operating time, work cycle, speed, and rotational speed, providing basic information about the equipment's operation and laying the foundation for prediction and analysis. Maintenance record data can include the equipment's maintenance history, maintenance activities, and repair records. This data can be used to analyze the equipment's maintenance needs and effectiveness to optimize maintenance strategies. Environmental condition data includes environmental parameters such as temperature, humidity, and air pressure. Environmental conditions have a certain impact on the equipment's operating status and performance; therefore, monitoring and recording environmental data is also important for maintenance decisions. Equipment characteristic data includes the equipment's specifications, model, production date, and parts information. This data can be used to build a baseline model of the equipment and conduct comparative analysis to determine the equipment's health status.

[0039] Figure 2 This is a schematic diagram illustrating word embedding in traffic data to be detected, as shown in an example.

[0040] refer to Figure 2 In this embodiment of the disclosure, word embedding processing can be performed on the traffic data to be detected (e.g., network traffic text data) to obtain the word embedding matrix (also known as the word vector matrix) of the traffic data to be detected; word embedding is a technique that maps words or phrases in text to a continuous vector space, which can capture the semantic and grammatical relationships between words.

[0041] For example, the Word2Vec (Word to Vector) word embedding algorithm can be used. Word2Vec converts each word into a relatively low-dimensional continuous vector, allowing these vectors to capture the semantic and syntactic relationships between words. This enables semantically similar words to be closely spaced within an embedding space. A specific implementation might involve: first, training a neural network; then, using the hidden layers of this network to calculate the probability distribution of the input words; and finally, selecting the word with the highest probability for output and representation. Since this process does not require manual annotation, it is an unsupervised learning method.

[0042] Figure 3 This is a schematic diagram based on the word vector weight matrix training shown in the example.

[0043] refer to Figure 3One-hot encoding can be used to encode network traffic data, resulting in a one-hot encoded vector, which can be, for example, a 1*10000 vector. Then, hidden linear units, linear units, and an output layer are used to calculate the probability set for each input word. Each probability represents the likelihood that the current word is an output word in the input sample. The word with the highest probability is selected for output and representation. This is achieved by sending IoT traffic data into... Figure 3 The input and output ends of the neural network shown are trained to obtain an embedding matrix with rich weight information after multiple iterations.

[0044] Figure 4 This is a schematic diagram based on an example of a continuous jump character model.

[0045] In this embodiment of the disclosure, the Word2Vec word embedding algorithm based on the Skip-Gram model can be used. In the Skip-Gram model, each input word is used to predict the words around it, that is, given a word, it is used to predict the context.

[0046] refer to Figure 4 W(t) represents the current input word, W(t-2) represents the second word before the current word, W(t-1) represents the word before the current word, W(t+1) represents the word after the current word, and so on. The number of words to predict each time is determined by the window. Here, the window value is 2 (representing the prediction of 2 words before and after the current word). The window value can be set according to actual needs.

[0047] In step S104, principal component analysis is performed based on the word embedding matrix to obtain multiple principal components.

[0048] In this embodiment of the disclosure, after word embedding processing of IoT traffic data, all IoT traffic data can be converted into word embedding vectors. However, since the types of industrial scenario data that IoT connections are oriented towards are diverse, the data will change significantly each time a new industrial terminal is connected or a new industrial scenario is encountered, thus changing the complexity of the problem. By performing principal component analysis (PCA) on the word embedding matrix, a model that matches the data can be quickly formed according to the industrial scenario.

[0049] Principal Component Analysis (PCA) is an unsupervised machine learning method primarily used for dimensionality reduction of high-dimensional data. The core idea of ​​PCA is to find a new coordinate system in which the projection of the original data maximizes variance, allowing most data variations to be described with fewer dimensions, thus achieving dimensionality reduction. For example, consider an m×n data matrix X, where each row represents an observation and each column represents a feature. In PCA, a new coordinate system can be found that maximizes variance when the data is projected onto it. In this new coordinate system, each coordinate axis (i.e., each principal component) is a linear combination in the original feature space. The first principal component is the direction of maximum variance in the original data, meaning the data points are most widely distributed in this direction; the second principal component is orthogonal to the first and has the second largest variance, and so on. Therefore, PCA is essentially searching for a rotational transformation such that the covariance matrix of the data is diagonal in the new coordinate system, with the diagonal elements arranged in descending order. Geometrically, PCA is like rotating an ellipsoid so that its principal axes are parallel to the coordinate axes. In this way, by extracting the first k principal axes (i.e., the first k principal components), a low-dimensional approximation of the original data can be obtained. Therefore, the physical meaning of PCA can be understood as: finding a new coordinate system such that the projection of the data into this new coordinate system maximizes the variance (i.e., keeping the data constant while dispersing it more widely). This allows most of the data variation to be described with fewer dimensions, thus achieving dimensionality reduction.

[0050] In this embodiment of the disclosure, principal component analysis of the matrix can be used to find the most important feature directions in the data, which can contain most of the information in the input data.

[0051] In an exemplary embodiment, principal component analysis is performed based on the word embedding matrix to obtain multiple principal components, including: centering and standardizing the word embedding matrix to obtain an intermediate word embedding matrix; determining the covariance matrix of the intermediate word embedding matrix; performing eigenvalue decomposition on the covariance matrix to obtain multiple eigenvectors and eigenvalues ​​corresponding to each eigenvector, and using the multiple eigenvectors as multiple principal components.

[0052] Specifically, the word embedding matrix X is first preprocessed, and then centered and standardized using formula (1). The resulting matrix can be called the intermediate word embedding matrix X′.

[0053] X′=(X-mean(X)) / std(X) (1)

[0054] The covariance matrix C of the middle word embedding matrix X′ is calculated using formula (2).

[0055] C=X^TX / (m-1) (2)

[0056] Where m represents the dimension of the matrix.

[0057] After calculating the covariance matrix, eigenvalue decomposition is performed to obtain multiple sets of eigenvectors and eigenvalues. The eigenvectors of the covariance matrix represent the directions of the principal components of the data, and the eigenvalues ​​represent the variance of the corresponding directions. Sorting the eigenvectors according to the magnitude of their corresponding eigenvalues ​​yields the principal components.

[0058] In step S106, a target number of target principal components are determined from multiple principal components, and the number of attention heads in the multi-head attention module of the converter model is set to the target number.

[0059] In an exemplary embodiment, determining a target number of target principal components from multiple principal components includes: sorting multiple feature vectors according to the feature values ​​corresponding to each feature vector; and determining the top target number of principal components as target principal components.

[0060] In this embodiment of the disclosure, the eigenvectors can be sorted according to the magnitude of the eigenvalues, and the first h principal components can be selected as the target principal components. The value of h can be set according to the actual situation. For example, the more complex the original data, the larger the value of h should be.

[0061] In this embodiment of the disclosure, the first h principal components are selected as target principal components, and the number of attention heads in the multi-head attention module of the Transformer model is also set to h.

[0062] In this embodiment, principal components are used to set up the multi-head attention mechanism of the converter model, so that each attention head corresponds to one principal component. In this way, each attention head can focus on a unique information direction, which can better capture the diversity of data.

[0063] In step S108, for each attention head, the output matrix of the attention head is determined based on the key weight matrix, value weight matrix, query weight matrix of the attention head, and the projection of the word embedding matrix onto the target principal component corresponding to the attention head.

[0064] In this embodiment of the disclosure, each attention head corresponds to its own key weight matrix, value weight matrix and query weight matrix. The output matrix of each attention head is determined based on the projection of the key weight matrix, value weight matrix and query weight matrix corresponding to each attention head and the word embedding matrix onto the target principal component corresponding to the attention head.

[0065] The following explanation uses one of the attention points as an example.

[0066] In an exemplary embodiment, determining the output matrix of the attention head based on the projections of the key weight matrix, value weight matrix, query weight matrix, and word embedding matrix onto the target principal component corresponding to the attention head includes: determining the key matrix of the attention head based on the projections of the key weight matrix and word embedding matrix onto the target principal component corresponding to the attention head; determining the value matrix of the attention head based on the projections of the value weight matrix and word embedding matrix onto the target principal component corresponding to the attention head; determining the query matrix of the attention head based on the projections of the query weight matrix and word embedding matrix onto the target principal component corresponding to the attention head; and determining the output matrix of the attention head based on the key matrix, value matrix, and query matrix.

[0067] For the i-th attention point, its corresponding query matrix Q can be calculated using the following formula. i Key matrix K i Sum matrix V i .

[0068] Q i =WQ i *X i (3)

[0069] K i =WK i *X i (4)

[0070] V i =WV i *X i (5)

[0071] Among them, X i WQ represents the projection of the data along the direction of the i-th principal component. i WK i WV i These represent the corresponding weight matrices.

[0072] Then, based on the key matrix, the value matrix of the attention head, and the query matrix of the attention head, a scaled dot product attention operation is performed to determine the output matrix of the attention head.

[0073] A i =softmax(Q) i *K i ^T / sqrt(d_k))*V i (6)

[0074] Where d_k represents the dimension of the key, A i This represents the output of the i-th attention head.

[0075] Figure 5 This is a schematic diagram illustrating the construction of a converter model based on principal component analysis, as shown in an example.

[0076] refer to Figure 5 In this embodiment of the disclosure, the word vector matrix (also known as the word embedding matrix) is subjected to linear algebraic transformation to form a matrix and principal component analysis is performed. The obtained principal components guide the attention count of the converter model. For each attention head, the query matrix, key matrix and value matrix are calculated, thereby calculating the output matrix of the attention module, and then processing it through a feedforward neural network.

[0077] In an exemplary embodiment, the key weight matrix, value weight matrix, and query weight matrix of the attention head are obtained based on the initial key weight matrix, initial value weight matrix, and initial query weight matrix, which are initialized based on the principal components of the training data.

[0078] In this embodiment of the disclosure, principal component analysis can be used to help the converter model initialize the weight matrix. Specifically, for each attention head, the key weight matrix, value weight matrix, and query weight matrix are initialized based on the principal components of the input data to obtain the initial key weight matrix, initial value weight matrix, and initial query weight matrix. Then, through continuous iterative optimization by model training, the key weight matrix, value weight matrix, and query weight matrix of each attention head in actual application are obtained.

[0079] Suppose that PCA is performed on the input data to obtain p principal components. For each attention head of the Transformer model, these feature vectors can be used to initialize its corresponding weight matrix and the attention matrix. By using these principal components as the initial values ​​of the attention weights, the model can converge to a good solution more quickly.

[0080] In step S110, the output matrix of the attention module is determined based on the output matrix of each attention head.

[0081] In an exemplary embodiment, determining the attention module output matrix based on the output matrices of each attention head includes: concatenating the output matrices of each attention head and then performing a linear transformation to obtain the attention module output matrix.

[0082] Specifically, the attention module output matrix can be calculated using the following formula.

[0083] A = [A1; A2; ...; A h ]*WO (7)

[0084] Among them, [A1; A2; ...; A h] indicates that the outputs of all attention heads are connected together, WO represents the output weight matrix, and A represents the output of the attention module.

[0085] In step S112, the output matrix of the attention module is processed by the converter model to obtain the traffic category corresponding to the traffic data to be detected.

[0086] Figure 6 This is a schematic diagram of a converter model illustrated in an example.

[0087] In this embodiment of the disclosure, reference is made to Figure 6 The converter model can include a multi-head self-attention module, a feedforward neural network, a jump and regularization module, and an output module. The output matrix of the attention module can be input into the feedforward neural network, and processed sequentially by the feedforward neural network, the jump and regularization module, and the output module to obtain the traffic category corresponding to the traffic data to be detected (i.e., the network traffic classification result in the figure).

[0088] Specifically, the self-attention mechanism in the Transformer is responsible for capturing long-distance dependencies in the input sequence. The self-attention mechanism generates a weighted representation by calculating the relationship between each word and other words in the input sequence, which is then used for subsequent network layer processing. In this embodiment, multiple self-attention mechanisms (e.g., eight) can be employed.

[0089] Specifically, the self-attention mechanism is calculated using three weight matrices: Q (Query), K (Key), and V (Value), representing the query, key, and value, respectively. Specifically, the input sequence is first transformed into Q, K, and V vectors through a linear layer; then, the dot product of Q and K is calculated to measure the contribution of each word in the input sequence to the current word; this dot product result is further normalized using a softmax function to obtain the final attention weights.

[0090] Specifically, a feedforward neural network may include two linear layers (fully connected layers) and an activation function, which can capture the non-linear features (RELU) of the input data.

[0091] Specifically, skip connections (also known as residual connections) and layer normalization can be used to optimize network performance.

[0092] Among them, skip connections can directly add the input of the feedforward neural network to its output, thereby achieving "skip" transmission of the original input; this structure helps to alleviate the gradient vanishing problem, enabling the model to perform deep training more effectively.

[0093] Layer normalization (also known as regularization) normalizes the output of each layer, ensuring smoother information transfer between different layers in the network and avoiding gradient explosion or vanishing problems.

[0094] In an exemplary embodiment, the loss function corresponding to the converter model is determined based on the difference between the attention weight matrix and the principal component matrix. The attention weight matrix is ​​determined based on the key weight matrix, the value weight matrix, and the query weight matrix, and the principal component matrix is ​​determined based on multiple principal components.

[0095] In this embodiment of the disclosure, the loss function of the converter model can be improved based on the principal component matrix. Specifically, the initial loss function L of the converter model is determined according to the predicted category and label. Based on the initial loss function L, the difference between the attention weight matrix and the principal component matrix is ​​added to obtain the target loss function L′. The model is then trained based on the target loss function.

[0096] Specifically, the target loss function L′ can be determined using the following formula.

[0097] L′ = L + λ * ||B - E||^2_F (8)

[0098] Where B represents the attention weight matrix of the model, which consists of the key weight matrix, the value weight matrix, and the query weight matrix; E represents the principal component matrix (the matrix composed of feature vectors); ||·||_F represents the Frobenius norm (the square root of the sum of the squares of all elements in the matrix); and λ is a hyperparameter that controls the strength of regularization.

[0099] In this embodiment of the disclosure, the regularization term encourages the model's attention weights B to be close to the principal components E. If λ is set large, the model will be more inclined to focus its attention on the principal components; if λ is set small, the model will have greater freedom to explore other possible attention distributions.

[0100] The traffic detection method provided in this disclosure performs principal component analysis on the word embedding matrix of the traffic data to be detected, sets the attention heads of the converter model based on the principal component analysis results, and determines the output matrix of each attention head based on the principal component analysis results. This method can set and adjust the converter model accordingly based on the principal components that can reflect the complexity of the data, that is, dynamically adjust the number of attention heads according to the complexity of the input data, so that the converter model can better adapt to different input data and tasks, improve the accuracy of traffic detection of the converter model, and improve the flexibility and adaptability of the model.

[0101] Furthermore, in this embodiment of the disclosure, each principal component can be used to initialize the weight matrix of the corresponding attention head. This method utilizes the results of principal component analysis, namely the most important feature directions in the data, which can make the model's learning process faster and more effective.

[0102] Furthermore, in this embodiment of the present disclosure, the difference between the attention weights and the principal components is used as a regularization term in the loss function of the converter model; in this way, the model can be encouraged to focus more attention on the principal components, thereby better capturing the main characteristics of the data.

[0103] The traffic detection method provided in this disclosure can be applied in the following areas: Intrusion detection and prevention: used to detect potential network attack behaviors, such as distributed denial-of-service (DDoS) attacks, botnets, and software propagation; Data leakage protection: detecting abnormal behaviors that may lead to data leakage, such as stealing user information or attacking internal systems; Load balancing and optimization: helping network operators detect and resolve traffic congestion problems; Network behavior analysis: conducting in-depth analysis of user behavior to identify fraudulent behavior; The traffic detection method provided in this disclosure can effectively identify and prevent potential network attacks, providing security for mobile communication networks.

[0104] It should also be understood that the above is only to help those skilled in the art better understand the embodiments of this disclosure, and is not intended to limit the scope of the embodiments of this disclosure. Those skilled in the art can obviously make various equivalent modifications or changes based on the examples given above. For example, some steps in the above methods may be unnecessary, or new steps may be added, etc. Alternatively, any combination of any two or more of the above embodiments may be used. Such modifications, changes, or combinations also fall within the scope of the embodiments of this disclosure.

[0105] It should also be understood that the above description of the embodiments of this disclosure focuses on highlighting the differences between the various embodiments. Similarities or differences not mentioned can be referred to each other, and for the sake of brevity, they will not be repeated here.

[0106] It should also be understood that the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.

[0107] It should also be understood that, in the various embodiments of this disclosure, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions between different embodiments are consistent and can be referenced by each other, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0108] The foregoing has detailed the method examples provided in this disclosure. It is understood that, in order to implement the above functions, a computer device includes corresponding hardware structures and / or software modules for performing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is implemented in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0109] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0110] Figure 7 This is a block diagram of a flow detection device in an exemplary embodiment of the present disclosure.

[0111] like Figure 7 As shown, the flow detection device 700 may include: an acquisition module 702, an analysis module 704, a determination module 706, and an acquisition module 708.

[0112] The module 702 is used to acquire traffic data to be detected and obtain a word embedding matrix based on the traffic data to be detected; the analysis module 704 is used to perform principal component analysis based on the word embedding matrix to obtain multiple principal components; the determination module 706 is used to determine a target number of target principal components from the multiple principal components and set the number of attention heads in the multi-head attention module of the converter model to the target number; the determination module 706 is also used to determine the output matrix of the attention head for each attention head based on the key weight matrix, value weight matrix, and query weight matrix of the attention head, and the projection of the word embedding matrix onto the target principal component corresponding to the attention head; the determination module 706 is also used to determine the output matrix of the attention module based on the output matrices of each attention head; the acquisition module 708 is used to process the output matrix of the attention module through the converter model to obtain the traffic category corresponding to the traffic data to be detected.

[0113] In an exemplary embodiment, the analysis module 704 is configured to: center and standardize the word embedding matrix to obtain an intermediate word embedding matrix; determine the covariance matrix of the intermediate word embedding matrix; perform eigenvalue decomposition on the covariance matrix to obtain multiple eigenvectors and eigenvalues ​​corresponding to each eigenvector, and use the multiple eigenvectors as the multiple principal components.

[0114] In an exemplary embodiment, the determining module 706 is configured to: sort the plurality of feature vectors according to the feature values ​​corresponding to each feature vector; and determine the first target number of principal components as target principal components.

[0115] In an exemplary embodiment, the determining module 706 is configured to: determine the key matrix of the attention head based on the projection of the key weight matrix of the attention head and the word embedding matrix onto the target principal component corresponding to the attention head; determine the value matrix of the attention head based on the projection of the value weight matrix of the attention head and the word embedding matrix onto the target principal component corresponding to the attention head; determine the query matrix of the attention head based on the query weight matrix of the attention head and the projection of the word embedding matrix onto the target principal component corresponding to the attention head; and determine the output matrix of the attention head based on the key matrix, the value matrix, and the query matrix.

[0116] In an exemplary embodiment, the determining module 706 is configured to: connect the output matrices of each attention head and then perform a linear transformation to obtain the output matrix of the attention module.

[0117] In an exemplary embodiment, the loss function corresponding to the converter model is determined based on the difference between the attention weight matrix and the principal component matrix, wherein the attention weight matrix is ​​determined based on the key weight matrix, the value weight matrix, and the query weight matrix, and the principal component matrix is ​​determined based on the plurality of principal components.

[0118] In an exemplary embodiment, the key weight matrix, value weight matrix, and query weight matrix of the attention head are obtained based on the initial key weight matrix, the initial value weight matrix, and the initial query weight matrix, which are initialized based on the principal components of the training data.

[0119] It should be noted that the block diagrams shown in the above figures are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor terminal devices and / or microcontroller terminal devices.

[0120] Figure 8 This is a schematic diagram illustrating the structure of an electronic device according to an exemplary embodiment. It should be noted that... Figure 8 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0121] like Figure 8As shown, the electronic device 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage section 808 into a random access memory (RAM) 803. The RAM 803 also stores various programs and data required for the operation of the electronic device 800. The CPU 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0122] The following components are connected to I / O interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to I / O interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 810 as needed so that computer programs read from it can be installed into storage section 808 as needed.

[0123] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by central processing unit (CPU) 801, it performs the functions defined above in the system of this disclosure.

[0124] It should be noted that the computer-readable medium disclosed herein may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, terminal device, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, terminal device, or device. In this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit programs for use by or in connection with an instruction execution system, terminal device, or apparatus. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0125] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0126] The units described in the embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor can be described as including a sending unit, an acquisition unit, a determining unit, and a first processing unit. The names of these units do not necessarily limit the specific unit; for example, a sending unit can also be described as "a unit that sends an image acquisition request to a connected server."

[0127] In another aspect, this disclosure also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable storage medium carries one or more programs that, when executed by the electronic device, cause the electronic device to perform the methods described in the following embodiments. For example, the electronic device may perform... Figure 1 The steps shown.

[0128] According to one aspect of this disclosure, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various optional implementations of the above embodiments.

[0129] It should be understood that any number of elements in the accompanying drawings is for illustrative purposes only and not for limitation, and any naming is for distinction only and has no limiting meaning.

[0130] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0131] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A flow rate detection method, characterized in that, include: Obtain the traffic data to be detected, and obtain the word embedding matrix based on the traffic data to be detected; Principal component analysis was performed based on the word embedding matrix to obtain multiple principal components; A target number of target principal components are determined from the multiple principal components, and the number of attention heads in the multi-head attention module of the converter model is set to the target number. For each attention head, the output matrix of the attention head is determined based on the key weight matrix, value weight matrix, and query weight matrix of the attention head, as well as the projection of the word embedding matrix onto the target principal component corresponding to the attention head. The output matrix of the attention module is determined based on the output matrix of each attention head. The output matrix of the attention module is processed by the converter model to obtain the traffic category corresponding to the traffic data to be detected.

2. The method according to claim 1, characterized in that, Principal component analysis is performed based on the word embedding matrix to obtain multiple principal components, including: The word embedding matrix is ​​centered and normalized to obtain the intermediate word embedding matrix; Determine the covariance matrix of the intermediate word embedding matrix; The covariance matrix is ​​subjected to eigenvalue decomposition to obtain multiple eigenvectors and eigenvalues ​​corresponding to each eigenvector, and the multiple eigenvectors are used as the multiple principal components.

3. The method according to claim 2, characterized in that, Determining a target number of target principal components from the plurality of principal components includes: The multiple feature vectors are sorted according to the feature values ​​corresponding to each feature vector; The number of principal components prior to the target number are determined as the target principal components.

4. The method according to claim 1, characterized in that, Based on the key weight matrix, value weight matrix, and query weight matrix of the attention head, and the projection of the word embedding matrix onto the target principal component corresponding to the attention head, the output matrix of the attention head is determined, including: The key matrix of the attention head is determined based on the key weight matrix of the attention head and the projection of the word embedding matrix onto the target principal component corresponding to the attention head. The value matrix of the attention head is determined based on the value weight matrix of the attention head and the projection of the word embedding matrix onto the target principal component corresponding to the attention head; The query matrix of the attention head is determined based on the query weight matrix of the attention head and the projection of the word embedding matrix onto the target principal component corresponding to the attention head. The output matrix of the attention head is determined based on the key matrix, the value matrix, and the query matrix of the attention head.

5. The method according to claim 4, characterized in that, The output matrix of the attention module is determined based on the output matrices of each attention head, including: The output matrices of each attention head are concatenated and then linearly transformed to obtain the output matrix of the attention module.

6. The method according to claim 1, characterized in that, The loss function corresponding to the converter model is determined based on the difference between the attention weight matrix and the principal component matrix. The attention weight matrix is ​​determined based on the key weight matrix, the value weight matrix, and the query weight matrix. The principal component matrix is ​​determined based on the multiple principal components.

7. The method according to claim 1, characterized in that, The key weight matrix, value weight matrix, and query weight matrix of the attention head are obtained based on the initial key weight matrix, initial value weight matrix, and initial query weight matrix, which are initialized based on the principal components of the training data.

8. A flow detection device, characterized in that, include: The acquisition module is used to acquire traffic data to be detected and to obtain a word embedding matrix based on the traffic data to be detected. The analysis module is used to perform principal component analysis based on the word embedding matrix to obtain multiple principal components; A determination module is used to determine a target number of target principal components from a plurality of principal components, and to set the number of attention heads in the multi-head attention module of the converter model to the target number. The determining module is further configured to, for each attention head, determine the output matrix of the attention head based on the key weight matrix, value weight matrix, and query weight matrix of the attention head, and the projection of the word embedding matrix onto the target principal component corresponding to the attention head; The determining module is also used to determine the output matrix of the attention module based on the output matrix of each attention head; The acquisition module is used to process the output matrix of the attention module through the converter model to obtain the traffic category corresponding to the traffic data to be detected.

9. An electronic device, characterized in that, include: At least one processor; A storage device for storing at least one program, which, when executed by the at least one processor, causes the at least one processor to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images