Traffic classification methods based on session context-sensitive BERT, electronic devices, and media.

Abstract

This invention discloses a traffic classification method, electronic device, and medium based on session context-sensitive BERT, comprising: acquiring network traffic data; performing session segmentation on the network traffic data to obtain several traffic sessions; tokenizing each traffic session to obtain a session byte sequence; extracting the packet length sequence of all data packets between multiple traffic sessions within a time window and performing multi-granularity temporal aggregation to obtain an inter-session packet length sequence; constructing a session context-sensitive BERT model; the processing includes: encoding the session byte sequence and the inter-session packet length sequence to obtain a first embedding vector and a second embedding vector; using the concatenated first embedding vector and second embedding vector as training samples to construct a joint loss function, thereby pre-training the model; fine-tuning the pre-trained model for downstream traffic classification tasks; and using the fine-tuned model to perform downstream traffic classification tasks.

Description

Technical Field

[0001] This invention relates to the field of computer network security technology, and in particular to a traffic classification method, electronic device, and medium based on session context-sensitive BERT. Background Technology

[0002] Network traffic classification aims to accurately identify traffic categories from various applications or web services, and is a core technology in network management and network security. In recent years, with increasing awareness of network security, network traffic has shown a trend towards increasing encryption, posing a serious challenge to traditional traffic classification technologies. To address this challenge, traffic analysis methods based on machine learning and deep learning have developed rapidly. These existing methods typically focus on in-depth analysis of encrypted network traffic from the perspectives of plaintext fingerprint features and session statistical features, and have achieved certain results in practical applications.

[0003] However, most of the traditional methods based on machine learning or deep learning mentioned above are supervised learning algorithms, whose model performance is highly dependent on a large amount of labeled training data. In real-world network environments, obtaining massive amounts of accurately labeled traffic data often faces problems such as high cost and difficulty. To address the model training bottleneck caused by the difficulty of data labeling, traffic analysis methods based on pre-training techniques have emerged and flourished. Pre-training-based methods typically employ a pre-training and fine-tuning paradigm, that is, firstly, self-supervised learning is performed using massive amounts of unlabeled traffic data to obtain unbiased general traffic representations, and then fine-tuning is performed using only a small amount of labeled data in specific downstream tasks. Currently, traffic analysis methods based on this paradigm have achieved significant breakthroughs in several traffic classification tasks, especially few-shot classification tasks.

[0004] While traffic classification methods based on pre-training techniques offer the aforementioned advantages, current research and applications primarily remain at the session level, revealing significant limitations when facing complex network threats. Specifically, many advanced network threat traffic (such as botnet attack traffic) employ sophisticated camouflage strategies to evade detection. At the session level, these threat traffic deliberately mimics the statistical characteristics of normal business traffic (such as POP3 email service), including packet intervals and packet sizes. In such cases, relying solely on single session-level features makes it difficult to capture the subtle, essential differences between threat traffic and normal traffic. This results in the system struggling to effectively distinguish some highly disguised threat traffic from normal background traffic, leading to a risk of decreased accuracy and missed detections. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a traffic classification method, electronic device, and medium based on session context-sensitive BERT.

[0006] In a first aspect, embodiments of the present invention provide a traffic classification method for context-sensitive BERT, the method comprising: Acquire network traffic data; perform session segmentation on the network traffic data to obtain several traffic sessions; tokenize each traffic session to obtain a session byte sequence; extract the packet length sequence of all data packets between multiple traffic sessions within a time window, and perform multi-granularity time aggregation to obtain the inter-session packet length sequence; Constructing a session context-sensitive BERT model; The process of the session context-sensitive BERT model processing session byte sequences and inter-session packet long sequences includes: encoding the session byte sequences and inter-session packet long sequences to obtain a first embedding vector and a second embedding vector; using the concatenated first and second embedding vectors as training samples; constructing a joint loss function; and pre-training the session context-sensitive BERT model based on the joint loss function and the training samples. The pre-trained session context-sensitive BERT model is fine-tuned for downstream traffic classification tasks; the fine-tuned session context-sensitive BERT model is then used to perform downstream traffic classification tasks.

[0007] In a second aspect, embodiments of the present invention provide an electronic device, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the above-described context-sensitive BERT traffic classification method.

[0008] Thirdly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the aforementioned context-sensitive BERT traffic classification method.

[0009] Fourthly, embodiments of the present invention provide a computer program product, including a computer program / instruction, which, when executed by a processor, implements the aforementioned context-sensitive BERT traffic classification method.

[0010] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention provides a traffic classification method based on session context-sensitive BERT. This invention combines a multi-perspective feature extraction method of session byte sequence and inter-session packet long sequence, which can automatically learn the deep representation of encrypted traffic from massive data without manual design and feature extraction. It effectively solves the problem that traditional methods rely on human experience and are difficult to deal with encrypted traffic. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart illustrating the traffic classification method based on session context-sensitive BERT provided in this embodiment of the invention; Figure 2 A schematic block diagram of a traffic classification method based on session context-sensitive BERT provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of traffic data preprocessing provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of the session context-sensitive BERT model provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0013] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0014] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.

[0015] like Figure 1 and Figure 2 As shown, this embodiment of the invention provides a traffic classification method based on session context-sensitive BERT, the method comprising the following steps: Step S1: Obtain network traffic data; perform session segmentation on the network traffic data to obtain several traffic sessions; perform tokenization on each session to obtain a session byte sequence; extract the packet length sequence of all data packets between multiple sessions within a time window, and perform multi-granularity time aggregation to obtain the packet length sequence between sessions.

[0016] Specifically, step S1 includes the following sub-steps: Step S101: The network traffic data is segmented into a five-tuple of source IP, destination IP, source port, destination port, and protocol information, and network traffic data with consistent five-tuple information are grouped into a session.

[0017] Step S102: Tokenize each traffic session to obtain a session byte sequence.

[0018] Specifically, such as Figure 3 As shown, step S102 includes: Step S1021: Randomize the source IP, destination IP, source port, and destination port of each data packet in the traffic session; Step S1022: Recalculate the checksum fields of the network layer and transport layer protocols in the data packet to ensure that the randomized data packet remains valid at the network protocol level. Step S1023: Extract the first 8 data packets in the traffic session. If there are fewer than 8 data packets, extract all data packets and remove the link layer information such as Ethernet frames from these 8 data packets. Extract the payload portion (64 bytes in total) starting from the network layer. If the payload portion of the data packet starting from the network layer is less than 64 bytes, extract only the last byte of the data packet. Step S1024: Concatenate the payload portions (up to 512 bytes) of the first 8 data packets in the traffic session, representing them as a hexadecimal string; Step S1025 involves tokenizing the data in units of 4 hexadecimal characters, concatenating each byte with the next byte to form a token, thus obtaining the session byte sequence. For example, 451dd0 should be converted into two tokens, 451d and 1dd0.

[0019] Step S103: Extract the packet length sequence of all data packets between multiple traffic sessions within a time window, and perform multi-granularity time aggregation to obtain the packet length sequence between sessions.

[0020] Specifically, such as Figure 3 As shown, step S103 includes: Step S1031, Define time aggregation unit The time aggregation unit represents the time granularity of context information aggregation, for example, the unit is seconds (s), milliseconds (ms), or minutes (min). Step S1032: Determine the start time of the target traffic session. ; Step S1033, preset aggregation point data Based on the start time of the target process session and the total number of aggregation points, calculate the time range for context information collection: In the formula, This indicates the start time of the time series used to collect contextual features. Indicates its end time, This represents the total number of aggregated time points contained in the context feature sequence; Step S1034, in Within the time window, according to time aggregation units The characteristic information of neighboring traffic is aggregated to generate the current time granularity. The corresponding context feature vector sequence; Step S1035 involves aggregating the context feature vector sequences corresponding to different time granularities to obtain inter-session packet long sequences. This allows for adaptation to different types of network traffic behavior patterns, enhancing the generalization ability and discriminative performance of the context-sensitive BERT model.

[0021] It should be noted that this invention combines a multi-perspective feature extraction method using session byte sequences and long inter-session packet sequences. This allows for the automatic learning of deep representations of encrypted traffic from massive datasets without the need for manual feature design and extraction, effectively solving the problem of traditional methods relying on human experience and struggling to handle encrypted traffic. Furthermore, this invention overcomes the limitations of single-session granularity analysis, enhancing the detection capability for masquerading threats. For complex threat traffic with highly deceptive characteristics, such as botnets, this invention is no longer limited to single-session level analysis but introduces long inter-session packet sequences for contextual modeling. This enables the model to identify attack traffic mimicking normal business behavior (such as POP3 services), significantly improving its discrimination capability in complex threat scenarios.

[0022] Step S2: Construct a session context-sensitive BERT model. The process of the session context-sensitive BERT model processing session byte sequences and inter-session packet long sequences includes: encoding the session byte sequences and inter-session packet long sequences to obtain a first embedding vector and a second embedding vector; constructing a joint loss function; and pre-training the session context-sensitive BERT model based on the joint loss function and the first and second embedding vectors.

[0023] Specifically, such as Figure 4 As shown, step S2 includes the following steps: Step S201: Construct a session context-sensitive BERT model.

[0024] Furthermore, the session context-sensitive BERT model employs a 12-layer encoding layer based on the Transformer structure and uses context-sensitive multi-head attention.

[0025] It should be noted that the context-sensitive multi-head attention differs from the traditional multi-head attention mechanism. The Q-vector of each layer's input is not combined with the K-vectors and V-vectors of all other tokens. The calculation.

[0026] Specifically, for the first embedding vector corresponding to the session byte sequence, the corresponding query vector (Q) is calculated using the key vector (K) and value vector (V) of all other input tokens (including the session byte sequence and the inter-session packet length sequence) when calculating the attention value, so that the session byte sequence can pay attention to the information of the inter-session context during the learning process; For the second embedding vector corresponding to the inter-session packet-long sequence, when calculating the attention value of its corresponding query vector (Q), only the key vector (K) and value vector (V) of the inter-session packet-long sequence are used for calculation, without paying attention to the session byte sequence information, so that the inter-session packet-long sequence can be free from the influence of irrelevant information such as the intra-session byte sequence during the learning process.

[0027] Step S202: Encode the session byte sequence to obtain the first embedding vector.

[0028] Specifically, the session byte sequence is divided into a first sub-session and a second sub-session; Randomly replace a% of the second sub-sessions; for example, in this instance, randomly replace 50% of the second sub-sessions. Add a [CLS] token at the beginning of the session byte sequence, add a [SEP] token between the first sub-session and the second sub-session, and add several [PAD] tokens after the second sub-session, so that the length of the token sequence is 512; Mask b% of the tokens in the token sequence, where b1% of the masked tokens are replaced with [MASK] tokens, b2% of the masked tokens are replaced with random tokens, and (1- b1- b2)% of the masked tokens retain the original tokens; for example, in this instance, 15% of the tokens are randomly selected for masking, where 80% of the masked tokens are replaced with [MASK] tokens, 10% of the masked tokens are replaced with random tokens, and 10% of the masked tokens retain the original tokens; The masked token sequence is encoded to obtain a 768-dimensional token encoding vector. The token encoding vector, the corresponding 768-dimensional position encoding vector, and the 768-dimensional segment encoding vectors corresponding to the first and second sub-sessions are then superimposed to obtain the first embedding vector. This ensures that the session context-sensitive BERT model can focus on the relative positions of each token and the sub-session to which the token belongs.

[0029] Step S203: Encode the long sequence of packets between sessions to obtain the second embedding vector.

[0030] Further, a mask matrix M is set, the size of which is the same as the size of the inter-session packet length sequence X; the mask matrix M and the inter-session packet length sequence X are multiplied element by element to obtain the second embedding vector; wherein, the inter-session packet length sequence X uses seconds, milliseconds, and minutes in the multi-granularity time aggregation process, and the number of aggregation points is 128, so the inter-session packet length sequence X is a 3×128 matrix; The mask matrix is ​​set according to the following conditions: For each mask column, assuming a mask ratio of r, elements with a ratio of r are set to 0, while the remaining elements are kept to 1. The mask column is generated by alternately setting segments of 0 and 1; the length of each mask segment follows a geometric distribution, and the mask segment is a continuous sequence of 0s with a mean of [value missing]. The length of each unmasked segment follows a geometric distribution, and the unmasked segments are a continuous sequence of 1s with a mean of 1. .

[0031] Furthermore, for each time step, the vector is mapped to a 768-dimensional encoded vector using a fully connected layer, thus obtaining the second embedding vector.

[0032] Step S204: The concatenated first and second embedding vectors are used as training samples; a joint loss function is constructed, and the conversation context-sensitive BERT model is pre-trained based on the joint loss function and the training samples.

[0033] Specifically, set up a first pre-training task, a second pre-training task, and a third pre-training task; The first pre-training task uses the output vector corresponding to the first token [CLS] to perform a binary classification task to determine whether the second sub-session has been replaced; the first loss function corresponding to the first pre-training task is constructed as follows: ; In the formula, For sub-conversation pairs, This represents the first sub-session corresponding to the j-th sample. This represents the second sub-session corresponding to the j-th sample, where n represents the number of samples. Let j be the true label of the j-th sample; These are the trainable parameters for the conversation context-sensitive BERT model; The second pre-training task is to predict the output vector corresponding to the [MASK] token in the session byte sequence. The second loss function corresponding to the second pre-training task is constructed as follows: ; In the formula, The output vector representing the [MASK] token. Let represent the masked token at position i in the original token sequence, and k represent the total number of tokens randomly masked. This represents the real word at the i-th mask position. These are the trainable parameters for the conversation context-sensitive BERT model; The third pre-training task involves regressing and predicting the position of the masked input portion of the long sequence of inter-session packets. The third loss function corresponding to this pre-training task is constructed as follows: ; In the formula, , Represents the weight matrix. This represents the bias vector. Let be the matrix of the output vector corresponding to time step t, and set . Meet the conditions t represents the t-th time step, and i represents the i-th time granularity corresponding to the t-th time step. This represents the elements of the mask matrix that are 0; The weighted sum of the first, second, and third loss functions is used as the joint loss function, and the session context-sensitive BERT model is trained based on the joint loss function.

[0034] The expression for the joint loss function is as follows: In the formula, , , These are the weighting coefficients.

[0035] It should be noted that this invention designs a context-sensitive attention mechanism. For the embedding vector of the session byte sequence portion, its query vector (Q) is calculated using the key vector (K) and value vector (V) of all input tokens (including byte sequences and packet-length sequences) when calculating the attention value. For the embedding vector of the inter-session packet-length sequence portion, its query vector (Q) is calculated using only the key vector (K) and value vector (V) of the inter-session packet-length sequence portion, without considering session byte sequence information. Simultaneously, by combining a multi-task pre-training strategy including the first pre-training task (i.e., same-origin session prediction), the second pre-training task (i.e., byte sequence mask modeling), and the third pre-training task (i.e., packet-length sequence mask modeling), the session context-sensitive BERT model's understanding of complex semantics and behavioral patterns in network traffic is enhanced. This allows it to maintain excellent discriminative performance even under limited sample conditions and exhibits good generalization ability.

[0036] Step S3: Fine-tune the pre-trained session context-sensitive BERT model for the downstream traffic classification task; the fine-tuned session context-sensitive BERT model is used to perform the downstream traffic classification task.

[0037] The process of fine-tuning the pre-trained conversation context-sensitive BERT model includes: Step S301: Obtain the 768-dimensional first output vector corresponding to the first token [CLS]; Step S302: Perform mean pooling on the output vector of the long packet sequence between sessions to obtain a second output vector of 768 dimensions; Step S303: Concatenate the first output vector and the second output vector to obtain a 1536-dimensional third output vector. The third output vector is used for fine-tuning for downstream traffic classification tasks. The downstream traffic classification tasks include at least malware identification tasks, botnet detection tasks, and APP classification tasks.

[0038] According to embodiments of the present invention, the present invention also provides an electronic device and a readable storage medium.

[0039] Figure 5A schematic block diagram of an electronic device that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0040] The electronic device includes a computing unit 101, which can perform various appropriate actions and processes according to a computer program stored in ROM 102 or a computer program loaded into RAM 103 from storage unit 108. RAM 103 may also store various programs and data required for the operation of the electronic device. The computing unit 101, ROM 102, and RAM 103 are interconnected via bus 104. I / O interface 105 is also connected to bus 104.

[0041] Multiple components in the electronic device are connected to the I / O interface 105, including: an input unit 106, such as a keyboard, mouse, etc.; an output unit 107, such as various types of displays, speakers, etc.; a storage unit 108, such as a disk, optical disk, etc.; and a communication unit 109, such as a network card, modem, wireless transceiver, etc. The communication unit 109 allows the electronic device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0042] The computing unit 101 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 101 performs the various methods and processes described above. For example, in some embodiments, the methods in the multidimensional early warning system for pressure injuries can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 108. In some embodiments, part or all of the computer program can be loaded and / or installed on an electronic device via ROM 102 and / or communication unit 109. When the computer program is loaded into RAM 103 and executed by the computing unit 101, one or more steps of the methods in the multidimensional early warning system for pressure injuries described above can be performed. Alternatively, in other embodiments, the computing unit 101 can be configured to perform the methods in the multidimensional early warning system for pressure injuries by any other suitable means (e.g., by means of firmware).

[0043] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0044] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0045] In the context of this invention, a readable storage medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A readable storage medium can be a machine-readable signal medium or a machine-readable storage medium. A readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0046] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including voice input, speech input, or tactile input).

[0047] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0048] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0049] The above embodiments are only used to illustrate the design concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made based on the principles and design ideas disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A context-sensitive BERT traffic classification method, characterized in that, The method includes: Acquire network traffic data; perform session segmentation on the network traffic data to obtain several traffic sessions; tokenize each traffic session to obtain a session byte sequence; extract the packet length sequence of all data packets between multiple traffic sessions within a time window, and perform multi-granularity time aggregation to obtain the inter-session packet length sequence; Constructing a session context-sensitive BERT model; The process of the session context-sensitive BERT model processing session byte sequences and inter-session packet long sequences includes: encoding the session byte sequences and inter-session packet long sequences to obtain a first embedding vector and a second embedding vector; using the concatenated first and second embedding vectors as training samples; constructing a joint loss function; and pre-training the session context-sensitive BERT model based on the joint loss function and the training samples. The pre-trained session context-sensitive BERT model is fine-tuned for downstream traffic classification tasks; the fine-tuned session context-sensitive BERT model is then used to perform downstream traffic classification tasks.

2. The traffic classification method of context-sensitive BERT according to claim 1, characterized in that, The process of segmenting network traffic data into several traffic sessions includes: Network traffic data is segmented into 5-tuples, which are source IP, destination IP, source port, destination port, and protocol. A traffic session is formed by combining network traffic data that are consistent across 5-tuples.

3. The traffic classification method of context-sensitive BERT according to claim 1, characterized in that, The process of tokenizing each traffic session to obtain a session byte sequence includes: The source IP, destination IP, source port, and destination port of each data packet in the traffic session are randomized. Calculate the checksum fields of the network layer and transport layer protocols in the data packet; Extract the first 8 data packets from the traffic session, concatenate the payload portion of the first 8 data packets from the traffic session, and tokenize them in units of 4 hexadecimal characters to obtain the session byte sequence.

4. The traffic classification method of context-sensitive BERT according to claim 1, characterized in that, The process of extracting the packet length sequence of all data packets between multiple traffic sessions within a time window and performing multi-granularity time aggregation to obtain the inter-session packet length sequence includes: Define the time granularity; Determine the start time of the target process session and set the total number of aggregation points; determine the time window based on the start time of the target process session and the total number of aggregation points; Within the time window, traffic sessions are aggregated according to time granularity to obtain the context feature vector sequence corresponding to the current time granularity; By aggregating the context feature vector sequences corresponding to different time granularities, a long sequence of packets between sessions can be obtained.

5. The traffic classification method of context-sensitive BERT according to claim 1, characterized in that, The process of encoding the session byte sequence to obtain the first embedding vector includes: Divide the session byte sequence into a first sub-session and a second sub-session; randomly replace a% of the second sub-session. Add a [CLS] token at the beginning of the session byte sequence, add a [SEP] token between the first sub-session and the second sub-session, and add several [PAD] tokens after the second sub-session, so that the length of the token sequence is 512; Mask the tokens in the token sequence b%, where the masked tokens of b1% are replaced with [MASK] tokens, the masked tokens of b2% are replaced with random tokens, and the masked tokens of (1-b1-b2)% retain the original tokens. The token sequence after adding the mask is encoded to obtain the token encoding vector; the token encoding vector, the position encoding vector and the segment encoding vectors corresponding to the first sub-session and the second sub-session are superimposed to obtain the first embedding vector.

6. The traffic classification method of context-sensitive BERT according to claim 1, characterized in that, The process of encoding the long sequence of inter-session packets to obtain the second embedding vector includes: Set a mask matrix, the size of which is the same as the size of the inter-session packet length sequence; multiply the mask matrix and the inter-session packet length sequence element by element to obtain the second embedding vector; The mask matrix is ​​set according to the following conditions: For each mask column, assuming a mask ratio of r, elements with a ratio of r are set to 0, while the remaining elements are kept to 1. The mask column is generated by alternately setting segments of 0 and 1; the length of each mask segment follows a geometric distribution, and the mask segment is a continuous sequence of 0s with a mean of [value missing]. The length of each unmasked segment follows a geometric distribution, and the unmasked segments are a continuous sequence of 1s with a mean of 1. .

7. The traffic classification method of context-sensitive BERT according to claim 1, characterized in that, The concatenated first and second embedding vectors are used as training samples to construct a joint loss function. The process of pre-training the conversation context-sensitive BERT model based on the joint loss function and the training samples includes: Set up a first pre-training task, a second pre-training task, and a third pre-training task; The first pre-training task uses the output vector corresponding to the first token [CLS] to perform a binary classification task to determine whether the second sub-session has been replaced; the first loss function corresponding to the first pre-training task is constructed as follows: ； In the formula, For sub-conversation pairs, This represents the first sub-session corresponding to the j-th sample. This represents the second sub-session corresponding to the j-th sample, where n represents the number of samples. Let j be the true label of the j-th sample; These are the trainable parameters for the conversation context-sensitive BERT model; The second pre-training task is to predict the output vector corresponding to the [MASK] token in the session byte sequence. The second loss function corresponding to the second pre-training task is constructed as follows: ； In the formula, The output vector representing the [MASK] token. Let represent the masked token at position i in the original token sequence, and k represent the total number of tokens randomly masked. This represents the real word at the i-th mask position. These are the trainable parameters for the conversation context-sensitive BERT model; The third pre-training task involves regressing and predicting the position of the masked input portion of the long sequence of inter-session packets. The third loss function corresponding to this pre-training task is constructed as follows: ； In the formula, , Represents the weight matrix. This represents the bias vector. Let be the matrix of the output vector corresponding to time step t, and set . Meet the conditions t represents the t-th time step, and i represents the i-th time granularity corresponding to the t-th time step. This represents the elements of the mask matrix that are 0; The weighted sum of the first, second, and third loss functions is used as the joint loss function, and the session context-sensitive BERT model is trained based on the joint loss function.

8. A traffic classification method for context-sensitive BERT according to claim 1 or 6, characterized in that, The process of fine-tuning the pre-trained conversation context-sensitive BERT model for downstream traffic classification tasks includes: Obtain the first output vector corresponding to the first token [CLS]; The output vector of the long packet sequence between sessions is averaged and pooled to obtain the second output vector; Step S303: Concatenate the first output vector and the second output vector to obtain the third output vector. Use the third output vector to fine-tune the downstream traffic classification task. The downstream traffic classification task includes at least malware identification task, botnet detection task, and APP classification task.

9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more of the computer programs being executed by the at least one processor to enable the at least one processor to perform the context-sensitive BERT traffic classification method as described in any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the traffic classification method of context-sensitive BERT as described in any one of claims 1-8.

Images