A security prediction method and device for encrypted traffic

Abstract

The application provides a security prediction method and device for encrypted traffic, applied to the field of Internet of Things and the field of finance, and after obtaining encrypted traffic, the encrypted traffic is pretreated to obtain pretreated encrypted traffic, the security prediction cost of the encrypted traffic is reduced, the pretreated encrypted traffic is input into an encrypted traffic security prediction model, so that the encrypted traffic security prediction model outputs a security prediction result of the encrypted traffic, the encrypted traffic security prediction model is obtained based on bert language representation model training, the bert language representation model is learned by transfer learning, manual feature screening is not needed, the accuracy of security prediction is improved, and the effect of end-to-end cooperation of encrypted traffic of an agency for security prediction is achieved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of network security, and particularly relates to a security prediction method and device for encrypted traffic. BACKGROUND

[0002] While the Internet technology is widely applied, there are also a large number of security risks, and various network information security incidents occur frequently, which makes the computer network face a severe information security situation. For example, in the credit field of commercial banks, cooperative institutions access commercial banks through encrypted traffic, and it is necessary to make security prediction on the cooperative institutions.

[0003] In the prior art, traditional machine learning algorithms such as support vector machines are used to classify encrypted traffic, but such methods need to extract a large number of features manually, and under the ciphertext message of encrypted traffic, the extraction cost is very high, the accuracy of security prediction is low, and the exposed interface of the cooperative institutions to the commercial bank is easy to become a security access vulnerability of the credit system. SUMMARY

[0004] In view of the above problems, the application provides a security prediction method and device for encrypted traffic, in order to improve the security prediction accuracy of encrypted traffic, the specific scheme is as follows:

[0005] A security prediction method for encrypted traffic, comprising:

[0006] obtaining encrypted traffic;

[0007] preprocessing the encrypted traffic to obtain preprocessed encrypted traffic;

[0008] inputting the preprocessed encrypted traffic into an encrypted traffic security prediction model, so that the encrypted traffic security prediction model outputs a security prediction result of the encrypted traffic, and the encrypted traffic security prediction model is obtained by training based on a bert language representation model.

[0009] Optionally, the training process of the encrypted traffic security prediction model comprises:

[0010] determining a first training encrypted traffic set and a second training encrypted traffic set, the first training encrypted traffic set being unlabeled, and the second training encrypted traffic set being labeled, and the label being used to indicate whether the training encrypted traffic is safe;

[0011] pre-training a bert language representation model by using the first training encrypted traffic set to obtain a pre-trained bert language representation model;

[0012] fine-tuning the pre-trained bert language representation model by using the second training encrypted traffic set to obtain the security prediction model for the encrypted traffic.

[0013] Optionally, the determination manner of the second training encrypted traffic set comprises:

[0014] obtaining an original training encrypted traffic set, each original training encrypted traffic in the original training encrypted traffic set having a label;

[0015] determining a second training encrypted traffic set from the original training encrypted traffic set based on the label of each original training encrypted traffic using an Easy Ensemble algorithm.

[0016] Optionally, the preprocessing of the encrypted traffic to obtain preprocessed encrypted traffic comprises:

[0017] performing data stream segmentation, key data extraction, time sequence information combination, dimension transformation and byte transformation on the encrypted traffic to obtain preprocessed encrypted traffic.

[0018] Optionally, the data stream segmentation comprises:

[0019] segmenting the encrypted traffic to obtain a plurality of data streams;

[0020] determining a plurality of data packets based on the plurality of data streams.

[0021] Optionally, the key data extraction comprises:

[0022] determining a plurality of key data packets based on the plurality of data packets.

[0023] Optionally, the dimension transformation and byte transformation comprise:

[0024] for each key data packet, extracting the first byte, and if the key data packet has no byte, performing zero padding to obtain a dimension-transformed key data packet;

[0025] for each byte in the processed key data packet, representing the byte using a vector to obtain a byte-transformed key data packet.

[0026] Optionally, the time sequence information combination comprises:

[0027] determining time sequence information of the encrypted traffic based on the traffic data.

[0028] Optionally, the pre-training of the BERT language representation model using the first training encrypted traffic set to obtain a pre-trained BERT language representation model comprises:

[0029] The first training encrypted traffic set is used for density estimation of the BERT language representation model, and a pre-trained BERT language representation model is obtained to determine the underlying features of the first training encrypted traffic set.

[0030] A security prediction device for encrypted traffic comprises:

[0031] An acquisition unit is configured to acquire encrypted traffic.

[0032] A preprocessing unit is configured to preprocess the encrypted traffic to obtain preprocessed encrypted traffic.

[0033] An input unit is configured to input the preprocessed encrypted traffic into an encrypted traffic security prediction model, so that the encrypted traffic security prediction model outputs a security prediction result of the encrypted traffic.

[0034] According to the above technical solution, the security prediction method and device for encrypted traffic are provided, and after the encrypted traffic is acquired, the encrypted traffic is preprocessed to obtain preprocessed encrypted traffic, thereby reducing the security prediction cost of the encrypted traffic. BRIEF DESCRIPTION OF DRAWINGS

[0035] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are intended to depict only preferred embodiments of the application, and therefore should not be considered to narrow the scope of the present application. Rather, the entire disclosure including the full description and the drawings are to be considered to define the scope of the application. Moreover, the same reference numerals in different drawings represent the same or similar components.

[0036] Figure 1 A flowchart of a security prediction method for encrypted traffic disclosed by an embodiment of the application is shown in FIG. 1.

[0037] Figure 2 A structural diagram of a BERT language representation model disclosed by an embodiment of the application is shown in FIG. 2.

[0038] Figure 3 A flowchart of a method for a training process of an encrypted traffic security prediction model disclosed by an embodiment of the application is shown in FIG. 3.

[0039] Figure 4A flowchart of a method for determining a second training encrypted traffic set according to an embodiment of the present application is disclosed.

[0040] Figure 5 A flowchart of a method for data stream segmentation according to an embodiment of the present application is disclosed.

[0041] Figure 6 A flowchart of a method for dimension conversion and byte conversion according to an embodiment of the present application is disclosed.

[0042] Figure 7 A structural diagram of a security prediction device for encrypted traffic according to an embodiment of the present application is disclosed. DETAILED DESCRIPTION

[0043] It should be noted that the application test method and device provided by the present application can be applied to the fields of artificial intelligence, block chain, distribution, cloud computing, big data, Internet of Things, mobile Internet, network security, chips, virtual reality, augmented reality, holographic technology, quantum computing, quantum communication, quantum measurement, digital twin or finance. The above are only examples and do not limit the application field of the application test method and device provided by the present application.

[0044] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0045] The terms "first", "second", and the like in the specification and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, which is only a distinguishing way used in describing the embodiments of the present application for the same attribute objects in the description. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, so that the processes, methods, systems, products or equipment containing a series of units do not have to be limited to those units, but can include other units not clearly listed or inherent to these processes, methods, products or equipment.

[0046] In view of the above problems, the present application provides a security prediction method and device for encrypted traffic, in order to improve the security prediction accuracy of encrypted traffic, the specific scheme is as follows:

[0047] In order to improve the security prediction accuracy of encrypted traffic, the application provides a security prediction method for encrypted traffic, which will be described in detail below in combination with the drawings and specific embodiments.

[0048] Please refer to the accompanying Figure 1 , Figure 1 A flowchart of a security prediction method for encrypted traffic provided by an embodiment of the application. The method can include the following steps:

[0049] Step S101: Obtain encrypted traffic.

[0050] In this application, the encrypted traffic of various cooperative institutions is sampled and collected from a personal credit management system.

[0051] Step S102: Preprocess the encrypted traffic to obtain preprocessed encrypted traffic.

[0052] In this application, the encrypted traffic data is processed and the corresponding related data is transformed to obtain preprocessed encrypted traffic. The specific implementation will be described in detail in the following embodiments, and will not be described here.

[0053] Step S103: Input the preprocessed encrypted traffic into an encrypted traffic security prediction model to make the encrypted traffic security prediction model output a security prediction result of the encrypted traffic. The encrypted traffic security prediction model is trained based on a bert language representation model.

[0054] In this application, the performance of the encrypted traffic security prediction model can be evaluated by macro-averaged accuracy, macro-averaged recall, and macro-averaged F1 score to determine the security prediction result of the encrypted traffic. The specific implementation will be described in detail in the following embodiments, and will not be described here.

[0055] Wherein, Figure 2 A structural diagram of a bert language representation model, the input of the bert pre-training model is based on the Transformer input form, and both need to be linearly added to the position vector and byte vector to form the input vector; the core component Trm of Bert is the Encoder unit of Transformer, and the feature of the Bert model is that it is a bidirectional model, so the model can not only learn the time sequence information from front to back, but also learn the time sequence information from back to front, and strive to learn more time sequence features. The structure of the bert language representation model in the application is 8-layer Transformer, and each Encoder unit contains 8 multi-head self-attention. Although the number of layers of the model is actually very large, since the Transformer unit contains a residual model, the number of layers will not cause problems such as gradient disappearance.

[0056] In summary, the encrypted traffic security prediction method provided by the application, after obtaining the encrypted traffic, the encrypted traffic is pretreated to obtain the pretreated encrypted traffic, which reduces the security prediction cost of the encrypted traffic, the pretreated encrypted traffic is input into the encrypted traffic security prediction model, so that the encrypted traffic security prediction model outputs the security prediction result of the encrypted traffic, the encrypted traffic security prediction model is obtained based on the training of the bert language representation model, the bert language representation model is learned by transfer learning, manual feature screening is not required, the accuracy of security prediction is improved, and the effect of end-to-end cooperation of encrypted traffic security prediction is achieved.

[0057] On the basis of the above-mentioned embodiments disclosed by the application, in still another embodiment of the application, the specific implementation mode of the training process of the encrypted traffic security prediction model is described in detail.

[0058] As an implementable mode, please refer to the accompanying Figure 3 The flowchart of the method for training the encrypted traffic security prediction model is shown in the figure. The method can include the following steps:

[0059] Step S201: determine the first training encrypted traffic set and the second training encrypted traffic set, the first training encrypted traffic set is unlabeled, and the second training encrypted traffic set is labeled, and the label is used to indicate whether the training encrypted traffic is safe.

[0060] In this application, according to the feedback of the encrypted traffic of the cooperation agency accessing the personal credit management system, part of the encrypted traffic is labeled, that is, the second training encrypted traffic set, and the encrypted traffic with access risk and normal encrypted traffic are distinguished; another part is unlabeled, that is, the first training encrypted traffic set.

[0061] Step S202: pre-training the bert language representation model using the first training encrypted traffic set to obtain the pre-trained bert language representation model.

[0062] It should be noted that the first training encrypted traffic set is used to pre-train the bert language representation model, and the specific implementation mode will be described in detail in the following embodiments, which will not be described here.

[0063] Step S203: fine-tune the pre-trained bert language representation model using the second training encrypted traffic set to obtain the encrypted traffic security prediction model.

[0064] It should be noted that the fine-tuning of the pre-trained BERT language representation model is essentially an application of transfer learning, which fine-tunes the pre-trained BERT language representation model on a large-scale corpus to adapt to downstream tasks.

[0065] Among them, transfer learning refers to migrating the pre-trained BERT language representation model parameters that have been trained to a new model to help the new model training, considering that most data or tasks are related, the pre-trained BERT language representation model parameters that have been learned can be shared with the new model in some way to speed up and optimize the learning efficiency of the model without learning from zero like most networks.

[0066] In this application, the subsequent fine-tuning is followed by a second round of training, and the data used is the labeled second training encrypted traffic set, which is performed on the pre-trained BERT language representation model in the previous stage, and a weak classifier is learned from the second training encrypted traffic set, 101 base weak classifiers are trained, and the soft classifier is integrated learning by voting to obtain the majority prediction vote as the final category.

[0067] In summary, the training process of the encrypted traffic security prediction model provided by the embodiment of the application overcomes the defect of too few encrypted traffic label data through pre-training of the unlabeled first training encrypted traffic set and fine-tuning of the labeled second training encrypted traffic set.

[0068] On the basis of the above-mentioned embodiments disclosed in the application, in still another embodiment of the application, the specific implementation of the determination method of the second training encrypted traffic set is described in detail.

[0069] As an implementable manner, please refer to the accompanying Figure 4 The flowchart of the method for determining the second training encrypted traffic set disclosed in the application is shown. The method can include the following steps:

[0070] Step S301: Obtain the original training encrypted traffic set, and each original training encrypted traffic in the original training encrypted traffic set is labeled.

[0071] In this application, the labeled number of original training encrypted traffic sets is screened and data exploration.

[0072] Step S302: Based on the label of each original training encrypted traffic, the Easy Ensemble algorithm is used to determine the second training encrypted traffic set from the original training encrypted traffic set.

[0073] In the present application, the EasyEnsemble algorithm is used to count the data with access risk and normal data in different fields, the original training encrypted traffic with the least category in the original training encrypted traffic set is taken as the minimum benchmark, and then the other original training encrypted traffic categories are quantified based on the benchmark, about the same encrypted traffic in the majority category is extracted each time, and the integrity of the original training encrypted traffic is not damaged, then the combination of the original training encrypted traffic is taken as the training set, that is, the second training encrypted traffic set.

[0074] In summary, the method for determining the second training encrypted traffic set provided by the embodiment of the present application performs data exploration and data balancing operation on the labeled second training encrypted traffic set, which is beneficial to subsequent fine-tuning.

[0075] On the basis of the above-mentioned disclosed embodiments of the present application, in another embodiment of the present application, the specific implementation of the pre-processing of the encrypted traffic to obtain the pre-processed encrypted traffic is described in detail.

[0076] As an implementable manner, the encrypted traffic can be subjected to data stream segmentation, key data extraction, time sequence information combination, dimension transformation and byte transformation to obtain the pre-processed encrypted traffic.

[0077] In summary, the method for pre-processing the encrypted traffic to obtain the pre-processed encrypted traffic provided by the embodiment of the present application deletes some redundant information of the encrypted traffic and conforms to the required format of the input data in the model.

[0078] On the basis of the above-mentioned disclosed embodiments of the present application, in another embodiment of the present application, the specific implementation of the data stream segmentation is described in detail.

[0079] As an implementable manner, please refer to the accompanying Figure 5 , the flowchart of the method for data stream segmentation disclosed by the present application. The method can include the following steps:

[0080] Step S401: The encrypted traffic is segmented to obtain a plurality of data streams.

[0081] In the present application, the form of the encrypted traffic mainly exists in the form of a pcap package, and a pcap package contains a plurality of data streams.

[0082] Step S402: Based on the plurality of data streams, a plurality of data packets are determined.

[0083] In the present application, the source IP, destination IP, source port, destination port, and transport layer protocol five-tuple are used to uniquely identify a data stream, that is, the five-tuple is the same, and the data packet belongs to the same data stream, and the original pcap packet needs to be split using the SplitCap tool in the form of the five-tuple, and the data packets of each data stream are saved as pcap packets in the original time sequence.

[0084] On the basis of the above-mentioned embodiments disclosed by the present application, in another embodiment of the present application, the specific implementation manner of key data extraction is described in detail.

[0085] As an implementable manner, a plurality of key data packets can be determined based on a plurality of data packets.

[0086] In the present application, the front part of the plurality of data packets is related to handshake information, and the rear information is related to the characteristics of the encrypted data, the first n data packets in the plurality of data packets are extracted, and other data packets exceeding this part are discarded.

[0087] On the basis of the above-mentioned embodiments disclosed by the present application, in another embodiment of the present application, the specific implementation manner of dimension conversion and byte conversion is described in detail.

[0088] As an implementable manner, please refer to the accompanying Figure 6 , a flowchart of a dimension conversion and byte conversion method disclosed by the present application. The method can include the following steps:

[0089] Step S501: For each key data packet, the first byte is extracted, and if the key data packet has no byte, zero processing is performed to obtain the key data packet after dimension conversion.

[0090] In the present application, when processing each key data packet, the encrypted traffic has similarities with natural language processing, the data stream corresponds to a sentence of text, the data packet corresponds to a word vector, and the type of byte is similar to the bag of words in text data. Thus, in order to uniformly process each data packet, the length is fixed, that is, the data in the first byte of each data packet is extracted, and the other parts are discarded, and zero processing is performed. Under this data processing, each data stream will be converted into an n*l two-dimensional matrix.

[0091] Step S502: For each byte in the processed key data packet, a vector is used for representation to obtain the key data packet after byte conversion.

[0092] In the present application, the one-hot encoding format is used, the possible value of each byte is between 0-255, and there are 256 values in total, so each byte in the processed key data packet can be encoded as a 256-dimensional vector for representation.

[0093] On the basis of the above disclosed embodiments of the application, in another embodiment of the application, the specific implementation of the time sequence information combination is described in detail.

[0094] As an implementable manner, the time sequence information of the encrypted traffic can be determined based on the flow data.

[0095] In the present application, the Self-Attention mechanism which can capture long-distance dependencies is used to overcome the local dependency characteristics of convolutional neural networks and recurrent neural networks, so as to establish long-distance information interaction between input sequences and determine the time sequence information of encrypted traffic.

[0096] In summary, the method of time sequence information combination provided by the embodiments of the application pays attention to the time sequence characteristics of encrypted traffic when using the bert language representation model, and compared with the local characteristics of encrypted traffic in the prior art, the application is beneficial to the prediction accuracy of the encrypted traffic security prediction model to reach the best.

[0097] On the basis of the above disclosed embodiments of the application, in another embodiment of the application, the specific implementation of the time sequence information combination is described in detail.

[0098] As an implementable manner, the time sequence information of the encrypted traffic can be determined based on the flow data.

[0099] In the present application, the Self-Attention mechanism which can capture long-distance dependencies is used to overcome the local dependency characteristics of convolutional neural networks and recurrent neural networks, so as to establish long-distance information interaction between input sequences and determine the time sequence information of encrypted traffic.

[0100] In summary, the method of time sequence information combination provided by the embodiments of the application pays attention to the time sequence characteristics of encrypted traffic when using the bert language representation model, and compared with the local characteristics of encrypted traffic in the prior art, the application is beneficial to the prediction accuracy of the encrypted traffic security prediction model to reach the best.

[0101] The above disclosed embodiments of the application describe the method in detail, and the method of the application can be implemented in various forms of devices, therefore the application also discloses an encrypted traffic security prediction device, and the following specific embodiments are given for detailed description.

[0102] Please refer to the accompanying drawings Figure 7 , Figure 7 A structure diagram of a security prediction device for encrypted traffic disclosed in an embodiment of the present application, the device comprising:

[0103] An acquisition unit 11 is configured to acquire encrypted traffic.

[0104] A preprocessing unit 12 is configured to preprocess the encrypted traffic to obtain preprocessed encrypted traffic.

[0105] An input unit 13 is configured to input the preprocessed encrypted traffic into an encrypted traffic security prediction model, so that the encrypted traffic security prediction model outputs a security prediction result of the encrypted traffic, and the encrypted traffic security prediction model is obtained based on a bert language representation model.

[0106] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0107] In addition, it should be noted that the device embodiments described above are only schematic, and the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or distributed on multiple network units. According to actual needs, part or all of the modules can be selected to achieve the purpose of the embodiment scheme. In addition, the connection relationship between the modules in the device embodiment provided by the present application indicates that there is a communication connection between them, which can be realized as one or more communication buses or signal lines. Those skilled in the art can understand and implement it without creative labor.

[0108] ​Through the above description of the embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software necessary general hardware, of course, can also be implemented by special hardware including special integrated circuit, special CPU, special memory, special components, etc. Generally, the functions completed by the computer program can be easily realized by the corresponding hardware, and the specific hardware structure for realizing the same function can also be various, such as analog circuit, digital circuit or special circuit, etc. However, for the present application, the software program implementation is a better embodiment. Based on this understanding, the technical solutions of the present application can be embodied in the form of software products, and the computer software product is stored in a readable storage medium, such as a computer floppy disk, U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc., including a number of instructions to make a computer device (which can be a personal computer, server, or network device, etc.) execute the method described in each embodiment of the present application.

[0109] In summary, the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit it. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the above embodiments, or make equivalent replacement for part of the technical features. The modification or replacement does not make the corresponding technical solution deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A secure prediction method for encrypted traffic, characterized in that, include: Obtain encrypted traffic; The encrypted traffic is preprocessed to obtain preprocessed encrypted traffic; The preprocessed encrypted traffic is input into the encrypted traffic security prediction model so that the encrypted traffic security prediction model outputs the security prediction result of the encrypted traffic. The encrypted traffic security prediction model is trained based on the BERT language representation model. The BERT language representation model has an 8-layer Transformer structure, and the Encoder unit of each Transformer layer includes 8 self-attention heads. The training process of the encrypted traffic security prediction model includes: A first training encrypted traffic set and a second training encrypted traffic set are determined. The first training encrypted traffic set is unlabeled, and the second training encrypted traffic set is labeled. The labels are used to indicate whether the training encrypted traffic is secure. The BERT language representation model is pre-trained using the first training encrypted traffic set to obtain the pre-trained BERT language representation model. The pre-trained BERT language representation model is fine-tuned using the second training encrypted traffic set to obtain the secure prediction model for the encrypted traffic; The step of pre-training the BERT language representation model using the first training encrypted traffic set to obtain the pre-trained BERT language representation model includes: Density estimation of the BERT language representation model is performed using the first training encrypted traffic set to obtain the pre-trained BERT language representation model, thereby determining the underlying features of the first training encrypted traffic set.

2. The method according to claim 1, characterized in that, The method for determining the second set of encrypted traffic for training includes: Obtain the original training encrypted traffic set, wherein each original training encrypted traffic in the original training encrypted traffic set is labeled; Based on the labels of the original training encrypted traffic sets, the Easy Ensemble algorithm is used to determine the second training encrypted traffic set from the original training encrypted traffic set.

3. The method according to claim 1, characterized in that, The preprocessing of the encrypted traffic to obtain preprocessed encrypted traffic includes: The encrypted traffic is subjected to data stream segmentation, key data extraction, time sequence information combination, dimension transformation, and byte transformation to obtain preprocessed encrypted traffic.

4. The method according to claim 3, characterized in that, The data stream segmentation includes: The encrypted traffic is segmented to obtain multiple data streams; Based on the multiple data streams, multiple data packets are determined.

5. The method according to claim 4, characterized in that, The key data extraction includes: Based on the aforementioned data packets, several key data packets were identified.

6. The method according to claim 5, characterized in that, The dimension conversion and byte conversion include: For each key data packet, the first byte is extracted. If the key data packet has no bytes, it is padded with zeros to obtain the key data packet after dimension transformation. For each byte in the processed key data packet, a vector is used to represent it, resulting in the key data packet after byte conversion.

7. The method according to claim 5, characterized in that, The combination of time-series information includes: Based on the traffic data, the timing information of the encrypted traffic is determined.

8. A secure prediction device for encrypted traffic, characterized in that, include: The acquisition unit is used to acquire encrypted traffic; A preprocessing unit is used to preprocess the encrypted traffic to obtain preprocessed encrypted traffic; The input unit is used to input the preprocessed encrypted traffic into the encrypted traffic security prediction model so that the encrypted traffic security prediction model outputs the security prediction result of the encrypted traffic. The encrypted traffic security prediction model is trained based on the BERT language representation model. The BERT language representation model has an 8-layer Transformer structure, and the Encoder unit of each Transformer layer includes 8 self-attention heads. The device further includes: A determining unit is used to determine a first training encrypted traffic set and a second training encrypted traffic set, wherein the first training encrypted traffic set is unlabeled and the second training encrypted traffic set is labeled, and the label is used to indicate whether the training encrypted traffic is secure; The pre-training unit is used to pre-train the BERT language representation model using the first training encrypted traffic set to obtain the pre-trained BERT language representation model. The fine-tuning unit is used to fine-tune the pre-trained BERT language representation model using the second training encrypted traffic set to obtain the secure prediction model of the encrypted traffic. The pre-training unit includes: The pre-training subunit is used to perform density estimation on the BERT language representation model using the first training encrypted traffic set to obtain the pre-trained BERT language representation model, so as to determine the underlying features of the first training encrypted traffic set.

Images