Internet of Things equipment identification method and system based on multilayer bidirectional GRU

Abstract

The invention relates to the technical field of Internet of Things equipment identification, in particular to an Internet of Things equipment identification method and system based on a multilayer bidirectional GRU, and the method comprises the steps: carrying out the preprocessing operation of original network traffic generated by Internet of Things equipment, obtaining a plurality of streams, and extracting a stream byte vector and an artificial definition feature from each stream; a multi-layer bidirectional GRU model is utilized to extract time sequence features and spatial features in the flow, and prediction labels of samples are output; and optimizing the multilayer bidirectional GRU model through supervised cross entropy loss, adversarial loss and unsupervised consistency loss. According to the method, part of labeled data is used for performing model parameter adjustment, flow in a large amount of label-free flow data is used for performing equipment type prediction to generate pseudo labels, and the pseudo labels are used as supervision signals during training of artificially defined features extracted from the same flow to optimize the model. The method effectively reduces the workload of training data marking while guaranteeing the high recognition accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things device identification, and particularly relates to an Internet of Things device identification method and system based on a multi-layer bidirectional GRU. Background Art

[0002] In recent years, the number of IoT (Internet of Things) devices connected to the Internet has been continuously increasing. However, the expansion and prosperity of IoT devices have also brought serious security challenges. Since IoT devices generally lack complete security mechanisms, they are more likely to become targets of malicious attacks, resulting in an increasingly intense attack situation against IoT devices. The discovery and identification of Internet of Things devices are the primary tasks of device attribute research and security analysis. Accurate device identification is almost the basis of all network security management, which helps administrators master the types of Internet of Things devices in the network, detect devices with vulnerabilities in the network, evaluate the risk level of network security, and provide support for security operations such as asset evaluation, vulnerability defense, and situation awareness, and has important significance for improving the security level of the cyberspace.

[0003] In reality, for the need of their own security management, some network devices do not provide access paths to the outside and do not respond to external detection requests. For example, some dark devices in a network or special devices in a business private network. These types of devices connect to a specific system through a specific port and transmit data. In order to detect the attributes of these types of devices, only the network traffic intercepted at the network gateway can be used to extract features at the packet level or protocol flow level in the traffic, and the research on device identification methods based on network traffic has become a current research hotspot.

[0004] Currently, most Internet of Things device identifications adopt supervised learning and unsupervised learning. For supervised learning methods, a large amount of labeled data is required to obtain good identification results. In the real world, it is unrealistic to label all communication data. Since labeling data usually requires manpower and even must be completed by experts, the success of supervised learning algorithms often also means huge costs. Unsupervised learning groups data according to the similarity of internal features of the data, does not require any prior information about the device type, and does not require labeled training data, thus eliminating the cumbersome data labeling work for researchers. However, its disadvantage is that when there are a large number of features in the dataset, it may suffer from the "high-dimensional feature disaster". And unsupervised learning lacks learning samples and cannot reliably identify unknown devices. The "black box" nature of unsupervised learning methods makes it difficult to interpret the decision results.

[0005] Semi-supervised learning usually uses a small set of labeled data and a large amount of unlabeled data to train a model, thereby reducing the requirement for the quantity of labeled data and improving the generalization ability of supervised learning tasks. Since unlabeled data can be obtained with less manpower, semi-supervised learning can bring performance improvement at a relatively low cost. However, there are few studies on device identification based on semi-supervised learning. A large number of existing studies have proved that both flows and manually defined features can be used to characterize the types of devices. Therefore, using unlabeled flows and manually defined features as the input of a classification model, the prediction results should not have significant differences, that is, the output is consistent. Summary of the Invention

[0006] The present invention aims to solve the problem of how to accurately identify devices based on a small number of labels and a large amount of unlabeled data and have interpretability. It proposes an Internet of Things device identification method and system based on a multi-layer bidirectional GRU. It uses part of the labeled data for model tuning, and uses the flows in a large amount of unlabeled traffic data to predict the device type to generate pseudo-labels, which are used as supervision signals during the training of manually defined features extracted from the same traffic, and optimizes the model; thereby improving the performance of the model trained on a small number of labeled samples and alleviating the difficulty of collecting and labeling large data sets.

[0007] To achieve the above object, the technical solutions adopted are as follows:

[0008] The present invention provides an Internet of Things device identification method based on a multi-layer bidirectional GRU, including the following steps:

[0009] Perform preprocessing operations on the original network traffic generated by Internet of Things devices to obtain multiple flows, and then extract flow byte vectors and manually defined features for each flow;

[0010] Use a multi-layer bidirectional GRU model to extract temporal features and spatial features in the traffic and output prediction labels for the samples;

[0011] Optimize the multi-layer bidirectional GRU model through supervised cross-entropy loss, adversarial loss, and unsupervised consistency loss.

[0012] According to the Internet of Things device identification method based on a multi-layer bidirectional GRU of the present invention, further, the process of the preprocessing operation is: group the original traffic data by MAC address, and then split the grouped traffic into multiple flows according to five-tuple information, and each flow is saved as an independent PCAP file.

[0013] According to the method for identifying Internet of Things devices based on multi-layer bidirectional GRU of the present invention, further, extracting the flow byte vector specifically includes: intercepting the first 784 bytes of the flow and dividing them into 28-dimensional vectors with 28 time steps for input to the multi-layer bidirectional GRU model; if the number of bytes in the flow is less than 784 bytes, padding it with 0x00 to 784 bytes, erasing the content of the IP header of each data packet in the flow and filling it with 0.

[0014] According to the method for identifying Internet of Things devices based on multi-layer bidirectional GRU of the present invention, further, extracting manually defined features specifically includes: using the Scrapy and Dpkt packages in Python to select features from the flow to obtain a 17-dimensional vector in csv format, and the 17-dimensional vector includes 4 numerical features such as source port number, destination port number, frame length, and time interval between two adjacent data packets, and 13 boolean protocol features.

[0015] According to the method for identifying Internet of Things devices based on multi-layer bidirectional GRU of the present invention, further, the 13 boolean protocol features of the manually defined features include: network layer protocols - ICMP, ICMPv6, transport layer protocols - TCP, UDP, application layer protocols - HTTP, HTTPS, DHCP, SSDP, DNS, MDNS, NTP, and packet options - IP options, TCP options.

[0016] According to the method for identifying Internet of Things devices based on multi-layer bidirectional GRU of the present invention, further, the multi-layer bidirectional GRU model includes an input layer, three bidirectional GRU layers, four fully connected layers, and an output layer. The input layer receives the flow byte vector, and the bidirectional GRU layers extract features by processing the time series data from the forward and reverse directions respectively. The output flow feature vector is transmitted to the fully connected layer for feature fusion, and finally the device type is output through the output layer.

[0017] According to the method for identifying Internet of Things devices based on multi-layer bidirectional GRU of the present invention, further, the calculation method of the adversarial loss is:

[0018] Using the fast gradient sign method to apply perturbations to the labeled data to generate an adversarial sample x adv :

[0019]

[0020] where sign is the sign function, x is the flow input sample, y is the true label corresponding to x, ε is the constraint parameter, is the first derivative of the loss function with respect to the flow input sample x, and x adv is used as the adversarial sample input to the standard cross-entropy loss to obtain the adversarial loss l adv .

[0021] According to the Internet of Things device recognition method based on a multi-layer bidirectional GRU of the present invention, further, the calculation method of the unsupervised consistency loss is as follows:

[0022] Input the flow byte vector of the unlabeled data into the multi-layer bidirectional GRU model to obtain pseudo-labels Input the manually defined features of the same unlabeled data into an independent fully connected layer and then obtain the predicted label q; calculate The cross-entropy loss between and q as the unsupervised consistency loss l u 。

[0023] According to the Internet of Things device recognition method based on a multi-layer bidirectional GRU of the present invention, further, the comprehensive loss expression of the multi-layer bidirectional GRU model is:

[0024] l = l s + λ1l adv + λ2l u

[0025] Wherein, l s is the supervised cross-entropy loss, l adv is the adversarial loss, l u is the unsupervised consistency loss, λ1 is the relative weight of the adversarial loss, and λ2 is the relative weight of the unsupervised consistency loss.

[0026] Further, the present invention also provides an Internet of Things device recognition system based on a multi-layer bidirectional GRU for implementing the above-mentioned Internet of Things device recognition method based on a multi-layer bidirectional GRU, including:

[0027] A preprocessing module for preprocessing the original network traffic generated by the Internet of Things device to obtain multiple flows, and then extracting the flow byte vector and the manually defined features for each flow;

[0028] A model construction module for using the multi-layer bidirectional GRU model to extract the temporal features and spatial features in the traffic and output the predicted label of the sample;

[0029] A loss function module for optimizing the multi-layer bidirectional GRU model through the supervised cross-entropy loss, the adversarial loss, and the unsupervised consistency loss.

[0030] Adopting the above technical solutions, the beneficial effects obtained are:

[0031] The present invention uses the flow representation of a small amount of traffic data as labeled data, the flow representation and manually defined representation of a large amount of traffic data as unlabeled data, and a multi-layer bidirectional GRU as the training model. Through a semi-supervised learning framework, it only requires a small amount of labeled data to achieve an identification accuracy rate of more than 90% of traditional supervised learning. While ensuring a high identification accuracy rate of Internet of Things devices, it effectively reduces the workload of training data labeling, and is particularly suitable for scenarios where it is difficult to obtain labels, such as private network devices.

[0032] The present invention uses an unsupervised consistency loss to constrain the prediction consistency of different representations (flow features and artificial features) of the same unlabeled data, and can effectively utilize the unlabeled data.

[0033] The present invention introduces FGSM adversarial training to improve the model's resistance to traffic camouflage attacks, and can effectively handle real-world noises such as packet loss, realizing high-robustness device identification. Brief Description of the Drawings

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings of the embodiments of the present invention will be briefly introduced below. Among them, the drawings are only used to show some embodiments of the present invention, rather than limiting all embodiments of the present invention thereto.

[0035] Figure 1 It is a framework diagram of the Internet of Things device identification method based on a multi-layer bidirectional GRU according to an embodiment of the present invention;

[0036] Figure 2 It is a schematic structural diagram of the multi-layer bidirectional GRU model according to an embodiment of the present invention. Detailed Embodiments

[0037] Below, the exemplary solutions of the embodiments of the present invention will be clearly and completely described in conjunction with the drawings of the specific embodiments of the present invention. Unless otherwise defined, the technical terms or scientific terms used in the present invention should have the ordinary meaning understood by those of ordinary skill in the art.

[0038] In many practical application scenarios, obtaining labeled data is often costly and time-consuming, while unlabeled data is relatively easy to obtain. This solution is based on semi-supervised learning and can effectively use a large amount of unlabeled data to improve the learning effect of the model. At the same time, it relies less on a large amount of labeled data. Supervised learning requires a large amount of labeled data to train an accurate model, and obtaining these labeled data usually requires the participation of domain experts, which is time-consuming and laborious. Semi-supervised learning can still train a better model with only a small amount of labeled data, thereby reducing the need for a large amount of labeled data. By utilizing the structural information in unlabeled data (such as data distribution), it can help the model better understand the overall structure of the data, thereby improving its generalization ability, that is, it is more robust when facing new data. Unsupervised learning is mainly used to explore the intrinsic structure and pattern of data, while semi-supervised learning can, on this basis, guide the model to focus on those typical features for identifying IoT devices by introducing a small amount of labeled data.

[0039] Inspired by semi-supervised learning, this embodiment discloses an IoT device identification method based on a multi-layer bidirectional GRU. Figure 1 As shown, the following steps are included:

[0040] Step S1: Convert the traffic data into a form that can be processed by the neural network model. First, the original network traffic generated by the IoT device is preprocessed to obtain multiple streams, and then the stream byte vector and manually defined features are extracted from each stream for input into the device classification model. This step includes sub-steps S101-S104.

[0041] Step S101, group the original traffic data by MAC address, and then further split the grouped traffic into streams according to the five-tuple information (source IP address, destination IP address, source port, destination port and transport layer protocol), and save each stream in the form of a pcap file.

[0042] Step S102: intercept the first 784 bytes of the stream (if the number of bytes of the stream is less than 784 bytes, fill it with 0x00 to 784 bytes), and erase the content of the IP header of each data packet in the stream and fill it with 0.

[0043] Step S103: Divide each obtained stream into 28 time steps, each time step is a 28-byte vector, and used as input of the deep learning network.

[0044] Step S104: To describe the device network activities, the source port number, destination port number, network layer protocols (ICMP, ICMPv6), transport layer protocols (TCP, UDP), and application layer protocols (HTTP, HTTPS, DHCP, SSDP, DNS, MDNS, NTP, etc.) in the flow are extracted as features. At the same time, the frame length, the time interval between two adjacent packets, whether there are IP options, whether there are TCP options, etc. are selected as the traffic characteristics of the IoT devices. Among them, the source port number, destination port number, frame length, and the time interval between two adjacent packets are four features that take actual numerical values, and the remaining 13 features take boolean values. If the feature exists in the flow, the value is 1, otherwise the value is 0. The Scrapy and Dpkt packages of Python are used to select features from the flow, and finally a set of 17-dimensional vectors in csv format is obtained.

[0045] Step S2: Use a multi-layer bidirectional GRU model to extract the temporal features and spatial features in the traffic, and output the prediction labels for the samples.

[0046] This solution designs a multi-layer bidirectional GRU (Gated Recurrent Unit) model, as Figure 2 shown. The temporal features of the traffic are characterized by GRU, and the spatial features such as traffic bytes, data packets, and flows are characterized by three layers of bidirectional GRU layers. The multi-layer bidirectional GRU model consists of four parts, namely the input layer, three layers of bidirectional GRU layers, four layers of fully connected layers, and the output layer.

[0047] ① Input layer

[0048] To construct the input of the multi-layer bidirectional GRU model, the input vector corresponding to each piece of data is divided into 28 time steps, and the dimension of the input vector for each time step is 28, and they are input serially according to each time step. After preprocessing, each piece of data is converted into a vector and input into the GRU network.

[0049] ② Multi-layer bidirectional GRU layer

[0050] The advantage of GRU is that it can capture long-term dependencies with fewer parameters and the model has a relatively fast training speed. The unidirectional GRU model only looks at the historical information of the traffic when learning context information, while the bidirectional GRU utilizes both the historical and future information of the traffic. The flow byte vector is input into the model from the beginning to the end and at the same time from the end to the beginning. The output of each GRU cell layer can be stacked into a matrix as the input for the next layer. A set of outputs will be obtained for each direction, and these two sets of outputs are combined together in the last layer of the model to form the output vector of the bidirectional GRU model. Finally, the outputs of the three-layer bidirectional GRU are input into the fully connected layer together.

[0051] The time step set in this model is 28. Assume that the previous time step is t - 1 and the current time step is t. The input at the current time step is x(t), and the output at the previous time step is y (t-1) , then the output y (t) at the current time step is calculated as shown in formula (1). The forget gate and the input gate are combined to obtain the update gate, and then the output at the current time step is obtained based on the update gate and the output at the previous time step. The y (t) at all time steps are concatenated together to form the output vector. Since it is a bidirectional GRU neural network, assume that the output vector in the forward direction is and the output vector in the reverse direction is then the final output vector is the concatenation of the two, as shown in formula (2).

[0052]

[0053] ③Fully connected layer

[0054] Four hidden fully connected layers are set between the Bi-GRU layer and the Output layer. The number of neurons in the last fully connected layer is equal to the total number of device categories, and the original features can be converted into low-dimensional features.

[0055] ④Output layer

[0056] The output layer represents the probability distribution of the possibility that the input sample belongs to a certain device category. For the Internet of Things device classification task, the category with the highest probability is selected as the final device type output.

[0057] Step S3: Optimize the multi-layer bidirectional GRU model using supervised cross-entropy loss, adversarial loss, and unsupervised consistency loss. The cross-entropy loss obtained by inputting a small amount of labeled data into the model is used to adjust the weights of the neurons in the model, and interference is imposed on this part of the labeled data to suppress model overfitting. Unlabeled data uses consistency regularization and pseudo-label semi-supervised learning algorithms to obtain unsupervised loss.

[0058] This solution optimizes the multi-layer bidirectional GRU model through three loss functions, namely supervised loss, adversarial loss, and unsupervised loss. Since the unsupervised loss is not reliable before the neural network is fully trained, therefore, first use labeled samples for supervised learning to obtain the standard loss, and use the adversarial loss to optimize the model parameters.

[0059] Let X = {(x b , y b )} be a batch of the B batches of labeled samples, where b ∈ (1, …, B), x b is the training sample, and y b is the true label. Let U = {u b} be a batch of the μB batches of unlabeled samples, where b ∈ (1, …, μB), and μ is the ratio of the quantity of X to U. Convert network traffic into two characteristics: flow and artificially defined features, denoted by M(·) and N(·) respectively.

[0060] (1) Supervised cross-entropy loss

[0061] For the labeled data, use a typical supervised learning algorithm, input the labeled flow samples into the multi-layer bidirectional GRU model, and the model outputs the probability of possible device categories. Without loss of generality, the standard cross-entropy loss function ls is shown in formula (3), where α(y b |M(x b )) is the probability of obtaining the label y b by inputting the sample x b into the GRU model, and α is the multi-layer bidirectional GRU model.

[0062]

[0063] (2) Adversarial loss

[0064] When using a small number of samples (such as 10%), the model may be overfitted. By applying interference, the overfitting of the model is suppressed. Since the gradient direction of the perturbation is more likely to cause the model to misclassify than other directions, interference is applied to the gradient direction. In this solution, the Fast Gradient Sign Method (FSGM) is used. An adversarial loss is introduced. By adding the calculated gradient direction to the input sample, the loss value of the modified input becomes larger when passing through the network, thereby increasing the difference between the same classes. The FGSM algorithm is used to calculate the gradient of the loss function with respect to the input sample, perform a sign function operation on the gradient, and multiply it by the constraint parameter. The adversarial sample is obtained as shown in formula (4).

[0065]

[0066] Where the sign takes a value of 1 or -1 (the direction of the adversarial interference), x is the streaming input sample, y is the true label corresponding to x. ε is the constraint parameter, is the first derivative (i.e., the gradient) of the loss function with respect to the input sample x. x adv As the adversarial sample is input into the standard cross-entropy loss (such as formula 3), the adversarial loss l adv is obtained.

[0067] (3) Unsupervised consistency loss

[0068] For unlabeled data, two features, streaming and artificial features, are extracted. The streaming samples are input into a multi-layer bidirectional GRU model for prediction. The probability distribution p of the predicted classes of the model is calculated, and then is used as the pseudo-label (inferred label) of the prediction result; then the artificial feature samples are input into an independent fully connected layer to obtain the predicted label q; the cross-entropy loss is calculated using the pseudo-label of the streaming samples and the predicted label q of the artificial feature samples, and the cross-entropy loss l u is obtained, as shown in formula (5), where, is the probability that the artificial feature sample is input into the fully connected layer to obtain the label .

[0069]

[0070] (4) Comprehensive loss

[0071] The comprehensive loss of the entire training network is shown in formula (6), where λ1 is the relative weight of the adversarial loss, and λ2 represents the relative weight of the unsupervised consistency loss.

[0072] l = l s + λ1l adv + λ2l u (6)

[0073] Correspondingly to the above method, this embodiment also proposes an Internet of Things device recognition system based on a multi-layer bidirectional GRU, including:

[0074] A preprocessing module, which is used to perform preprocessing operations on the original network traffic generated by the Internet of Things device to obtain multiple flows, and then extract flow byte vectors and manually defined features for each flow.

[0075] A model construction module, which is used to extract temporal features and spatial features in the traffic by using a multi-layer bidirectional GRU model and output the predicted labels for the samples.

[0076] A loss function module, which is used to optimize the multi-layer bidirectional GRU model through supervised cross-entropy loss, adversarial loss, and unsupervised consistency loss.

[0077] Finally, it should be noted that the above embodiments are only specific implementation manners of the present invention, which are used to illustrate the technical solutions of the present invention, rather than limiting them. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present invention can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the protection scope of the claims.

Claims

1. An Internet of Things device recognition method based on a multi-layer bidirectional GRU, characterized in that, The following steps are involved: The original network traffic generated by IoT devices is preprocessed to obtain multiple flows, and then the flow byte vector and manually defined features are extracted for each flow; Use a multi-layer bidirectional GRU model to extract temporal and spatial features from traffic and output predicted labels for samples; The multi-layer bidirectional GRU model is optimized by supervised cross entropy loss, adversarial loss and unsupervised consistency loss.

2. The method for identifying Internet of Things devices based on a multi-layer bidirectional GRU according to claim 1, wherein The process of the preprocessing operation is: grouping the original traffic data by MAC address, and then splitting the grouped traffic into multiple flows according to the five-tuple information, and saving each flow as an independent PCAP file.

3. The method for identifying Internet of Things devices based on multi-layer bidirectional GRU according to claim 2, characterized in that, Extracting the stream byte vector specifically includes: intercepting the first 784 bytes of the stream and dividing it into a 28-dimensional vector of 28 time steps for the input of the multi-layer bidirectional GRU model; if the number of bytes of the stream is less than 784 bytes, fill it with 0x00 to 784 bytes, erase the content of the IP header of each data packet in the stream and fill it with 0.

4. The method for identifying Internet of Things devices based on a multi-layer bidirectional GRU according to claim 2, wherein, Extracting manually defined features specifically includes: using Python's Scrapy and Dpkt packages to select features from the stream and obtain a 17-dimensional vector in csv format. The 17-dimensional vector contains four numerical features: source port number, destination port number, frame length, and the time interval between two adjacent data packets, and 13 Boolean protocol features.

5. The method for identifying Internet of Things devices based on a multi-layer bidirectional GRU according to claim 4, characterized in that The 13 Boolean protocol features of manually defined features include: network layer protocols - ICMP, ICMPv6, transport layer protocols - TCP, UDP, application layer protocols - HTTP, HTTPS, DHCP, SSDP, DNS, MDNS, NTP, and data packet options - IP options, TCP options.

6. The method for identifying Internet of Things devices based on a multi-layer bidirectional GRU according to claim 1, characterized in that, The multi-layer bidirectional GRU model includes an input layer, three bidirectional GRU layers, four fully connected layers and an output layer. The input layer receives the stream byte vector. The bidirectional GRU layer extracts features from the forward and reverse processing time series data respectively. The output stream feature vector is passed to the fully connected layer for feature fusion, and finally the device type is output through the output layer.

7. The method for identifying Internet of Things devices based on a multi-layer bidirectional GRU according to claim 1, wherein The adversarial loss is calculated as: Apply perturbations to the labeled data using the Fast Gradient Sign Method to generate adversarial example x adv : where sign is the sign function, x is the streaming input sample, y is the true label corresponding to x, ε is the constraint parameter, is the first-order derivative of the loss function with respect to the streaming input sample x, and x adv is used as the adversarial sample input into the standard cross-entropy loss to obtain the adversarial loss l adv .

8. The method for identifying Internet of Things devices based on a multi-layer bidirectional GRU according to claim 7, wherein The unsupervised consistency loss is calculated as: Input the stream byte vector of unlabeled data into the multi-layer bidirectional GRU model to obtain pseudo-labels Input the manually defined features of the same unlabeled data into an independent fully connected layer to obtain the predicted label q; calculate The cross-entropy loss between and q as the unsupervised consistency loss l u 。 9. The method for identifying Internet of Things devices based on a multi-layer bidirectional GRU according to claim 8, wherein The comprehensive loss expression of the multi-layer bidirectional GRU model is: l = l s + λ1l adv + λ2l u Among them, l s is the supervised cross-entropy loss, l adv is the adversarial loss, l u is the unsupervised consistency loss, λ1 is the relative weight of the adversarial loss, and λ2 is the relative weight of the unsupervised consistency loss.

10. An Internet of Things device recognition system based on a multi-layer bidirectional GRU, characterized in that, The method for identifying an IoT device based on a multi-layer bidirectional GRU as claimed in any one of claims 1 to 9 comprises: The preprocessing module is used to preprocess the original network traffic generated by IoT devices to obtain multiple flows, and then extract the flow byte vector and manually defined features for each flow; The model building module is used to extract the temporal and spatial features in the traffic using a multi-layer bidirectional GRU model and output the predicted label for the sample; Loss function module, used to optimize the multi-layer bidirectional GRU model through supervised cross entropy loss, adversarial loss and unsupervised consistency loss.

Images