VPN handshake abnormity identification method based on self-supervised learning

Abstract

The invention discloses a VPN handshake anomaly identification method based on self-supervised learning. The method comprises the following steps: constructing a handshake feature vector sequence from unlabeled handshake traffic; generating a protocol semantic graph and a semantic template correlation matrix based on the handshake feature vector sequence; constructing a multi-view handshake sample set according to the session, the certificate cluster and the server fingerprint; establishing a handshake coding network and normal and abnormal branch projection head self-supervision structures, and dividing training samples; in a normal branch, using improved Barlow Twins loss to learn normal subspace representation; sparse correlation, semantic decoupling and cross-branch orthogonal constraint are applied to the abnormal branches to obtain abnormal subspace representation; and updating the semantic template correlation matrix in online operation, generating an exception score for a handshake to be detected, and outputting an exception type. According to the method, the semantic template and the double-subspace model are established under the non-labeling condition, and the method is used for identifying degradation, playback and certificate replacement anomalies.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and in particular to a method for identifying VPN handshake anomalies based on self-supervised learning. Background Technology

[0002] Existing VPN communication systems typically rely on a handshake phase to complete protocol version negotiation, cipher suite negotiation, certificate chain verification, and key derivation. The handshake behavior largely determines the security of the VPN connection. However, traditional handshake anomaly detection methods are mostly based on feature engineering and static rules. For example, they rely on fixed thresholds to detect protocol version changes, on packet sequence template comparison to detect replay behavior, or on certificate field matching to discover certificate replacement risks. These methods are poorly adaptable to encrypted traffic, cannot model structural changes in the handshake sequence, and can only rely on limited observable fields for inference. They often make misjudgments when faced with different implementation versions, different network environments, and dynamic encryption strategies. Traditional methods struggle to parse the implicit patterns in VPN handshake behavior and cannot effectively represent the high-dimensional temporal features in real network environments, resulting in limited identification capabilities.

[0003] As VPN protocols are widely used in public network access, remote work, and encrypted tunnel construction, attackers are increasingly employing various penetration methods targeting the handshake mechanism. These include cipher suite downgrade attacks, handshake message replay attacks, and certificate chain replacement attacks. Existing research attempts to classify handshake steps using machine learning to improve anomaly detection capabilities. However, most methods still rely on manually labeled training data, while reliable labels are often difficult to obtain for handshake traffic in real networks. Furthermore, handshake sequences contain various heterogeneous features such as protocol version, cipher suite, certificate chain, message interval, timestamp, random number, and message length. Existing models typically concatenate these features directly into a classification network, failing to explicitly model the semantic dependencies between these features. This results in the model's inability to capture key structures in protocol behavior patterns. The lack of semantic relationship modeling leads to insufficient generalization ability when facing protocol extensions or encryption algorithm updates, making it difficult to identify unknown attack methods.

[0004] In the context of deep learning development, some methods have attempted to introduce self-supervised learning to reduce reliance on manual annotation. However, most existing self-supervised frameworks are geared towards computer vision and natural language processing, and their loss function structures do not match the internal semantic relationships of high-dimensional encrypted handshake behavior, making direct transfer to VPN handshake analysis difficult. Existing methods typically only establish similarity constraints between different handshake segments through temporal comparison learning, failing to utilize the inherent semantic structure of the handshake protocol to restrict the representation space and unable to distinguish between normal semantic dependencies, weak dependencies, and prohibited dependencies. Furthermore, anomalous behavior often differs slightly from normal behavior in some field dimensions, and traditional self-supervised methods cannot construct independent anomalous feature subspaces, resulting in insufficient ability to distinguish between degradation anomalies, replay anomalies, and certificate replacement anomalies. Lacking protocol semantic graph modeling, semantic template correlation matrix constraints, and mechanisms for separating anomalous and normal subspaces, existing technologies struggle to provide refined representations of VPN handshake behavior and accurately identify unknown attack behaviors.

[0005] Therefore, how to provide a VPN handshake anomaly identification method based on self-supervised learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a VPN handshake anomaly identification method based on self-supervised learning. This invention utilizes handshake feature vector sequences to construct a protocol semantic graph and a semantic template correlation matrix, and combines a handshake coding network, normal subspace representation, and abnormal subspace representation. Through an improved Barlow Twins loss and anomaly branch constraints, it achieves modeling of downgrade, replay, and certificate replacement risks. It has the advantages of being label-free, highly adaptable, and able to identify early signs of anomalies.

[0007] A VPN handshake anomaly identification method based on self-supervised learning according to an embodiment of the present invention includes the following steps: Obtain unlabeled VPN handshake traffic, group it by session, parse the handshake fields, and construct a handshake feature vector sequence; A protocol semantic graph is constructed based on the handshake feature vector sequence, generating a protocol semantic adjacency matrix and converting it into a semantic template relevance matrix; Multi-view handshake samples are constructed based on session, certificate cluster, and server fingerprint to form a multi-view handshake sample set. A self-supervised network including a handshake encoding network, a normal branch projection head, and an abnormal branch projection head is constructed. The handshake risk indicator is calculated and the multi-view handshake samples are divided into normal training samples and high-risk training samples. An improved Barlow Twins loss is used in the normal branch for normal training samples to align the projection correlation matrix with the semantic template correlation matrix and perform redundancy suppression to obtain the normal subspace representation. Sparse correlation constraints, semantic decoupling constraints, and cross-branch orthogonality constraints are applied to high-risk training samples in the abnormal branches to obtain an abnormal subspace representation. During online operation, the semantic template correlation matrix is ​​updated based on low-risk handshake samples. The VPN handshake under test is processed by the handshake coding network and normal branch projection head and abnormal branch projection head to obtain normal subspace representation and abnormal subspace representation. The template deviation degree and abnormal activation degree are calculated and combined to obtain the VPN handshake anomaly score. The anomaly identification result and anomaly type are output.

[0008] Optionally, the construction of the handshake feature vector sequence specifically includes: Unlabeled VPN handshake traffic is acquired and grouped by session. The grouping by session involves generating a session identifier based on the source IP address, destination IP address, source port, destination port, and transport layer protocol, and classifying network packets with the same session identifier into the same session. Within each session, handshake phase messages are filtered and arranged in chronological order. The handshake phase messages include client handshake messages and server handshake messages, and the chronological order is determined based on the message timestamps. The handshake phase message is parsed, and the handshake field parsing includes extracting the handshake protocol version, the list of supported cipher suites, the list of supported compression methods, the client random number, the client session identifier, and the client extended field information from the client handshake message, and extracting the target cipher suite, the target compression method, the server random number, the server session identifier, and the server extended field information from the negotiation result from the server handshake message. The certificate chain field is parsed for the certificate-related messages in the handshake phase. The certificate chain field parsing includes extracting the certificate issuer name, certificate user name, certificate validity period range, certificate serial number and certificate fingerprint information, and establishing a one-to-one association between each level of certificate in the certificate chain and the corresponding session identifier. Based on the timestamp and message length of the handshake phase messages, the round-trip delay, message interval, and message length sequence of the handshake phase are calculated. The handshake protocol version field, cipher suite related fields, certificate chain field, random number field, timestamp field, and round-trip delay and message length related fields are numerically encoded and normalized. The encoding results are arranged according to the time order of the handshake phase messages in their respective sessions to generate the handshake feature vector sequence of the corresponding session.

[0009] Optionally, the generation of the semantic template relevance matrix specifically includes: A semantic node set is established based on the handshake protocol version field, cipher suite field, certificate chain field, random number segment, timestamp field, round-trip delay feature and message length sequence feature in the handshake feature vector sequence. A semantic node number is assigned to each handshake semantic element in the semantic node set. Based on the VPN handshake specification and security policy, the dependency relationship types between semantic nodes are marked in the semantic node set. The dependency relationship types include strong dependency type, weak dependency type and prohibited dependency type. A protocol semantic graph is constructed based on the semantic nodes and dependency relationships. A protocol semantic adjacency matrix is ​​generated based on the protocol semantic graph. The protocol semantic adjacency matrix uses the semantic node number as the row index and column index. The dependency relationship encoding between semantic nodes is recorded at the intersection of each row and each column. Different dependency relationship types correspond to different encoding values. A semantic template relevance matrix is ​​generated based on the protocol semantic adjacency matrix, and target relevance levels are set for different dependencies. In the semantic template relevance matrix, a first target relevance level is set for positions corresponding to strong dependencies, a second target relevance level is set for positions corresponding to weak dependencies, a third target relevance level is set for positions corresponding to prohibited dependencies, and a unified target relevance level is set for positions with the same row index and column index. The processed semantic template relevance matrix is ​​used as the semantic template relevance matrix in the training phase.

[0010] Optionally, the construction of the multi-view handshake sample specifically includes: Based on the session identifier, the handshake feature vector sequence is divided into session sets. The handshake feature vectors of each session set are arranged in chronological order to form a session view. The session view represents the handshake feature vector sequence within a single session. A certificate cluster is constructed based on the certificate chain field in the handshake feature vector sequence. Sessions with the same certificate chain field content are clustered. The session view sets in each cluster are merged to form a certificate cluster view corresponding to the certificate cluster. A server fingerprint is generated based on server network address information, certificate fingerprint information and cipher suite field according to preset rules. The server fingerprint is calculated from server network address information, certificate fingerprint information and cipher suite field and is used to identify the server's behavioral characteristics in the VPN handshake. Sessions with the same server fingerprint are clustered, and the session views in each cluster are merged to form a server fingerprint view corresponding to the server fingerprint. Multi-view handshake samples are constructed based on session view, certificate cluster view and server fingerprint view. For each session view, the corresponding session view and the certificate cluster view containing the same session are selected to form a first view combination. For each session view, the corresponding session view and the server fingerprint view containing the same session are selected to form a second view combination. The first view combination and the second view combination are recorded as multi-view handshake samples. All multi-view handshake samples are summarized to form a multi-view handshake sample set.

[0011] Optionally, the division between normal training samples and high-risk training samples specifically includes: A self-supervised network is built based on a multi-view handshake sample set. The self-supervised network includes a handshake encoding network, a normal branch projection head, and an abnormal branch projection head. The handshake encoding network is used to receive the handshake feature vector sequence in the multi-view handshake sample and output the handshake embedding vector. The normal branch projection head and the abnormal branch projection head are used to receive the handshake embedding vector and output the normal branch projection vector and the abnormal branch projection vector, respectively. In the handshake coding network, an encoding structure is set up to process the handshake feature vector sequence in time order. The encoding structure performs time-series encoding on the handshake protocol version field, cipher suite field, certificate chain field, random number segment, timestamp field, round-trip delay feature and message length sequence feature to generate a handshake embedding vector that corresponds one-to-one with the multi-view handshake sample. Based on the handshake feature vector sequence and handshake embedding vector, we calculate the downgrade risk correlation quantity, replay risk correlation quantity, and certificate replacement risk correlation quantity. The downgrade risk correlation quantity includes the number of times the security level of the cipher suite is reduced, the number of times the weak cipher suite appears, and the number of times the handshake protocol version is rolled back. The replay risk correlation quantity includes the deviation measure of the time interval sequence and message length sequence relative to the historical pattern. The certificate replacement risk correlation quantity includes the number of certificate chain field changes, the number of certificate fingerprint changes, and the number of abnormal certificate validity period markers. Calculate the downgrade risk indicator based on the downgrade risk related quantity, calculate the replay risk indicator based on the replay risk related quantity, calculate the certificate replacement risk indicator based on the certificate replacement risk related quantity, multiply the count field, deviation measurement field, and anomaly marker field in each risk related quantity by the preset weight coefficient and sum them to obtain the downgrade risk indicator, replay risk indicator, and certificate replacement risk indicator. Each risk indicator is a scalar. The handshake risk indicator is calculated based on the downgrade risk indicator, replay risk indicator, and certificate replacement risk indicator. The handshake risk indicator is obtained by multiplying the downgrade risk indicator, replay risk indicator, and certificate replacement risk indicator by their corresponding weight coefficients and summing them. The handshake risk indicator is a single scalar used to characterize the risk level of multi-view handshake samples. The handshake risk indicator is compared with a first risk threshold and a second risk threshold. Multi-view handshake samples with a handshake risk indicator not higher than the first risk threshold are classified as normal training samples, and multi-view handshake samples with a handshake risk indicator higher than the second risk threshold are classified as high-risk training samples.

[0012] Optionally, the generation of the normal subspace representation specifically includes: Select multi-view handshake samples that have been divided into normal training samples from the multi-view handshake sample set. Input the handshake feature vector sequence in the normal training samples into the handshake coding network and the normal branch projection head to obtain the normal branch projection vector. The normal branch projection vector is the vector representation obtained by projecting the embedded vector output by the handshake coding network through the normal branch projection head. In each training batch, a projection correlation matrix is ​​constructed based on the normal branch projection vector. The projection correlation matrix is ​​indexed by the projection dimension, and the matrix elements represent the correlation level between different projection dimensions. During the construction process, the sample mean within the batch is calculated for each projection dimension. For any two projection dimensions, the average of the sample-by-sample product within the batch is calculated and the product of the means of the two projection dimensions is subtracted. The result is written into the projection correlation matrix as the correlation level at the corresponding position. An improved Barlow Twins loss is defined based on the projection correlation matrix, semantic template correlation matrix, and protocol semantic adjacency matrix. The improved Barlow Twins loss includes semantic alignment constraints and redundancy suppression constraints. The semantic alignment constraints set different target correlation levels in the semantic template correlation matrix according to the strong dependency type, weak dependency type, and prohibited dependency type in the protocol semantic adjacency matrix, and apply weighted constraints on the correlation level deviation at the corresponding positions of the projection correlation matrix and the semantic template correlation matrix. The redundancy suppression constraints suppress the correlation level in the projection correlation matrix at the positions marked as prohibited dependency types in the protocol semantic adjacency matrix. The semantic alignment constraints are obtained by summing the weighted squares of the differences between the corresponding positions of the projection correlation matrix and the semantic template correlation matrix, and the redundancy suppression constraints are obtained by summing the squares of the differences between the correlation level at the prohibited dependency type position and the preset redundancy target level. The semantic alignment constraint and the redundancy suppression constraint are combined according to a preset coefficient to form an improved Barlow Twins loss. During the training process of normal training samples, the improved Barlow Twins loss is used as the objective function to update the parameters of the handshake coding network and the normal branch projection head. During the parameter update process, the gradient is calculated based on the improved Barlow Twins loss and the network parameters are iteratively updated. After completing the preset training rounds, the parameters of the handshake coding network and the normal branch projection head are fixed, and the output space obtained by passing the normal training samples through the handshake coding network and the normal branch projection head is determined as the normal subspace representation.

[0013] Optionally, the generation of the abnormal subspace representation specifically includes: Select multi-view handshake samples that are classified as high-risk training samples from the multi-view handshake sample set, and input the handshake feature vector sequence in the high-risk training samples into the handshake coding network and the abnormal branch projection head to obtain the abnormal branch projection vector. In each training batch, an anomaly branch projection correlation matrix is ​​constructed based on the anomaly branch projection vector. The anomaly branch projection correlation matrix is ​​indexed by the projection dimension, and each matrix element represents the correlation level between the corresponding two projection dimensions. The anomaly branch projection correlation matrix is ​​obtained by statistically analyzing the joint changes of the anomaly branch projection vector in each projection dimension within the training batch. Each matrix element is calculated from the joint statistics of the corresponding two projection dimensions on the batch samples. Based on the anomaly branch projection correlation matrix, a sparse correlation constraint is defined. The position where the row index and column index of the anomaly branch projection correlation matrix are different is regarded as the off-diagonal position. The sparse correlation constraint term is constructed by the correlation level at the off-diagonal position. The sparse correlation constraint term is obtained by weighted aggregation of the correlation levels of all off-diagonal positions and is used to measure the overall correlation strength of the anomaly branch projection correlation matrix at the off-diagonal position. Based on the anomaly branch projection correlation matrix and semantic node field type definition, semantic decoupling constraints are defined. Semantic nodes are divided into semantic groups according to the handshake protocol version field, cipher suite field, certificate chain field, random number segment, timestamp field, delay feature field, and message length feature field. Semantic node pairs belonging to different semantic groups are selected, and semantic decoupling constraint terms are constructed based on the correlation level at the corresponding matrix position. The semantic decoupling constraint terms are obtained by aggregating the correlation level at the position across semantic groups, and are used to measure the correlation strength of the anomaly branch projection correlation matrix at the position across semantic groups. Based on the normal branch projection vector and the abnormal branch projection vector, cross-branch orthogonality constraints are defined. For each high-risk training sample, the cross-branch correlation metric between the normal branch projection vector and the abnormal branch projection vector is calculated. The cross-branch orthogonality constraint term is constructed by aggregating the cross-branch correlation metrics of all high-risk training samples. The cross-branch orthogonality constraint term is obtained by aggregating the cross-branch correlation metrics and is used to measure the degree of close orthogonality between the normal branch projection vector and the abnormal branch projection vector. The sparse correlation constraint, semantic decoupling constraint, and cross-branch orthogonality constraint are weighted and summed according to the first loss weight, the second loss weight, and the third loss weight to obtain the abnormal branch loss function. The abnormal branch loss function is used to simultaneously constrain the sparsity of the abnormal branch projection correlation matrix at off-diagonal positions, the correlation strength at cross-semantic grouping positions, and the orthogonality between the normal branch projection vector and the abnormal branch projection vector in the same target. During the training process of high-risk training samples, gradient calculation and parameter updates are performed according to the abnormal branch loss function. After completing the preset training rounds, the output space of the handshake coding network and the abnormal branch projection head on the high-risk training samples is determined as the abnormal subspace representation.

[0014] Optionally, the output of the anomaly identification result and anomaly type specifically includes: During online operation, VPN handshake traffic is received, handshake risk indicator is calculated, and a low-risk handshake sample set is determined based on the comparison between the handshake risk indicator and a preset update threshold. An update batch is constructed based on low-risk handshake samples. The handshake feature vector sequence in the update batch is input into the handshake coding network and the normal branch projection head to obtain the normal subspace representation of the update batch. The update batch projection correlation matrix is ​​constructed, and the update batch projection correlation matrix is ​​weighted and combined with the current semantic template correlation matrix according to the preset update coefficient to obtain a new semantic template correlation matrix. During the detection phase, the handshake feature vector sequence corresponding to the handshake of the VPN under test is received. The handshake feature vector sequence is input into the handshake coding network and the normal branch projection head to obtain the normal subspace representation. The handshake feature vector sequence is input into the handshake coding network and the abnormal branch projection head to obtain the abnormal subspace representation. The normal subspace representation and the abnormal subspace representation are used for template deviation degree and abnormal activation degree calculation, respectively. Template deviation is calculated based on the normal subspace representation and the semantic template correlation matrix. The template deviation is obtained by statistically analyzing the differences in correlation levels at corresponding positions of the detection projection correlation matrix obtained from the normal subspace representation and the semantic template correlation matrix, and then aggregating them. The abnormal activation degree is calculated based on the abnormal subspace representation. The abnormal activation degree is obtained by aggregating the responses of the abnormal subspace representation on the preset abnormal sensitivity dimension. The VPN handshake anomaly score is calculated based on the template deviation and abnormal activation. The template deviation and abnormal activation are combined into a single scalar VPN handshake anomaly score according to a preset weight. The VPN handshake anomaly score is then compared with a set of scoring thresholds to distinguish different anomaly types, resulting in anomaly identification results and anomaly type output content.

[0015] The beneficial effects of this invention are: This invention constructs a protocol semantic graph of handshake feature vector sequences and a semantic template correlation matrix, enabling explicit modeling of semantic dependencies between handshake protocol versions, cipher suites, certificate chains, temporal behavior, and length features. This achieves a unified expression of the structural patterns of handshake behavior. Compared to traditional methods that rely on rules or single-dimensional features, this invention can automatically learn the semantic structure of a normal handshake under unlabeled conditions and maintain stable generalization ability in unknown scenarios.

[0016] This invention constructs a normal subspace representation by introducing an improved Barlow Twins loss and an abnormal subspace representation by constructing an abnormal subspace representation through sparse correlation constraints, semantic decoupling constraints, and cross-branch orthogonality constraints, thereby effectively separating normal behavior from potential abnormal behavior in the representation space. Based on the normal and abnormal subspace representations, this invention can generate two types of indices: template deviation and abnormal activation, providing clear distribution differences when facing different types of abnormal signals such as degradation, replay, and certificate replacement.

[0017] This invention also utilizes low-risk handshake samples from the online phase to dynamically update the semantic template relevance matrix, enabling the model to continuously adapt to changes in the real network environment and avoiding the performance degradation issues of traditional methods when encryption strategies are updated, server configurations change, or handshake implementations differ. By constructing a VPN handshake anomaly score, this invention can more finely measure anomalies, alerting potential risks at an early stage and achieving highly sensitive identification and continuous monitoring of the VPN handshake security status. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0019] Figure 1 This is a flowchart of a VPN handshake anomaly identification method based on self-supervised learning proposed in this invention; Figure 2 This is a schematic diagram illustrating the semantic template construction of a VPN handshake anomaly identification method based on self-supervised learning proposed in this invention. Figure 3This is a schematic diagram of the normal and abnormal subspace representation structures of a VPN handshake anomaly identification method based on self-supervised learning proposed in this invention. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0021] refer to Figure 1-3 A VPN handshake anomaly detection method based on self-supervised learning includes the following steps: Obtain unlabeled VPN handshake traffic, group it by session, parse the handshake fields, and construct a handshake feature vector sequence; A protocol semantic graph is constructed based on the handshake feature vector sequence, generating a protocol semantic adjacency matrix and converting it into a semantic template relevance matrix; Multi-view handshake samples are constructed based on session, certificate cluster, and server fingerprint to form a multi-view handshake sample set. A self-supervised network including a handshake encoding network, a normal branch projection head, and an abnormal branch projection head is constructed. The handshake risk indicator is calculated and the multi-view handshake samples are divided into normal training samples and high-risk training samples. An improved Barlow Twins loss is used in the normal branch for normal training samples to align the projection correlation matrix with the semantic template correlation matrix and perform redundancy suppression to obtain the normal subspace representation. Sparse correlation constraints, semantic decoupling constraints, and cross-branch orthogonality constraints are applied to high-risk training samples in the abnormal branches to obtain an abnormal subspace representation. During online operation, the semantic template correlation matrix is ​​updated based on low-risk handshake samples. The VPN handshake under test is processed by the handshake coding network and normal branch projection head and abnormal branch projection head to obtain normal subspace representation and abnormal subspace representation. The template deviation degree and abnormal activation degree are calculated and combined to obtain the VPN handshake anomaly score. The anomaly identification result and anomaly type are output.

[0022] In this embodiment, the construction of the handshake feature vector sequence specifically includes: Unlabeled VPN handshake traffic is acquired and grouped by session. The grouping by session involves generating a session identifier based on the source IP address, destination IP address, source port, destination port, and transport layer protocol, and classifying network packets with the same session identifier into the same session. Within each session, handshake phase messages are filtered and arranged in chronological order. The handshake phase messages include client handshake messages and server handshake messages, and the chronological order is determined based on the message timestamps. The handshake phase message is parsed, and the handshake field parsing includes extracting the handshake protocol version, the list of supported cipher suites, the list of supported compression methods, the client random number, the client session identifier, and the client extended field information from the client handshake message, and extracting the target cipher suite, the target compression method, the server random number, the server session identifier, and the server extended field information from the negotiation result from the server handshake message. The certificate chain field is parsed for the certificate-related messages in the handshake phase. The certificate chain field parsing includes extracting the certificate issuer name, certificate user name, certificate validity period range, certificate serial number and certificate fingerprint information, and establishing a one-to-one association between each level of certificate in the certificate chain and the corresponding session identifier. Based on the timestamp and message length of the handshake phase messages, the round-trip delay, message interval, and message length sequence of the handshake phase are calculated. The handshake protocol version field, cipher suite related fields, certificate chain field, random number field, timestamp field, and round-trip delay and message length related fields are numerically encoded and normalized. The encoding results are arranged according to the time order of the handshake phase messages in their respective sessions to generate the handshake feature vector sequence of the corresponding session.

[0023] In this embodiment, the generation of the semantic template relevance matrix specifically includes: A semantic node set is established based on the handshake protocol version field, cipher suite field, certificate chain field, random number segment, timestamp field, round-trip delay feature and message length sequence feature in the handshake feature vector sequence. A semantic node number is assigned to each handshake semantic element in the semantic node set. Based on the VPN handshake specification and security policy, the dependency relationship types between semantic nodes are marked in the semantic node set. The dependency relationship types include strong dependency type, weak dependency type and prohibited dependency type. A protocol semantic graph is constructed based on the semantic nodes and dependency relationships. A protocol semantic adjacency matrix is ​​generated based on the protocol semantic graph. The protocol semantic adjacency matrix uses the semantic node number as the row index and column index. The dependency relationship encoding between semantic nodes is recorded at the intersection of each row and each column. Different dependency relationship types correspond to different encoding values. A semantic template relevance matrix is ​​generated based on the protocol semantic adjacency matrix, and target relevance levels are set for different dependencies. In the semantic template relevance matrix, a first target relevance level is set for positions corresponding to strong dependencies, a second target relevance level is set for positions corresponding to weak dependencies, a third target relevance level is set for positions corresponding to prohibited dependencies, and a unified target relevance level is set for positions with the same row index and column index. The processed semantic template relevance matrix is ​​used as the semantic template relevance matrix in the training phase.

[0024] This invention constructs a semantic node set from elements such as handshake protocol version, cipher suite, certificate chain, random number, timestamp, round-trip delay, and message length. Based on the protocol semantic graph and protocol semantic adjacency matrix, a semantic template correlation matrix is ​​generated, mapping strong dependencies, weak dependencies, and prohibited dependencies to differentiated target correlation levels. This allows the structural semantic relationships between handshake features to be explicitly solidified in a unified template form during the training phase. Thus, in the self-supervised learning process, a clear alignment reference and constraint benchmark are provided for the improved Barlow Twins loss and subsequent subspace representation, improving the model's modeling accuracy of protocol behavior patterns and its sensitivity to abnormal deviations.

[0025] In this embodiment, the construction of the multi-view handshake sample specifically includes: Based on the session identifier, the handshake feature vector sequence is divided into session sets. The handshake feature vectors of each session set are arranged in chronological order to form a session view. The session view represents the handshake feature vector sequence within a single session. A certificate cluster is constructed based on the certificate chain field in the handshake feature vector sequence. Sessions with the same certificate chain field content are clustered. The session view sets in each cluster are merged to form a certificate cluster view corresponding to the certificate cluster. A server fingerprint is generated based on server network address information, certificate fingerprint information and cipher suite field according to preset rules. The server fingerprint is calculated from server network address information, certificate fingerprint information and cipher suite field and is used to identify the server's behavioral characteristics in the VPN handshake. Sessions with the same server fingerprint are clustered, and the session views in each cluster are merged to form a server fingerprint view corresponding to the server fingerprint. Multi-view handshake samples are constructed based on session view, certificate cluster view and server fingerprint view. For each session view, the corresponding session view and the certificate cluster view containing the same session are selected to form a first view combination. For each session view, the corresponding session view and the server fingerprint view containing the same session are selected to form a second view combination. The first view combination and the second view combination are recorded as multi-view handshake samples. All multi-view handshake samples are summarized to form a multi-view handshake sample set.

[0026] This invention constructs a session view based on the handshake feature vector sequence, generates a certificate cluster view by combining the certificate chain field, and generates a server fingerprint view based on server network address information, certificate fingerprint information, and cipher suite fields. This allows each handshake session to have a complete representation from the perspectives of a single sequence, certificate structure, and server behavior. Furthermore, by combining the session view, certificate cluster view, and server fingerprint view to form a multi-view handshake sample set, the model can simultaneously learn the temporal features, certificate association features, and server feature distribution of the handshake within the session during the training phase, thereby enhancing the comprehensiveness and robustness of handshake behavior modeling.

[0027] In this embodiment, the division between normal training samples and high-risk training samples specifically includes: A self-supervised network is built based on a multi-view handshake sample set. The self-supervised network includes a handshake encoding network, a normal branch projection head, and an abnormal branch projection head. The handshake encoding network is used to receive the handshake feature vector sequence in the multi-view handshake sample and output the handshake embedding vector. The normal branch projection head and the abnormal branch projection head are used to receive the handshake embedding vector and output the normal branch projection vector and the abnormal branch projection vector, respectively. In the handshake coding network, an encoding structure is set up to process the handshake feature vector sequence in time order. The encoding structure performs time-series encoding on the handshake protocol version field, cipher suite field, certificate chain field, random number segment, timestamp field, round-trip delay feature and message length sequence feature to generate a handshake embedding vector that corresponds one-to-one with the multi-view handshake sample. Based on the handshake feature vector sequence and handshake embedding vector, we calculate the downgrade risk correlation quantity, replay risk correlation quantity, and certificate replacement risk correlation quantity. The downgrade risk correlation quantity includes the number of times the security level of the cipher suite is reduced, the number of times the weak cipher suite appears, and the number of times the handshake protocol version is rolled back. The replay risk correlation quantity includes the deviation measure of the time interval sequence and message length sequence relative to the historical pattern. The certificate replacement risk correlation quantity includes the number of certificate chain field changes, the number of certificate fingerprint changes, and the number of abnormal certificate validity period markers. Calculate the downgrade risk indicator based on the downgrade risk related quantity, calculate the replay risk indicator based on the replay risk related quantity, calculate the certificate replacement risk indicator based on the certificate replacement risk related quantity, multiply the count field, deviation measurement field, and anomaly marker field in each risk related quantity by the preset weight coefficient and sum them to obtain the downgrade risk indicator, replay risk indicator, and certificate replacement risk indicator. Each risk indicator is a scalar. The handshake risk indicator is calculated based on the downgrade risk indicator, replay risk indicator, and certificate replacement risk indicator. The handshake risk indicator is obtained by multiplying the downgrade risk indicator, replay risk indicator, and certificate replacement risk indicator by their corresponding weight coefficients and summing them. The handshake risk indicator is a single scalar used to characterize the risk level of multi-view handshake samples. The handshake risk indicator is compared with a first risk threshold and a second risk threshold. Multi-view handshake samples with a handshake risk indicator not higher than the first risk threshold are classified as normal training samples, and multi-view handshake samples with a handshake risk indicator higher than the second risk threshold are classified as high-risk training samples.

[0028] This invention introduces a handshake coding network, a normal branch projection head, and an abnormal branch projection head into a self-supervised network to perform temporal encoding on the protocol version, cipher suites, certificate chains, temporal behavior, and length features in multi-view handshake samples. It also constructs degradation risk, replay risk, and certificate replacement risk metrics, mapping the number of times the cipher suite security level has decreased, the number of times weak cipher suites have appeared, the number of times the protocol version has been rolled back, the time interval and message length deviation metrics, the number of certificate chain changes, the number of certificate fingerprint changes, and the number of abnormal certificate validity period markers to a unified handshake risk indicator. Combined with a first risk threshold and a second risk threshold, the samples are automatically divided into normal training samples and high-risk training samples. This allows the self-supervised training process to obtain a sample distribution with clearly distinguishable risk levels without manual annotation, providing a solid data foundation for subsequent targeted modeling of the normal and abnormal subspaces.

[0029] In this embodiment, the generation of the normal subspace representation specifically includes: Select multi-view handshake samples that have been divided into normal training samples from the multi-view handshake sample set. Input the handshake feature vector sequence in the normal training samples into the handshake coding network and the normal branch projection head to obtain the normal branch projection vector. The normal branch projection vector is the vector representation obtained by projecting the embedded vector output by the handshake coding network through the normal branch projection head. In each training batch, a projection correlation matrix is ​​constructed based on the normal branch projection vector. The projection correlation matrix is ​​indexed by the projection dimension, and the matrix elements represent the correlation level between different projection dimensions. During the construction process, the sample mean within the batch is calculated for each projection dimension. For any two projection dimensions, the average of the sample-by-sample product within the batch is calculated and the product of the means of the two projection dimensions is subtracted. The result is written into the projection correlation matrix as the correlation level at the corresponding position. An improved Barlow Twins loss is defined based on the projection correlation matrix, semantic template correlation matrix, and protocol semantic adjacency matrix. The improved Barlow Twins loss includes semantic alignment constraints and redundancy suppression constraints. The semantic alignment constraints set different target correlation levels in the semantic template correlation matrix according to the strong dependency type, weak dependency type, and prohibited dependency type in the protocol semantic adjacency matrix, and apply weighted constraints on the correlation level deviation at the corresponding positions of the projection correlation matrix and the semantic template correlation matrix. The redundancy suppression constraints suppress the correlation level in the projection correlation matrix at the positions marked as prohibited dependency types in the protocol semantic adjacency matrix. The semantic alignment constraints are obtained by summing the weighted squares of the differences between the corresponding positions of the projection correlation matrix and the semantic template correlation matrix, and the redundancy suppression constraints are obtained by summing the squares of the differences between the correlation level at the prohibited dependency type position and the preset redundancy target level. The semantic alignment constraint and the redundancy suppression constraint are combined according to a preset coefficient to form an improved Barlow Twins loss. During the training process of normal training samples, the improved Barlow Twins loss is used as the objective function to update the parameters of the handshake coding network and the normal branch projection head. During the parameter update process, the gradient is calculated based on the improved Barlow Twins loss and the network parameters are iteratively updated. After completing the preset training rounds, the parameters of the handshake coding network and the normal branch projection head are fixed, and the output space obtained by passing the normal training samples through the handshake coding network and the normal branch projection head is determined as the normal subspace representation.

[0030] This invention constructs normal branch projection vectors on normal training samples and generates projection correlation matrices accordingly, enabling the embedding vectors output by the handshake coding network to form a measurable correlation structure at the batch scale. It also constructs an improved Barlow Twins loss by combining the semantic template correlation matrix and the protocol semantic adjacency matrix, integrating semantic alignment constraints and redundancy suppression constraints into the training process. This ensures that different types of semantic dependencies have clear correlation targets in the representation space, strongly suppresses prohibited dependency positions, and ultimately enables the normal subspace representation to stably reflect the normal semantic structure of the VPN handshake at the protocol behavior level. It also establishes a structurally consistent benchmark space for subsequent anomaly deviation detection.

[0031] In this embodiment, the generation of the abnormal subspace representation specifically includes: Select multi-view handshake samples that are classified as high-risk training samples from the multi-view handshake sample set, and input the handshake feature vector sequence in the high-risk training samples into the handshake coding network and the abnormal branch projection head to obtain the abnormal branch projection vector. In each training batch, an anomaly branch projection correlation matrix is ​​constructed based on the anomaly branch projection vector. The anomaly branch projection correlation matrix is ​​indexed by the projection dimension, and each matrix element represents the correlation level between the corresponding two projection dimensions. The anomaly branch projection correlation matrix is ​​obtained by statistically analyzing the joint changes of the anomaly branch projection vector in each projection dimension within the training batch. Each matrix element is calculated from the joint statistics of the corresponding two projection dimensions on the batch samples. Based on the anomaly branch projection correlation matrix, a sparse correlation constraint is defined. The position where the row index and column index of the anomaly branch projection correlation matrix are different is regarded as the off-diagonal position. The sparse correlation constraint term is constructed by the correlation level at the off-diagonal position. The sparse correlation constraint term is obtained by weighted aggregation of the correlation levels of all off-diagonal positions and is used to measure the overall correlation strength of the anomaly branch projection correlation matrix at the off-diagonal position. Based on the anomaly branch projection correlation matrix and semantic node field type definition, semantic decoupling constraints are defined. Semantic nodes are divided into semantic groups according to the handshake protocol version field, cipher suite field, certificate chain field, random number segment, timestamp field, delay feature field, and message length feature field. Semantic node pairs belonging to different semantic groups are selected, and semantic decoupling constraint terms are constructed based on the correlation level at the corresponding matrix position. The semantic decoupling constraint terms are obtained by aggregating the correlation level at the position across semantic groups, and are used to measure the correlation strength of the anomaly branch projection correlation matrix at the position across semantic groups. Based on the normal branch projection vector and the abnormal branch projection vector, cross-branch orthogonality constraints are defined. For each high-risk training sample, the cross-branch correlation metric between the normal branch projection vector and the abnormal branch projection vector is calculated. The cross-branch orthogonality constraint term is constructed by aggregating the cross-branch correlation metrics of all high-risk training samples. The cross-branch orthogonality constraint term is obtained by aggregating the cross-branch correlation metrics and is used to measure the degree of close orthogonality between the normal branch projection vector and the abnormal branch projection vector. The sparse correlation constraint, semantic decoupling constraint, and cross-branch orthogonality constraint are weighted and summed according to the first loss weight, the second loss weight, and the third loss weight to obtain the abnormal branch loss function. The abnormal branch loss function is used to simultaneously constrain the sparsity of the abnormal branch projection correlation matrix at off-diagonal positions, the correlation strength at cross-semantic grouping positions, and the orthogonality between the normal branch projection vector and the abnormal branch projection vector in the same target. During the training process of high-risk training samples, gradient calculation and parameter updates are performed according to the abnormal branch loss function. After completing the preset training rounds, the output space of the handshake coding network and the abnormal branch projection head on the high-risk training samples is determined as the abnormal subspace representation.

[0032] This invention constructs anomaly branch projection vectors on high-risk training samples and forms anomaly branch projection correlation matrix, enabling the model to extract representative joint change patterns from anomalous behaviors. It also suppresses off-diagonal correlation structures through sparse correlation constraints, resulting in anomaly subspaces exhibiting sparse, sharp, and clearly directional anomalous feature distributions. Furthermore, semantic decoupling constraints weaken interference correlations between different semantic groups, ensuring clear differentiation of anomalous representations across semantic group dimensions. Cross-branch orthogonality constraints ensure that the anomalous branch projection vectors and normal branch projection vectors remain nearly orthogonal in the representation space, allowing the model to establish two independent structured representation spaces during training. This enables the anomalous subspace to centrally express the change patterns of anomalous behaviors while avoiding interference from normal behavior features, providing a more stable and sensitive foundational representation for subsequent anomalous activation calculations and anomalous type differentiation.

[0033] In this embodiment, the output of the anomaly identification result and the anomaly type specifically includes: During online operation, VPN handshake traffic is received, handshake risk indicator is calculated, and a low-risk handshake sample set is determined based on the comparison between the handshake risk indicator and a preset update threshold. An update batch is constructed based on low-risk handshake samples. The handshake feature vector sequence in the update batch is input into the handshake coding network and the normal branch projection head to obtain the normal subspace representation of the update batch. The update batch projection correlation matrix is ​​constructed, and the update batch projection correlation matrix is ​​weighted and combined with the current semantic template correlation matrix according to the preset update coefficient to obtain a new semantic template correlation matrix. During the detection phase, the handshake feature vector sequence corresponding to the handshake of the VPN under test is received. The handshake feature vector sequence is input into the handshake coding network and the normal branch projection head to obtain the normal subspace representation. The handshake feature vector sequence is input into the handshake coding network and the abnormal branch projection head to obtain the abnormal subspace representation. The normal subspace representation and the abnormal subspace representation are used for template deviation degree and abnormal activation degree calculation, respectively. Template deviation is calculated based on the normal subspace representation and the semantic template correlation matrix. The template deviation is obtained by statistically analyzing the differences in correlation levels at corresponding positions of the detection projection correlation matrix obtained from the normal subspace representation and the semantic template correlation matrix, and then aggregating them. The abnormal activation degree is calculated based on the abnormal subspace representation. The abnormal activation degree is obtained by aggregating the responses of the abnormal subspace representation on the preset abnormal sensitivity dimension. The VPN handshake anomaly score is calculated based on the template deviation and abnormal activation. The template deviation and abnormal activation are combined into a single scalar VPN handshake anomaly score according to a preset weight. The VPN handshake anomaly score is then compared with a set of scoring thresholds to distinguish different anomaly types, resulting in anomaly identification results and anomaly type output content.

[0034] Example 1: To verify the feasibility of this invention in practice, it was applied to a typical enterprise-level VPN access security monitoring scenario. This scenario involves a large number of remote office terminals, automated operation and maintenance scripts, and encrypted access traffic from different regions. The VPN server generates a considerable number of handshake message sequences every day. In the past, monitoring systems relied on rules and manual experience to judge handshake anomalies, which often resulted in delayed identification, missed detections, and false alarms. Especially when facing hidden anomalies such as downgrade attacks, replay behavior, and certificate chain replacement, traditional methods are difficult to accurately capture early signs. This invention aims to solve this problem by establishing protocol semantic templates, normal subspaces, and abnormal subspaces through self-supervised learning, enabling the system to identify potential anomalies under unlabeled conditions.

[0035] In this scenario, the present invention first continuously collects the handshake traffic generated by the VPN server, and parses the message sequence corresponding to each handshake into a handshake feature vector sequence according to the session order, covering seven types of features: protocol version, cipher suite, certificate chain, random number, timestamp, round-trip time, and message length. Then, based on the protocol semantic relationships between these features, a protocol semantic graph is constructed, and a semantic template correlation matrix is ​​generated for training, enabling the model to understand the internal logical structure of the VPN handshake. After establishing the basic semantic structure, the present invention constructs a session view, a certificate cluster view, and a server fingerprint view. Figure 3 The class views are combined to form a multi-view handshake sample input to the self-supervised network. The self-supervised network includes a handshake encoding network, a normal branch projection head, and an abnormal branch projection head. The normal branch uses an improved Barlow Twins loss to construct a normal subspace representation, while the abnormal branch constructs an abnormal subspace representation through sparse correlation constraints, semantic decoupling constraints, and cross-branch orthogonal constraints. The two types of subspaces together constitute a bidirectional characterization of the handshake behavior.

[0036] In practical use, after the self-supervised model of this invention is trained, it is deployed in the real-time analysis module on the VPN server side. Whenever a new VPN handshake is established, the handshake feature vector sequence is sent into the encoding network and projected into the normal subspace and the abnormal subspace respectively. The normal subspace is used to compare with the online updated semantic template correlation matrix to generate the template deviation degree, and the abnormal subspace is used to calculate the abnormal activation degree. Finally, the system merges the two types of indicators to generate a single abnormal score, which is used to determine whether there is a risk of downgrade, replay or certificate replacement in the handshake.

[0037] To visually demonstrate the effectiveness of this invention, its model is compared with traditional rule-matching-based recognition methods, traditional single-branch self-supervised contrastive learning methods, and common shallow feature classifier-based methods. Under the same dataset conditions, handshake anomaly detection was performed, and the performance of each method is shown in Table 1:

[0038] Table 1. Performance Comparison of Handshake Anomaly Detection

[0039] The comparative results show that the present invention achieves an identification rate of over 90% for all three types of anomalies, which is a significant improvement over other methods. Specifically, the present invention implements protocol structure constraints through a semantic template correlation matrix, enabling earlier identification of minor protocol version changes in degradation anomalies. Through the anomaly subspace construction mechanism, anomalies exhibited by replay behavior in terms of latency and message length patterns can be captured. Through certificate chain-related semantic relationship modeling, certificate replacement behavior no longer relies on simple fingerprint comparison, but can identify more subtle change patterns. At the same time, the false alarm rate is significantly reduced, indicating that the present invention can maintain a more stable discrimination capability in normal traffic.

[0040] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A VPN handshake anomaly identification method based on self-supervised learning, characterized in that, The method comprises the following steps: acquiring unlabeled VPN handshake traffic, grouping by session and parsing handshake fields, and constructing a handshake feature vector sequence; constructing a protocol semantic graph based on the handshake feature vector sequence, generating a protocol semantic adjacency matrix and converting it into a semantic template correlation matrix; constructing a multi-view handshake sample according to the session, certificate cluster and server fingerprint to form a multi-view handshake sample set; building a self-supervised network including a handshake encoding network, a normal branch projection head and an abnormal branch projection head, calculating a handshake risk indicator and dividing the multi-view handshake sample into normal training samples and high-risk training samples; aligning the projection correlation matrix with the semantic template correlation matrix and performing redundancy suppression for the normal training samples in the normal branch using an improved Barlow Twins loss to obtain a normal subspace representation; applying sparse correlation constraints, semantic decoupling constraints and cross-branch orthogonal constraints to the high-risk training samples in the abnormal branch to obtain an abnormal subspace representation; updating the semantic template correlation matrix based on low-risk handshake samples during online operation, obtaining normal subspace representation and abnormal subspace representation for the handshake under test through the handshake encoding network and the normal branch projection head and the abnormal branch projection head, calculating the template deviation and the abnormal activation and combining to obtain the VPN handshake anomaly score, and outputting the anomaly recognition result and the anomaly type.

2. The VPN handshake anomaly identification method based on self-supervised learning according to claim 1, characterized in that, The construction of the handshake feature vector sequence specifically comprises: acquiring unlabeled VPN handshake traffic and grouping by session; selecting handshake stage packets in each session and arranging them in chronological order; performing handshake field analysis on the handshake stage packets; performing certificate chain field analysis on the certificate related packets in the handshake stage; calculating the handshake stage round trip time, packet interval time and packet length sequence based on the timestamp and packet length of the handshake stage packets, performing numerical coding and normalization processing on the handshake protocol version field, the cipher suite related field, the certificate chain field, the random number field, the timestamp field, and the round trip time and packet length related field, and arranging the coding results in chronological order in their respective sessions to generate the handshake feature vector sequence for the corresponding session.

3. The VPN handshake anomaly identification method based on self-supervised learning according to claim 1, characterized in that, The generation of the semantic template correlation matrix specifically comprises: establishing a semantic node set based on the handshake protocol version field, the cipher suite field, the certificate chain field, the random number field, the timestamp field, the round trip time feature and the packet length sequence feature in the handshake feature vector sequence; annotating the dependency relationship type between the semantic nodes in the semantic node set according to the VPN handshake specification and the security policy; generating a protocol semantic adjacency matrix according to the protocol semantic graph; generating a semantic template correlation matrix based on the protocol semantic adjacency matrix and setting the target correlation level corresponding to different dependency relationships, and using the processed semantic template correlation matrix as the semantic template correlation matrix used in the training stage.

4. The VPN handshake anomaly identification method based on self-supervised learning according to claim 1, characterized in that, The construction of the multi-view handshake sample specifically comprises: dividing the handshake feature vector sequence into session sets based on the session identifier, arranging the handshake feature vectors in chronological order for each session set to form a session view; constructing a certificate cluster based on a certificate chain field in the handshake feature vector sequence, clustering session sets with same certificate chain field content, and merging session views in each cluster to form a certificate cluster view corresponding to the certificate cluster; generating a server fingerprint based on server network address information, certificate fingerprint information, and a cipher suite field according to a preset rule, clustering session sets with same server fingerprint, and merging session views in each cluster to form a server fingerprint view corresponding to the server fingerprint; constructing a multi-view handshake sample based on the session view, the certificate cluster view, and the server fingerprint view, selecting a corresponding session view and a certificate cluster view containing the same session to form a first view combination for each session view, selecting a corresponding session view and a server fingerprint view containing the same session to form a second view combination for each session view, recording the first view combination and the second view combination as the multi-view handshake sample, and merging all multi-view handshake samples to form a multi-view handshake sample set.

5. The VPN handshake anomaly identification method based on self-supervised learning according to claim 1, characterized in that, The division of the normal training sample and the high-risk training sample specifically includes: building a self-supervised network based on the multi-view handshake sample set, the self-supervised network including a handshake encoding network, a normal branch projection head, and an abnormal branch projection head; setting an encoding structure for processing the handshake feature vector sequence in time sequence in the handshake encoding network, performing time sequence encoding on the handshake feature vector sequence to obtain a handshake embedding vector corresponding to the multi-view handshake sample; constructing a downgrade risk related quantity, a replay risk related quantity, and a certificate replacement risk related quantity based on the handshake feature vector sequence and the handshake embedding vector, the downgrade risk related quantity being used to represent cipher suite negotiation and protocol version change, the replay risk related quantity being used to represent time interval mode and message length mode deviation, and the certificate replacement risk related quantity being used to represent certificate chain field and certificate fingerprint change; calculating a downgrade risk indicator based on the downgrade risk related quantity, a replay risk indicator based on the replay risk related quantity, and a certificate replacement risk indicator based on the certificate replacement risk related quantity, the downgrade risk indicator, the replay risk indicator, and the certificate replacement risk indicator all being scalars; calculating a handshake risk indicator based on the downgrade risk indicator, the replay risk indicator, and the certificate replacement risk indicator, combining the downgrade risk indicator, the replay risk indicator, and the certificate replacement risk indicator according to a preset weight to obtain the handshake risk indicator, and dividing the multi-view handshake sample into the normal training sample and the high-risk training sample according to a comparison result of the handshake risk indicator and a preset risk threshold.

6. The VPN handshake anomaly identification method based on self-supervised learning according to claim 1, characterized in that, The generation of the normal subspace representation specifically includes: selecting a multi-view handshake sample divided into the normal training sample from the multi-view handshake sample set, inputting the handshake feature vector sequence in the normal training sample into the handshake encoding network and the normal branch projection head to obtain a normal branch projection vector; in each training batch, constructing a projection correlation matrix based on the normal branch projection vector, the projection correlation matrix being indexed by projection dimensions, and each matrix element being used to represent a correlation level between two projection dimensions; The improved Barlow Twins loss is defined based on a projection correlation matrix, a semantic template correlation matrix and a protocol semantic adjacency matrix, and the improved Barlow Twins loss includes a semantic alignment constraint and a redundancy suppression constraint. The semantic alignment constraint sets different target correlation levels in the semantic template correlation matrix according to strong dependency types, weak dependency types and forbidden dependency types in the protocol semantic adjacency matrix, and performs weighted constraint on correlation level deviations at corresponding positions of the projection correlation matrix and the semantic template correlation matrix, and the redundancy suppression constraint suppresses the correlation level in the projection correlation matrix at positions marked as forbidden dependency types in the protocol semantic adjacency matrix. The semantic alignment constraint and the redundancy suppression constraint are combined according to a preset coefficient group to form the improved Barlow Twins loss, and the improved Barlow Twins loss is used as a target function to update parameters of the handshake encoding network and the normal branch projection head in a normal training sample training process.

7. The VPN handshake anomaly identification method based on self-supervised learning according to claim 1, characterized in that, After a preset training round is completed, the parameters of the handshake encoding network and the normal branch projection head are fixed, and an output space of the normal training sample obtained through the handshake encoding network and the normal branch projection head is determined as a normal subspace representation. The generation of the abnormal subspace representation specifically includes: A multi-view handshake sample divided into a high-risk training sample is selected from a multi-view handshake sample set, and a handshake feature vector sequence in the high-risk training sample is input into the handshake encoding network and the abnormal branch projection head to obtain an abnormal branch projection vector. In each training batch, an abnormal branch projection correlation matrix is constructed according to the abnormal branch projection vector, and the abnormal branch projection correlation matrix is indexed by a projection dimension, and each matrix element represents a correlation level between corresponding two projection dimensions. A sparse correlation constraint is defined based on the abnormal branch projection correlation matrix, positions with different row indexes and column indexes in the abnormal branch projection correlation matrix are regarded as non-diagonal positions, and a sparse correlation constraint term is constructed based on correlation levels at the non-diagonal positions, to limit the overall correlation strength of the abnormal branch projection correlation matrix at the non-diagonal positions. A semantic decoupling constraint is defined based on the abnormal branch projection correlation matrix and a semantic node field type, semantic nodes are divided into semantic groups according to a handshake protocol version field, a cipher suite field, a certificate chain field, a random number field, a timestamp field, a time delay feature field and a message length feature field, semantic node pairs belonging to different semantic groups are selected, and a semantic decoupling constraint term is constructed based on correlation levels at corresponding matrix positions, to limit the correlation strength of the abnormal branch projection correlation matrix at cross-semantic group positions. A cross-branch orthogonality constraint is defined based on the normal branch projection vector and the abnormal branch projection vector, a cross-branch correlation measure of the normal branch projection vector and the abnormal branch projection vector is calculated for each high-risk training sample, and a cross-branch orthogonality constraint term is constructed based on cross-branch correlation measures of all high-risk training samples, to depict the orthogonality degree between the normal branch projection vector and the abnormal branch projection vector. The sparse correlation constraint term, the semantic decoupling constraint term and the cross-branch orthogonal constraint term are weighted and summed according to the first loss weight, the second loss weight and the third loss weight to obtain an abnormal branch loss function, the parameters of the handshake coding network and the abnormal branch projection head are updated in the high-risk training sample training process by taking the abnormal branch loss function as a target function, and after a preset training round is completed, the output space of the handshake coding network and the abnormal branch projection head on the high-risk training sample is determined as an abnormal subspace representation.

8. The VPN handshake anomaly identification method based on self-supervised learning according to claim 1, characterized in that, The output of the abnormal identification result and the abnormal type specifically includes: During the online running process, the VPN handshake traffic is received, the handshake risk indicator is calculated, and the low-risk handshake sample is determined based on the handshake risk indicator; Based on the low-risk handshake sample, an update batch is constructed, the handshake feature vector sequence in the update batch is input into the handshake coding network and the normal branch projection head to obtain a normal subspace representation of the update batch, and the semantic template correlation matrix is updated according to the normal subspace representation of the update batch; In the detection stage, the handshake feature vector sequence corresponding to the to-be-tested VPN handshake is received, the handshake feature vector sequence is input into the handshake coding network and the normal branch projection head to obtain a normal subspace representation, and the handshake feature vector sequence is input into the handshake coding network and the abnormal branch projection head to obtain an abnormal subspace representation; Based on the normal subspace representation and the semantic template correlation matrix, a template deviation degree is calculated; Based on the abnormal subspace representation, an abnormal activation degree is calculated; According to the template deviation degree and the abnormal activation degree, a VPN handshake abnormal score is calculated, an abnormal identification result is determined according to the VPN handshake abnormal score, and the VPN handshake abnormal score and the abnormal type in the abnormal identification result are output.

Images