An encrypted traffic anomaly detection method and system based on a pre-trained model and a residual quantization codebook

Abstract

The application discloses an encrypted traffic anomaly detection method and system based on a pre-training model and a residual quantization codebook, comprising the following steps: (1) obtaining original network encrypted traffic, constructing a burst sequence, inputting the burst sequence into a pre-trained BERT model, and extracting high-dimensional continuous context embedding features; (2) inputting the continuous context embedding features into a multi-layer residual vector quantizer RQ-VAE, and reconstructing quantized embedding features; (3) in the training stage, collecting and saving the activated codebook entries of each layer of the training data, and constructing a core set memory library representing the boundary of normal traffic distribution; (4) in the inference stage, the quantization error score and the local scaling memory score are calculated and fused to determine abnormal traffic; (5) in the inference process, online adaptive defense based on contrastive learning. By using the application, efficient and accurate anomaly traffic detection can be realized.

Description

Technical Field

[0001] This invention relates to the intersection of network security and artificial intelligence, and in particular to a method and system for detecting encrypted traffic anomalies based on a pre-trained model and residual quantization code. Background Technology

[0002] With increasing awareness of internet privacy, network traffic encryption has become an irreversible trend. Currently, over 90% of web traffic is transmitted using strong encryption protocols such as TLS 1.3, QUIC, Tor, and VPN. While encryption protects user privacy, it also provides a natural haven for malware, ransomware, and advanced persistent threats (APTs). Traditional deep packet inspection (DPI) techniques rely on keywords or signatures in plaintext payloads, and their effectiveness decreases significantly when facing encrypted traffic.

[0003] In recent years, deep learning-based encrypted traffic classification technology has made significant progress. In particular, drawing inspiration from the Transformer architecture in Natural Language Processing (NLP), Lin et al. proposed ET-BERT (Lin X, Xiong G, Gou G, Li Z, Shi J, Yu J. ET-BERT: A Contextualized Datagram Representation with Pre-training Transformers for Encrypted Traffic Classification. In Proceedings of the ACM Web Conference 2022 (WWW'22), April 25–29, 2022, Virtual Event, Lyon, France. ACM, New York, NY, USA, 11 pages.). ET-BERT learns byte-level contextual dependencies in data packets through pre-training on massive amounts of unlabeled encrypted traffic, achieving industry-leading accuracy in downstream classification tasks.

[0004] However, existing encrypted traffic detection methods based on pre-trained models (such as ET-BERT) face three fatal flaws in practical industrial applications: Defect 1: The "long sequence modeling bottleneck" leads to severe truncation of global features at the flow level.

[0005] The computational complexity of the self-attention mechanism in the Transformer architecture increases quadratically with the sequence length. In practical applications of BERT, due to limitations in model input length, only the first few hundred bytes or the first few packets of a traffic session can typically be truncated for input (e.g., truncated to 512 tokens). However, real malicious network interactions are usually persistent and multi-stage. Truncation causes the model to completely lose its global behavioral patterns at the flow level, making it extremely vulnerable to attackers who can bypass detection by padding the connection with useless bytes at the beginning.

[0006] Defect 2: The traditional fine-tuning paradigm is based on the "closed-world" assumption and cannot identify zero-day anomalies.

[0007] Existing solutions typically involve supervised fine-tuning using known categories of malicious and normal traffic after pre-training. However, anomalies in cyberspace are unbounded, and such supervised discriminative models can only detect malware categories that have appeared in the training set. When faced with constantly evolving new Trojans or zero-day vulnerability attacks, the model often misclassifies them as normal traffic, resulting in extremely poor generalization ability.

[0008] Defect 3: It ignores the "concept drift" of network traffic, causing the model's performance to degrade over time.

[0009] The real-world internet environment is highly dynamic; the distribution of normal business traffic can drift due to updates to application layer architecture, changes in CDN nodes, or alterations to the DOM structure. If the model adheres rigidly to the "normal" criteria used during training, it will generate a large number of false positives several weeks after deployment. Existing methods lack the ability to adapt online without supervision, and relabeling data and retraining is extremely costly.

[0010] In summary, how to break the sequence length limitation of pre-trained models to achieve long-flow modeling, how to get rid of the dependence on training samples to achieve open-world unknown anomaly detection, and how to enable models to have online adaptive capabilities to resist concept drift are the technical challenges that urgently need to be solved in the field of encrypted traffic analysis. Summary of the Invention

[0011] To overcome the shortcomings of existing pre-trained models, such as information truncation, inability to detect unknown anomalies, and lack of resistance to concept drift, this invention proposes an encrypted traffic anomaly detection method and system based on a pre-trained model and residual quantization codebook. It innovatively introduces multi-layer residual vector quantization (RQ-VAE) into the output of BERT for dimensionality reduction representation, and utilizes the logic of online adaptive codebook to construct a dual anomaly scoring mechanism, thereby achieving efficient and accurate anomaly traffic detection.

[0012] Furthermore, the multilayer residual vector quantizer RQ-VAE described in this invention is based on the residual quantization framework proposed by Lee et al. (Lee D, Kim C, Kim S, Cho M, Han W S. Autoregressive Image Generation using Residual Quantization. In Proceedings of the IEEE / CVFConference on Computer Vision and Pattern Recognition (CVPR), 2022, pp.11523-11532.), which was originally used for image autoregressive generation tasks. This invention is the first to transfer the aforementioned residual quantization mechanism to the field of encrypted traffic anomaly detection and deeply couple it with the encrypted traffic pre-trained representation represented by ET-BERT. On the one hand, it utilizes the strong general contextual semantics learned by ET-BERT at the packet byte level to provide RQ-VAE with a high-fidelity continuous input representation, avoiding the semantic loss caused by directly quantizing the original bytes. On the other hand, it uses the multi-level discrete codebook of RQ-VAE to compress the aforementioned high-dimensional continuous representation into a hierarchical semantic index with an extremely short length, thereby breaking through the inherent input length limitation of ET-BERT and enabling it to carry tens or even hundreds of times the length of session-level traffic information within the same context window. The coupling method of "ET-BERT continuous semantic representation + RQ-VAE multi-level discrete quantization" is not disclosed in existing encrypted traffic detection schemes (such as using only ET-BERT for supervised classification) or existing RQ-VAE applications (such as image generation and generative retrieval), and is the key innovation of this invention that distinguishes it from existing technologies.

[0013] A method for detecting encrypted traffic anomalies based on a pre-trained model and residual quantization codebook includes the following steps: (1) Obtain the original encrypted network traffic, aggregate it according to the session flow, and extract the burst sequence consisting of continuous data packets from the client to the server and from the server to the client; input the burst sequence into the pre-trained BERT model to extract high-dimensional continuous context embedding features. ; (2) Input the continuous context embedding features into the multi-layer residual vector quantizer RQ-VAE. By calculating the residuals layer by layer and matching the corresponding discrete codebooks of each layer, the high-dimensional continuous features are compressed into a hierarchical semantic index sequence, and the quantized embedding features are reconstructed. ; (3) During the training phase which only includes normal traffic, collect and save the code entries of each layer that are finally activated by the training data, and construct a core set memory that represents the boundary of the normal traffic distribution. ; (4) During the inference phase, the input traffic to be tested is processed by the BERT model and RQ-VAE to obtain the corresponding quantized embedding features. Calculate the quantization error score of each relative to the matched code entry. And the locally scaled memory fraction relative to the nearest neighbor entries in the core set memory. The final anomaly score is obtained by combining the two scores. If the score exceeds the threshold, it is judged as abnormal traffic; (5) During the inference process, check the set of codebook entries activated in RQ-VAE for the input flow to be tested, generate pseudo-labels based on the activation state of the codebooks, and use the pseudo-labels to perform contrastive learning, dynamically updating the parameters of the BERT model and RQ-VAE as well as the core set memory. To adapt to the drift of flow distribution.

[0014] In step (1), this invention abandons the traditional approach of truncating fixed bytes and uses "session" as the granularity to divide data packets from the same source and destination into burst sequences for the client and server. A pre-trained BERT model is used to extract continuous embeddings for each data packet.

[0015] In step (2), to address the bottleneck of storing and computing long sequence features, this invention introduces Residual Vector Quantization (RQ-VAE). The high-dimensional continuous vector output by BERT is mapped to multiple layers of discrete codebooks, from coarse to fine. Each codebook captures traffic features of different granularities (from macroscopic protocol features to microscopic load randomness). In this way, the originally massive floating-point continuous matrix is ​​compressed into extremely short discrete "hierarchical semantic IDs," enabling the model to process stream-level data of hundreds or thousands of times longer within the same context window.

[0016] In step (2), the working process of the multilayer residual vector quantizer RQ-VAE is as follows: Set the number of quantization layers to Each layer corresponds to an independent feature codebook. ,in , Size of the codebook; For the Input residual features of the layer In the Find the entry with the closest Euclidean distance in the layer codebook As a quantized representation of the current layer: ; Calculate the input residuals of the next layer: ; go through After layer quantization, the original continuous context is embedded in the features. Compressed to a length of semantic index sequence And calculate the final quantized embedding features. .

[0017] Furthermore, when hour, Continuous context embedding features output by pre-trained BERT models .

[0018] In step (3), this invention abandons the traditional mode of collecting malicious samples for binary classification fine-tuning. During the training phase, only pure normal traffic (such as whitelisted traffic of normal business) is input. Through the RQ-VAE quantization process, all code entries activated by normal traffic are recorded, and these frequently activated entries are extracted to construct a "coreset memory bank". This memory bank is essentially a discretized boundary dictionary of the normal traffic feature manifold at multiple scales.

[0019] In step (4), the quantization error score The formula used to measure the degree to which the quantized embedding features of the input deviate from the normal codebook manifold space is: ; in, This represents the continuous context embedding features obtained by the BERT model from the traffic to be tested. This invention calculates the MSE distance between the continuous features of the input traffic and the quantized features reconstructed through the codebook. Since the codebook only contains normal patterns during training, it will face significant reconstruction errors for unseen malicious or abnormal traffic.

[0020] Local scaling memory fraction The calculation process is as follows: In the core set memory Quantized embedding features for retrieval of nearest neighbor samples Local scaling distance calculation is used to adapt to density differences in the feature space: ; in, For samples in the memory bank Local density scale, This is the smoothing constant.

[0021] This design enables the model to reduce false negatives in densely populated areas and false positives in sparsely populated areas.

[0022] The final anomaly score is obtained by combining the two scores, as follows: Quantization error fraction and local scaling memory fraction Normalization yields and ;Will and The final outlier score is obtained by weighted summation. : ; in, These are the weight parameters.

[0023] In step (5), pseudo-tags are generated based on the activation state of the codebook, specifically as follows: If the activated code exists in the normal traffic core set memory built during the training phase, this entry is valid. If a sample is found to be normal and has a normal confidence level, it is assigned a pseudo-label; if an unseen code entry is activated, it is assigned a pseudo-label representing a potential anomaly.

[0024] Based on pseudo-labels, an InfoNCE contrastive loss is constructed for confidence samples. This approach shortens the distance between samples with the same pseudo-label in the feature space while widening the distance between samples with different pseudo-labels; it fine-tunes the BERT and RQ-VAE parameters using only samples with known pseudo-labels, and incrementally updates the core set memory with newly activated normal code entries. middle.

[0025] Compared with the prior art, the present invention has the following beneficial effects: 1. Overcoming the "truncation bottleneck" to achieve global flow modeling for long sequences: This invention compresses BERT's high-dimensional continuous features into discrete codebook indices using RQ-VAE. This dimensionality reduction and reconstruction mechanism reduces space usage by several orders of magnitude, enabling the system to perform global context analysis on complete session streams of several thousand bytes without truncation. This significantly reduces the impact of malicious traffic evading detection by pre-padded junk data.

[0026] 2. Overcomes the "closed world" limitation and possesses powerful zero-day anomaly identification capabilities: By innovatively constructing a "core codebase" containing only normal patterns, and combining it with a dual scoring mechanism of "quantization error + local scaling memory distance," this invention eliminates the need for prior knowledge of abnormal patterns. Any unknown anomalies or zero-day vulnerabilities that do not conform to normal network behavior logic will generate a significant deviation score when mapped to the normal codebase, thus enabling accurate identification.

[0027] 3. Immune flow "concept drift," enabling online learning and adaptive defense: To address the rising false alarm rate caused by the high-frequency evolution of network environments, this invention's online adaptive mechanism utilizes the matching states of discrete codebooks as "natural pseudo-labels." Without introducing manual annotation, it continuously updates the model during the inference phase through contrastive learning. While ensuring it is not "contaminated" by abnormal traffic, the model can spontaneously learn and adapt to new normal business patterns, significantly reducing the long-term maintenance costs of the system. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 The flowchart illustrates an encrypted traffic anomaly detection method based on a pre-trained model and residual quantization codebook, as provided in this embodiment of the invention.

[0030] Figure 2 This is a schematic diagram of encrypted traffic preprocessing and continuous context embedding feature extraction in an embodiment of the present invention.

[0031] Figure 3 This is a schematic diagram of feature discretization based on multilayer residual vector quantization (RQ-VAE) in an embodiment of the present invention.

[0032] Figure 4 This is a schematic diagram of the core set construction and unknown anomaly detection process based on a dual scoring mechanism in an embodiment of the present invention.

[0033] Figure 5 This is a schematic diagram of online adaptive defense update based on contrastive learning in an embodiment of the present invention. Detailed Implementation

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.

[0036] like Figure 1 As shown, an encrypted traffic anomaly detection method based on a pre-trained model and residual quantization codebook mainly includes the following steps: Step S1: Encrypted traffic preprocessing and continuous feature (embedding) extraction.

[0037] Traffic in a real network environment contains packets from multiple sources and destinations. In order to capture logically meaningful context, the original packet capture file (PCAP) is first restored into a complete session flow based on the five-tuple (source IP, destination IP, source port, destination port, protocol).

[0038] In application-layer interactions, client requests and server responses are typically bursty. A BURST is defined as a continuous, unidirectional collection of data packets transmitted within a single session stream.

[0039] For the extracted continuous BURST sequence, this invention adopts the same Datagram2Token mechanism as BERT, using Bi-gram to convert the hexadecimal bytes of the message into a token sequence, with each token consisting of two adjacent bytes (vocabulary size set to 65536). A special marker [CLS] is added to indicate the start of the sequence.

[0040] The preprocessed token sequence is input into the pre-trained BERT model. BERT contains a 12-layer bidirectional Transformer encoder. After calculation using the self-attention mechanism, for the input token sequence... We extract the last hidden state of BERT as a continuous embedding feature, denoted as . ,in It is usually 512.

[0041] Traditional methods are used directly here Linear classification not only limits the input length but also restricts it to supervised classification. This invention leads to a completely different path of depth quantization and open-world detection.

[0042] Step S2, multi-level residual vector quantization (RQ-VAE) of features.

[0043] The aforementioned continuous features are then fed into the Residual Vector Quantization module.

[0044] RQ-VAE reconstructs the input in a coarse-to-fine manner through multiple cascaded quantizers. Let the total number of quantization layers be... (For example Each layer independently maintains a learnable discrete codebook. ,in For codebook capacity (e.g.) ).

[0045] The quantization process is a recursive residual matching process. Let the initial residuals be the output features of BERT, i.e. .

[0046] In the layer( ), the system in the Search for the current residual in the layer codebook The closest Euclidean entry (Prototype): ; in That is, the first The quantization index (Semantic ID) of the layer. The quantization feature of the current layer is denoted as... .

[0047] Calculate the residual and use it as input to the next quantizer: ; After layer quantization, the original high-dimensional floating-point continuous features are compressed to an extreme into a vector composed of discrete integers.

[0048] Final reconstructed quantized embedding features It is the sum of all matched entries across all layers: ; During this training phase, the model's objective function is optimized. Includes refactoring loss, code alignment loss, and commitment loss: ; in This indicates that the gradient (Stop-gradient) operation is stopped. and This is the balancing factor. Only normal traffic data is used for training in this stage.

[0049] Step S3: Construction of the core set codebook memory.

[0050] After completing the above training on pure normal traffic, the codebooks of RQ-VAE have converged. At this point, the entries in the codebooks represent the typical feature centroids (prototypes) of "normal network traffic behavior".

[0051] To improve inference efficiency and filter redundant noise, this system scans the entire training set, counts all code entries that have actually been activated (hit), and groups these high-frequency entries into a set, defined as the core set memory. : ; At this point, the offline phase is complete. The system has established a compact memory representing the "known safe world".

[0052] S4, Unknown anomaly detection based on a dual scoring mechanism.

[0053] In the practical reasoning phase, when an unknown encrypted traffic enters the system, the system uses BERT to extract continuous features. The quantized features are obtained through RQ-VAE mapping. .

[0054] To identify anomalies with high accuracy and low false positives, this invention proposes a dual anomaly scoring mechanism in conjunction with the code-based mechanism: Score 1: Quantization Error Score ).

[0055] Quantization error reflects the degree to which input traffic can be represented by a "normal codebook". Malicious traffic has encryption randomness and burst patterns that differ from normal business traffic. Forcibly mapping it to a normal codebook will produce a huge truncation residual: ; Score 2: Local Scaling Memory Score ).

[0056] Due to the wide variety of network traffic protocols (e.g., the feature space density differs drastically between HTTPS web browsing and DNS over TLS), relying solely on global absolute distance can lead to numerous false alarms in sparse traffic regions. Therefore, this invention calculates quantized features. To the core set memory The most recent memory vectors The distance, and the introduction of a local density scale. Perform dynamic scaling: Define core set samples Local scale For itself until recently The median distance between neighbors: ; Define test samples scale For it to be in the memory bank most recently The median distance between each neighbor.

[0057] The local scaling memory fraction is calculated as follows: ; This design makes the model extremely sensitive to small anomalous disturbances in dense clusters of normal traffic (such as regular web access), while it has better tolerance in sparse clusters of normal traffic (such as rare background heartbeats).

[0058] Total anomaly judgment (Score Aggregation): Due to the different physical meanings and dimensions of the two scores, the exponential moving average (EMA) is used to track historical statistics and dynamically normalize them (denoted as ). and The total score is a weighted sum: ; when Greater than the system's set judgment threshold When this occurs, an alarm is triggered to identify the abnormal traffic.

[0059] Step S5: Online adaptive update based on contrastive learning.

[0060] Static models are prone to aging due to concept drift caused by frequent iterations of web applications. This invention designs a secure adaptive evolution mechanism based on inference flow.

[0061] (1) Pseudo-tag generation based on code activation: During inference, the system monitors the sequence of indices activated by the quantizer in real time. If an index triggered by a test traffic sample falls entirely within our initially constructed core set... Within the range, and its total score Extremely low, meaning that although this batch of traffic may have slight offsets, its underlying logic is highly consistent with the normal baseline. The system automatically generates a "normal" pseudo-label for it. Conversely, generate an "abnormal" pseudo-label. Because it relies on the strict boundaries of the discrete codebook, the generation of these pseudo-labels is extremely reliable and does not require manually setting complex thresholds.

[0062] (2) Online comparative learning and model update: A contrastive learning loss (InfoNCE loss) is constructed using real-time received streaming data. In the feature space, features of the test samples labeled as normal are brought closer together. The distance between them, while pushing them away from the features of samples marked as anomalous ( ).

[0063] ; The system utilizes a small learning rate The security sample, combined with the aforementioned loss, fine-tunes the upper-level parameters of BERT in the background and updates the code. This ensures that the model will not be "poisoned" by malicious traffic, while the code itself can evolve with the deviation of normal traffic distribution, achieving "maintenance-free" lifelong learning in industrial-grade deployments.

[0064] The embodiments described above provide a detailed explanation of the technical solutions and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting encrypted traffic anomalies based on a pre-trained model and residual quantization codebook, characterized in that, Includes the following steps: (1) Obtain the original encrypted network traffic, aggregate it according to the session flow, and extract the burst sequence consisting of continuous data packets from the client to the server and from the server to the client; input the burst sequence into the pre-trained BERT model to extract high-dimensional continuous context embedding features. ; (2) Input the continuous context embedding features into the multi-layer residual vector quantizer RQ-VAE. By calculating the residuals layer by layer and matching the corresponding discrete codebooks of each layer, the high-dimensional continuous features are compressed into a hierarchical semantic index sequence, and the quantized embedding features are reconstructed. ; (3) During the training phase which only includes normal traffic, collect and save the code entries of each layer that are finally activated by the training data, and construct a core set memory that represents the boundary of the normal traffic distribution. ; (4) During the inference phase, the input traffic to be tested is processed by the BERT model and RQ-VAE to obtain the corresponding quantized embedding features. Calculate the quantization error score of each relative to the matched code entry. And the locally scaled memory fraction relative to the nearest neighbor entries in the core set memory. The final anomaly score is obtained by combining the two scores. If the score exceeds the threshold, it is judged as abnormal traffic; (5) During the inference process, check the set of codebook entries activated in RQ-VAE for the input flow to be tested, generate pseudo-labels based on the activation state of the codebooks, and use the pseudo-labels to perform contrastive learning, dynamically updating the parameters of the BERT model and RQ-VAE as well as the core set memory. To adapt to the drift of flow distribution.

2. The encrypted traffic anomaly detection method based on a pre-trained model and residual quantization codebook according to claim 1, characterized in that, In step (2), the working process of the multilayer residual vector quantizer RQ-VAE is as follows: Set the number of quantization layers to Each layer corresponds to an independent feature codebook. ,in , Size of the codebook; For the Input residual features of the layer In the Find the entry with the closest Euclidean distance in the layer codebook As a quantized representation of the current layer: ; Calculate the input residuals of the next layer: ; go through After layer quantization, the original continuous context is embedded in the features. Compressed to a length of semantic index sequence And calculate the final quantized embedding features. .

3. The encrypted traffic anomaly detection method based on a pre-trained model and residual quantization codebook according to claim 2, characterized in that, when hour, Continuous context embedding features output by pre-trained BERT models .

4. The encrypted traffic anomaly detection method based on a pre-trained model and residual quantization codebook according to claim 1, characterized in that, In step (4), the quantization error score The formula used to measure the degree to which the quantized embedding features of the input deviate from the normal codebook manifold space is: ； in, This represents the continuous contextual embedding features obtained by the BERT model for the traffic to be tested.

5. The encrypted traffic anomaly detection method based on a pre-trained model and residual quantization codebook according to claim 1, characterized in that, In step (4), the local scaling memory fraction is calculated. The calculation process is as follows: In the core set memory Quantized embedding features for retrieval of nearest neighbor samples Local scaling distance calculation is used to adapt to density differences in the feature space: ； in, For samples in the memory bank Local density scale, This is the smoothing constant.

6. The encrypted traffic anomaly detection method based on a pre-trained model and residual quantization codebook according to claim 1, characterized in that, In step (4), the final anomaly score is calculated by merging the two scores, specifically as follows: Quantization error fraction and local scaling memory fraction Normalization yields and ;Will and The final outlier score is obtained by weighted summation. : ； in, These are the weight parameters.

7. The encrypted traffic anomaly detection method based on a pre-trained model and residual quantization codebook according to claim 1, characterized in that, In step (5), pseudo-tags are generated based on the activation state of the codebook, specifically as follows: If the activated code exists in the normal traffic core set memory built during the training phase, this entry is valid. If a sample is found to be normal and has a normal confidence level, it is assigned a pseudo-label; if an unseen code entry is activated, it is assigned a pseudo-label representing a potential anomaly.

8. An encrypted traffic anomaly detection system based on a pre-trained model and residual quantization codebook, used to implement the encrypted traffic anomaly detection method according to any one of claims 1 to 7, characterized in that, include: Feature extraction module: Used to construct burst sequences from the raw encrypted traffic and extract continuous context embedding features using a pre-trained BERT model; Deep quantization compression module: Used to apply the multi-layer residual vector quantization network RQ-VAE to map continuous context embedding features to discrete code text semantic indices and generate quantized embedding features; Core set construction and management module: Used to record the code entry activated by normal traffic during the offline training phase and build a core set memory library representing the normal mode; Dual anomaly scoring module: Used to calculate quantization error score and local scaling memory score during inference, and combine them to determine whether the traffic is anomaly or a zero-day attack; Adaptive Evolution Module: During runtime, it generates pseudo-labels based on the activation state of the codebook, initiates comparative learning to update the parameters and core set memory of the BERT model and RQ-VAE.

Images