Method and system for detecting abnormal traffic of cloud-side collaborative dynamic memory compression network

Abstract

The invention discloses a cloud edge collaborative dynamic memory compression network abnormal traffic detection method and system, and relates to the technical field of network security. Traffic time sequence and semantic features are extracted through windowing preprocessing and a lightweight double-branch encoder at an edge side, and compression representation is realized by utilizing learnable low-rank mapping; a compression vector is decomposed into a public subspace and a private subspace, a cloud end adopts a security aggregation protocol to obtain global statistics, feature spaces of different edge nodes are unified through an orthogonal alignment method, and a global prototype is generated in an alignment space for cooperative judgment. A federated dynamic memory structure is constructed, abnormal statistical information of multiple nodes is fused to form a global dynamic memory library, a cloud side generates a model based on a global prototype and a memory construction condition to generate pseudo-abnormal features, and the pseudo-abnormal features are issued to edge nodes for open world incremental training. A closed-loop cooperative mechanism is formed through end-side rapid detection and cloud high-precision re-judgment, and low communication traffic, strong privacy protection and adaptive detection capability for unknown attacks are realized. According to the method, the anomaly detection performance and the adaptive capability in a distributed complex network environment can be effectively improved.

Description

Technical Field

[0001] This invention relates to the field of network security technology, specifically to a method and system for detecting abnormal traffic in cloud-edge collaborative dynamic memory compression networks. Background Technology

[0002] With the widespread deployment of technologies such as the Internet of Things (IoT), edge computing, and 5G / SDN, network traffic has become extremely dispersed, complex, and diverse—devices are heterogeneous, traffic sources are widely distributed, protocols are mixed, and packet header fields and timing characteristics are rich. At the same time, many edge devices / IoT devices are resource-constrained (computing, storage, bandwidth, energy consumption), and their requirements for privacy and data protection are becoming increasingly stringent. This makes traditional methods of aggregating all traffic or complete characteristics to the cloud for centralized detection difficult to meet real-world needs (bandwidth / privacy / scalability, etc.).

[0003] To adapt to this trend, the research community has made attempts. Existing patent CN117792726A proposes an abnormal traffic detection method based on a dynamic security baseline. This method constructs a long short-term memory network model for the number of traffic flows, packets, and IP addresses respectively, forming a multi-dimensional dynamic baseline. Anomaly alerts are then triggered based on the deviation of the prediction error from the threshold. Patent CN120768623A proposes a network intrusion detection method based on dynamic graph attention and contrastive learning. This method represents network traffic as a heterogeneous dynamic graph structure and utilizes a dynamic graph attention network (GATv2) and optimal transmission contrastive learning to align feature and structural distribution, thereby improving the detection performance against complex and unknown attacks. Patent CN114900343A, "An Abnormal Traffic Detection Method for IoT Devices Based on Clustered Federated Learning," addresses the problem of heterogeneous / non-independent identically distributed devices by clustering devices and aggregating / distributing models within each cluster.

[0004] However, these methods still have shortcomings: 1. They mostly focus on overall modeling based on statistics or graph structures. Although they can capture some temporal or topological features, their ability to jointly express packet-level fields, protocol semantics, and cross-field nonlinear dependencies in raw network traffic is limited. They are difficult to fully characterize fine-grained anomalous behavior in scenarios with mixed multi-protocol and high-dimensional heterogeneous traffic. 2. Some methods rely on centralized model training or high-complexity graph neural network computation. The model parameters are large, and the computation and communication overhead is high, resulting in high costs for deployment and long-term operation on resource-constrained edge devices. 3. In scenarios with multiple edge nodes working together, the data distribution collected by different nodes naturally has significant differences. It is difficult to directly align the feature space or judgment baseline learned locally. Existing methods mostly perform simple fusion at the level of model parameters or judgment results, lacking an effective alignment and unified modeling mechanism for feature semantic space offsets. 4. Most methods mainly make judgments based on short time windows or local structures, lacking a mechanism for continuous modeling of long-term behavior across time and nodes, making it difficult to detect low-frequency, latent, or multi-node collaborative evolutionary anomalous behaviors in a timely manner.

[0005] The aforementioned problems overlap in complex IoT and edge computing environments, which can easily lead to decreased detection accuracy, increased false alarm rate, and limited collaborative capabilities of anomaly detection systems when facing unknown attacks, covert attacks, or scenarios with changing distributions. Summary of the Invention

[0006] To address the shortcomings mentioned in the background section, the present invention aims to provide a method and system for detecting abnormal traffic in cloud-edge collaborative dynamic memory compression networks.

[0007] Firstly, the objective of this invention can be achieved through the following technical solution: a method for detecting abnormal traffic in a cloud-edge collaborative dynamic memory compression network, the method comprising the following steps: The raw network traffic of the computing node is obtained, and the raw network traffic of the computing node is preprocessed by windowing to obtain an intermediate representation vector. After feature extraction and compression of the intermediate representation vector, a compressed common vector is obtained. The compressed common vectors are securely aggregated to obtain global statistics. The global statistics are spatially aligned to obtain a global alignment matrix. Based on the global alignment matrix, the compressed common vectors are mapped to a unified semantic space to obtain aligned common vectors. The aligned common vectors are clustered to obtain a global prototype set. Based on the global prototype set and the pre-built global dynamic memory, a pseudo-anomaly feature vector is generated. The pseudo-anomaly feature vector is then distributed to the computing nodes to obtain the compressed common vector and local confidence of the samples to be judged within the computing nodes. Based on the global alignment matrix, the compressed common vector of the samples to be judged within the computing nodes is mapped to a unified semantic space to obtain the common vector of the aligned samples. The minimum prototype distance and minimum memory distance of the common vectors of the aligned samples are calculated based on the global alignment matrix. The comprehensive score of the sample to be judged is calculated based on the minimum prototype distance, minimum memory distance and confidence score. The comprehensive score of the sample to be judged is used to determine whether the sample to be judged is abnormal traffic.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the process of feature extraction and compression of the intermediate representation vector, comprising: The intermediate representation vector is compressed into an end-side compressed vector by a learnable low-rank mapping matrix, and a joint loss function is used to simultaneously constrain the detection loss, reconstruction loss and communication cost loss. The common subspace projection matrix and the private subspace projection matrix are used to decompose the end-side compressed vector into a compressed common vector.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the calculation process of compressing the intermediate representation vector into an end-side compressed vector using a learnable low-rank mapping matrix, comprising: Set a learnable low-rank mapping matrix on the end side. Map h to a compressed vector: Where z represents the end-side compression vector; Let represent a learnable low-rank mapping matrix with the number of rows equal to the compression dimension d; The term represents the bias term; h represents the end-side intermediate representation vector. The endpoints use joint loss during the training phase: in, This represents the local classification or anomaly detection loss, with the input being the true label y and the predicted value y. ; For reconstruction loss; Indicates communication cost loss; and These are the weighting coefficients; Introducing two sets of projection moments P and Q at the end makes z approximately decomposed as follows: in, The common subvectors are used for uploading to the cloud. Private subvectors are preserved locally. Here, P represents the projection matrix of the common subspace, and the output dimension is represented by k, i.e. The dimension of , where Q represents the private subspace projection matrix, and the output dimension is . ; Add the following orthogonal constraints to prevent the overlap of public and private subspaces: The projection matrix can be mapped to a low-rank matrix at the end. They are trained together to ensure effective separation of subspaces, and P and Q are constrained to be approximately orthogonal to reduce information overlap.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the uploading method of the compressed common vector includes one of three modes: statistical summary uploading, quantized and noisy uploading, and encrypted uploading, including: Statistical summary mode: Upload summary ,in Here, n is the number of samples in the current summary window; It is the mean vector; It is the variance vector; This represents the compressed vector of the i-th sample in the common subspace. Quantization and Noise Addition Mode: First, quantize for Add Laplace or Gaussian noise: in, Indicates the noise variance; Encrypted reporting mode: Report Homomorphic encryption is used to perform secure aggregation and alignment statistics in an encrypted domain in the cloud.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the global alignment matrix is ​​obtained by orthogonal Procrustes alignment or a secure multi-party statistical matrix construction method, as shown in the following formula: Where W represents the global alignment transformation matrix to be solved; This represents the i-th common eigenvector; This represents the i-th reference eigenvector; represents the regularization coefficient; N represents the total number of edge nodes.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: within the pre-built global dynamic memory, the global dynamic memory is organized into hash buckets or small clusters, and each unit records the mean, variance, count, and most recent update time of the embedding within that bucket; when a low-confidence or suspected anomalous sample appears locally, the sample's compressed vector is updated into the corresponding bucket and the statistics are adjusted: Where j represents the index of the bucket or cluster; Let z be the mean vector of bucket j, with the same dimensions as z. This indicates that bucket j already contains the sample count; represents the covariance matrix or diagonal variance vector of bucket j; z represents the compression vector assigned to the bucket. The global dynamic memory is updated by using an exponential decay fusion strategy on the cloud to fuse summaries from multiple nodes, specifically in the following form: in, It represents the global memory state, which contains several buckets, each with mean, variance, weight, and timestamp; This represents the local summary uploaded by the i-th node in period t, which can be used as a set of several buckets; This indicates the node weight, which can be set according to the amount of data in the node or its trust level. This represents the forgetting factor.

[0013] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the generation of pseudo-anomaly feature vectors based on the global prototype set and a pre-built global dynamic memory is processed based on a cloud-based conditional generation model, including: The generator for building conditional generative models in the cloud takes a conditional vector c as input, and the conditional vector contains the selected prototype. With memory statistics and random noise The output is the composite embedding. , as pseudo-anomaly feature vectors; When training the generator, a feature-matching term is introduced on top of cGAN to make the generator... With noise Generate and synthesize the embedding with the condition vector c. Discriminator Providing discrimination, the generator's objectives during training include adversarial loss and feature-matching constraints: in, Indicates hyperparameters; This represents the intermediate activation of the discriminator or pre-trained feature extractor; To represent diversity constraints, we can write: After training, the cloud allocates a set of samples to each edge node according to the strategy. The edge node adds the samples as unknown or synthetic anomaly labels to the local training set and uses an open-world strategy to fine-tune the local classifier in a short period of time. The cloud allocates resources to the i-th edge node. One synthetic sample: in, Indicates the global allocation coefficient; This represents the computing or storage capacity of edge node i; This indicates the upper and lower bounds.

[0014] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the minimum prototype distance is the minimum distance between the common vector of the aligned samples and the global prototype set, and the minimum memory distance is the minimum distance between the common vector of the aligned samples and the global dynamic memory. The cloud uses the updated global alignment matrix after receiving the data. Will Mapping to the global semantic space: Then calculate the values ​​related to the prototype set respectively. Minimum prototype distance: in, This represents the Euclidean distance between the sample and the closest global prototype; For the reported common vector Alignment matrix in the cloud The transformed representation; Represents the global prototype set The k-th prototype vector in; And minimum memory distance: in, This represents the Euclidean distance between a sample and the mean of the nearest bucket in global memory; This represents the mean vector of the j-th bucket in the global memory.

[0015] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the calculation process of deriving the comprehensive score of the sample to be judged based on the minimum prototype distance, the minimum memory distance, and the confidence level, including: Where s represents the final overall score; These are weighting coefficients used to adjust the relative importance of prototype distance, memory distance, and local confidence to the final score. This is a scale parameter that controls the rate at which distance decays; This represents the maximum confidence level of the local classifier output on the edge side.

[0016] Secondly, in order to achieve the above objectives, this invention discloses a cloud-edge collaborative dynamic memory compression network abnormal traffic detection system, comprising: The vector processing module is used to obtain the raw network traffic of the computing node, preprocess the raw network traffic of the computing node by windowing to obtain the intermediate representation vector, and extract and compress the intermediate representation vector to obtain the compressed common vector. The global clustering module is used to securely aggregate compressed common vectors to obtain global statistics, spatially align the global statistics to obtain a global alignment matrix, map the compressed common vectors to a unified semantic space based on the global alignment matrix to obtain aligned common vectors, and cluster the aligned common vectors to obtain a global prototype set. The vector mapping module is used to generate pseudo-anomaly feature vectors based on the global prototype set and the pre-built global dynamic memory library, distribute the pseudo-anomaly feature vectors to the computing nodes, obtain the compressed common vectors and local confidence of the samples to be judged within the computing nodes, and map the compressed common vectors of the samples to be judged within the computing nodes to the unified semantic space based on the global alignment matrix to obtain the common vectors of the aligned samples. The anomaly detection module is used to calculate the minimum prototype distance and minimum memory distance of the common vectors of aligned samples based on the global alignment matrix. Based on the minimum prototype distance, minimum memory distance and confidence level, it calculates the comprehensive score of the sample to be judged and determines whether the sample to be judged is abnormal traffic based on the comprehensive score of the sample to be judged.

[0017] The beneficial effects of this invention are: This invention achieves effective dimensionality reduction of high-dimensional traffic features through edge-side learnable low-rank compression, reducing the bandwidth burden on edge computing and communication. Through public / private subspace decomposition and differentiated reporting strategies, reported information can still support cloud-edge collaboration under privacy-controlled conditions. Secure aggregation and orthogonal alignment mechanisms solve the problem of inconsistent feature spaces among multiple edge devices, enabling the cloud to aggregate features from multiple devices in a unified space and generate a shareable global prototype. By constructing a federated dynamic memory structure to fuse anomaly statistics from different nodes and time periods, long-term modeling of low-frequency, latent, or cross-device anomaly behaviors is achieved. Furthermore, a conditional generation model based on prototypes and memory is used to construct pseudo-anomaly features, which are then distributed for edge-side open-world training, thereby improving the system's ability to identify unknown attacks. Finally, combined with a rapid judgment and high-precision re-judgment closed-loop mechanism for edge-cloud collaboration, the real-time performance, accuracy, and adaptability of the entire system are significantly improved, effectively enhancing anomaly detection performance and adaptability in distributed complex network environments. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the method framework of the present invention; Figure 3 This is a schematic diagram of the edge-side process of the method of the present invention; Figure 4 This is a schematic diagram of the cloud-side process of the method of the present invention; Figure 5 This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Example 1: like Figure 1 As shown, the cloud-edge collaborative dynamic memory compression network abnormal traffic detection method includes the following steps: S101: Obtain the raw network traffic of the computing node, perform windowing preprocessing on the raw network traffic of the computing node to obtain the intermediate representation vector, and extract and compress the intermediate representation vector to obtain the compressed common vector. The process of feature extraction and compression of intermediate representation vectors includes: The intermediate representation vector is compressed into an end-side compressed vector by a learnable low-rank mapping matrix, and a joint loss function is used to simultaneously constrain the detection loss, reconstruction loss and communication cost loss. The common subspace projection matrix and the private subspace projection matrix are used to decompose the end-side compressed vector into a compressed common vector.

[0021] End-side flow feature extraction and learnable low-rank compression 1. Data Preprocessing The raw network traffic data received by the edge device is a structured network connection record or traffic statistics record, which is uniformly represented at the edge as a set of data samples at the connection level or flow level. Its basic data structure can be represented as follows: in, This represents the i-th network connection or traffic record. Each record consists of multiple fields describing the statistical characteristics of a network communication activity. N represents the total number of network connection or traffic records in the sample set.

[0022] Edge devices receive the aforementioned network connection records through standardized structured data interfaces, including but not limited to: structured record interfaces based on traffic export or log transmission; connection-level feature interfaces based on local agents, virtual switches, or network function modules; or file-based batch data interfaces for offline experiments or model initialization.

[0023] To adapt to the lightweight edge model and characterize local temporal correlations, the edge device groups continuously received network connection records into fixed or adaptive windows. Each window contains L consecutive connection records, forming a traffic window. in, This represents the k-th traffic window, used to group continuously received network connection records by time or sequence. L represents the length of the network connection record j in the k-th traffic window, preferably 128, but not limited to this.

[0024] Within each window, the edge device performs field-level preprocessing operations on the fields, including: performing numerical mapping or One-Hot encoding on discrete fields such as protocol type, service type, and status flags; performing normalization or online standardization on continuous fields such as duration, number of bytes, and statistical types; and performing logarithmic transformation on some time-related or statistical fields to compress the dynamic range of the values.

[0025] 2. Feature Encoding The preprocessed multi-channel features are then input into a lightweight encoder for edge feature extraction. This encoder can be implemented by parallel temporal convolutional branches and bag-field convolutional branches, outputting an intermediate representation vector h (denoted as dimension D) for low-latency inference on the same device. Let X be the multi-channel input obtained after processing a window. Encoder Output intermediate representation vector: 3. Can learn low-rank compression To reduce upload overhead and facilitate privacy protection, a learnable low-rank mapping matrix is ​​set on the client side. Map h to a compressed vector: Where z represents the end-side compression vector; Let represent a learnable low-rank mapping matrix with the number of rows equal to the compression dimension d; represents the bias term (optional); h represents the end-side intermediate representation vector.

[0026] The mapping is learned during the training phase by jointly optimizing the detection loss (to maintain discriminative ability) and the constrained reconstruction / compression cost (to maintain information integrity and control traffic).

[0027] 4. Joint Training Objectives The endpoints use joint loss during the training phase: in, This represents the local classification / anomaly detection loss (e.g., cross-entropy or binary classification loss), with the input being the true label y and the predicted value. ; Reconstruction error (used to measure information loss when an optional decoder is present); Represents the communication cost term, used to encourage sparse / smaller norm compression vectors; and These are the weighting coefficients.

[0028] 5. Local determination When the local classifier outputs the maximum confidence level (i.e., the classifier's predicted probability vector) The maximum value is higher than the local threshold. At that time, the edge directly with The judgment is not reported to the cloud as output; when At that time, z will be handed over to the next step for public-private separation and differentiated reporting.

[0029] Differentiated private / public low-rank reporting While balancing collaborative effects and user privacy, the edge compression vector will be optimized. It is broken down into "shareable public parts" and "private parts that are kept local", and reported selectively based on bandwidth / privacy policies.

[0030] 1. Public-Private Separation Method The edge device introduces two sets of projection matrices P and Q on the end side, making z approximately decomposed as: in, The common subvectors are used for uploading to the cloud. Private subvectors are preserved locally. Here, P represents the projection matrix of the common subspace, and the output dimension is represented by k (i.e., ...). The dimension of Q is the private subspace projection matrix, and the output dimension is the dimension of Q. .

[0031] To avoid overlap between public and private subspaces, the following orthogonal constraints are added: To ensure effective separation of subspaces, the projection matrix can be mapped to a low-rank mapping at the end. They are trained together, and P and Q are constrained to be approximately orthogonal to reduce information overlap.

[0032] 2. Upload Strategy The upload strategy is selected based on the current bandwidth and privacy requirements: in scenarios with extremely limited bandwidth or high privacy requirements, only statistical summaries (such as local mean and variance) are uploaded; in normal scenarios, quantized and noisy summaries are uploaded. When collaboration is critical and computational overhead is acceptable, upload the encrypted version. (For example, using secure multi-party computation or homomorphic encryption).

[0033] (1) Statistical summary mode: Upload summary ,in Here, n is the number of samples in the current summary window; It is the mean vector; It is the variance vector; Let represent the compressed vector of the i-th sample in the common subspace. Its dimension is the same as that of the common subspace. It is a shareable feature representation obtained from the compressed vector at the end through the common projection matrix.

[0034] (2) Quantization and noise addition mode: First, quantize for (e.g., fixed codebook or uniform quantization), then add Laplace or Gaussian noise (differential privacy): in, This represents the noise variance and controls the level of privacy.

[0035] (3) Encrypted reporting mode (for high collaboration needs and computational capacity): Report (Using homomorphic encryption), secure aggregation and alignment statistics are performed in an encrypted domain in the cloud.

[0036] Through the above design, this step will generate reported data that can be used for cross-node statistics and aggregation without directly leaking the original traffic, thus enabling the cloud to perform the next step of alignment and aggregation within controlled privacy boundaries.

[0037] S102: Securely aggregate the compressed common vectors to obtain global statistics, spatially align the global statistics to obtain a global alignment matrix, map the compressed common vectors to a unified semantic space based on the global alignment matrix to obtain aligned common vectors, and cluster the aligned common vectors to obtain a global prototype set. like Figure 4 The cloud-side flowchart shown includes the following: Cloud-based security prototype alignment (SPA) and common feature aggregation Common vectors from different edge nodes, learned locally, and potentially with distribution bias Through secure aggregation and alignment, a unified global semantic space is formed, on which a global prototype set for judgment and generation is created.

[0038] This embodiment provides a specific implementation example comprising three edge nodes (denoted as N1, N2, and N3) and a cloud server S. It aims to improve the collaborative detection capability for abnormal network traffic by achieving global alignment of the common feature space through secure aggregation, while protecting the local privacy data of each node.

[0039] Each edge node holds heterogeneous local network traffic data: Node N1: Data mainly originates from regular office networks, including web browsing, email sending and receiving traffic, and a small number of port scanning attack samples. Node N2: Data mainly originates from video surveillance networks, including continuous video stream data, and a small number of denial-of-service (DDoS) attack samples. Node N3: Data mainly originates from IoT device networks, including a large amount of device heartbeat and status reporting traffic, and a small number of brute-force attack samples.

[0040] In a round of collaborative training initiated by cloud S, each node and the cloud perform the following steps: Local feature computation: Each node Ni, locally, is based on a common subspace vector of a batch of samples. Calculate its local statistical matrix ,matrix It contains the distribution information of the local public characteristics of the node, but it is no longer possible to deduce the original sample.

[0041] Secret share generation: Each node uses an additive secret sharing algorithm to generate the matrix. Encryption is performed. Specifically, node Ni randomly generates two random matrices of the same dimension. and And calculate the third matrix. ,matrix Divided into three secret shares .

[0042] Share distribution: Node Ni will secretly distribute shares. and Send the secret share to the other two edge nodes respectively. Send to cloud S. During this process, any single entity (including cloud S or other nodes) only holds meaningless random matrix shares and cannot know... Any information.

[0043] Cloud-based security aggregation: Cloud S collects shares from three nodes. Then, calculate the aggregation share. Simultaneously, cloud S requests the remaining shares held by each node. Ultimately, cloud S obtains three aggregated shares. .

[0044] Global statistics restoration: Cloud S (or a specified node) calculation Based on the homomorphic property of secret sharing, we have The cloud successfully obtained the sum of the local statistical matrices of all nodes without needing to know any individual nodes' local statistical matrices. .

[0045] The global statistical matrix obtained by aggregation in the cloud By solving the orthogonal problem, the global alignment transformation matrix is ​​calculated. The cloud will Distributed to each edge node, and applied on each node. Align it with its local public subspace.

[0046] To quantitatively evaluate the effectiveness of the secure aggregation method of this invention, this embodiment compares the anomaly detection performance under two training paradigms. Comparative Paradigm (Non-Privacy Collaborative Training): Each node uploads the common subspace vector to the cloud in plaintext for aggregation and alignment. This scheme represents the upper limit of collaborative learning performance but lacks privacy protection. Invention Method (Secure Aggregation Collaboration): Employs the aforementioned secure aggregation and spatial alignment scheme based on secret sharing, performing collaborative learning while protecting data privacy. The quantitative evaluation results on the NSL-KDD test set are shown in the table below: Table 1. Performance and Privacy Comparison of Aggregation Schemes Experimental data demonstrate that this invention achieves data privacy and security without significantly sacrificing detection performance, providing a practical and feasible technical approach for solving privacy-preserving collaborative detection in distributed network environments. These results fully validate the invention's good balance between practicality, security, and effectiveness.

[0047] 1. Secure Aggregation Framework The cloud first uses a secure aggregation protocol on the received reported data to obtain global statistics without exposing the plaintext of individual devices, thus acquiring all device data through secure aggregation. The common vector statistical matrix: in, This represents the set of common vectors reported by the i-th device (rows or columns arranged by sample).

[0048] 2. Embedded Space Alignment Training at different edge nodes There may be affine / linear shifts. To ensure that the common vectors of the nodes after this transformation are as consistent as possible with a reference semantic distribution, an alignment transformation is required. (Can be an orthogonal matrix or an affine matrix): Where W represents the alignment transformation matrix to be solved; This represents the i-th common eigenvector; This represents the i-th reference eigenvector; represents the regularization coefficient; N represents the total number of edge nodes.

[0049] When feasible, the Procrustes orthogonal solution is used (obtained by performing singular value decomposition (SVD) on the aggregation matrix). ), Where k represents the dimension of the common vector. The closed-form solution of orthogonal Procrustes can be obtained through singular value decomposition: in, The matrix represents the result of singular value decomposition; the input matrix is... It is the aggregated statistical matrix used to solve for global orthogonal alignment; Dimensions and common vectors Same (k×k).

[0050] Within a secure multi-party computation framework, the input for SVD can be constructed without exposing individual points by merging statistical matrices (such as the covariance or cross-correlation matrices of each node). 3. Prototype Generation After alignment, the cloud uses a clustering algorithm to generate a prototype set on the aligned vector set. (Where the number of clusters K can be chosen from experience or the validation set), each prototype It represents a typical traffic behavior or abnormal pattern, and includes timestamps and weight information for memory fusion and version management.

[0051] K prototypes are generated from the aligned global vector set using k-means clustering. : in, Let each prototype represent the set of samples belonging to the k-th class. Additional weights (Can be cluster size or confidence level) and timestamp .

[0052] 4. Prototype distribution The cloud will (or its alignment parameters) and necessary prototype subsets are periodically sent to the edge to ensure that the semantic representation of the edge gradually synchronizes with the cloud and supports more accurate local determination on the edge.

[0053] S103: Generate pseudo-anomaly feature vectors based on the global prototype set and the pre-built global dynamic memory library, distribute the pseudo-anomaly feature vectors to the computing nodes, obtain the compressed common vectors and local confidence of the samples to be judged within the computing nodes, and map the compressed common vectors of the samples to be judged within the computing nodes to the unified semantic space based on the global alignment matrix to obtain the common vectors of the aligned samples. The pre-built global dynamic memory is constructed as follows: Build and maintain a scalable global memory library that reflects suspicious behavior patterns observed in the system over time by merging local memory summaries from edge nodes, thereby enabling the detection of unknown or low-frequency attack patterns.

[0054] 1. Local memory structure and compression format Edge devices maintain a dynamic memory structure locally, using hash buckets or small clusters as units. Each unit records the mean, variance, count, and last update time of the embeddings within that bucket. When a low-confidence or suspected anomalous sample appears locally, the compressed vector of that sample is updated into the corresponding bucket, and the statistics are adjusted. Where j represents the index of the bucket or cluster; The mean vector of bucket j (with the same dimension as z) is used to represent the center of the bucket. This indicates the count (integer) of samples that bucket j already contains; The covariance matrix (or diagonal variance vector) of bucket j is used to measure the dispersion within the bucket; z represents the compression vector assigned to the bucket.

[0055] For communication efficiency, the edge periodically uploads a compressed summary of the bucket to the cloud. The uploaded summary includes the mean vector, diagonal variance term, and occurrence count of each bucket.

[0056] 2. Cloud-based global memory fusion The cloud uses an exponential decay fusion strategy to update the global memory for summaries from multiple nodes. That is, to merge in the following forms: in, Represents the global memory state (containing several buckets, each with mean, variance, weight, and timestamp). This represents the local summary (which can be a set of several buckets) uploaded by the i-th node in period t. Indicates the node weight (which can be set according to the node's data volume or trust level); This represents the forgetting factor (the closer to 1, the more historical the forgetting factor; the closer to 0, the more recent the forgetting factor).

[0057] To avoid memory bloat, the cloud employs a strategy of merging nearby buckets or discarding the oldest records to control the capacity of the global memory. It also records the weight and timestamp of each memory entry for subsequent generator conditionalization and cloud-based decision-making. Through this mechanism, this step compresses anomalous signs from different nodes and timescales into actionable statistical information, providing diverse conditional inputs for the generator and a long-term behavioral context for decision-making.

[0058] Federation generation enhancement and open-world training By leveraging global prototypes and global memory in the cloud, “pseudo-anomaly embeddings” representing potential but unseen anomaly patterns are conditionally generated, and these synthetic samples are distributed to the edge to support local open-world training, thereby improving the ability to identify unknown attacks and the overall robustness of the system.

[0059] 1. Generator and Discriminator A conditional generative model is built in the cloud, whose generator takes a condition vector c as input, which contains the selected prototype. (i.e., the prototype obtained in step 3 above) and corresponding memory statistics (e.g., the mean of a bucket in global memory). and random noise The output is the composite embedding. (Dimensions and Reporting) Consistent).

[0060] When training the generator, a feature-matching term is introduced on top of cGAN to make the generated samples match the real samples in the discriminator on high-order features, while diversity constraints prevent the generation pattern from collapsing.

[0061] Let the generator With noise Generate a composite embedding with the conditional vector c (composed of prototypes and memory statistics). Discriminator Provides discrimination. The generator's objectives during training include adversarial loss and feature-matching constraints. in, Indicates hyperparameters; This represents the intermediate activation of the discriminator or pre-trained feature extractor (used for feature-matching). The diversity constraint (e.g., a penalty based on the cosine similarity between samples within a batch) can be written as: 2. Strategy Deployment and Local Training After training, the cloud allocates a set of samples to each edge node according to the strategy. The edge node adds these synthetic samples as "unknown / synthetic anomaly" labels to the local training set and uses an open-world strategy to fine-tune the local classifier in a short period of time.

[0062] The cloud allocates resources to the i-th edge node. A number of synthetic samples (the proportion can be dynamically allocated according to the scarcity of node data / historical unknown rate): in, Indicates the global allocation coefficient; This represents the computing / storage capacity (or current load) of edge node i; the smaller the value, the more resources are allocated. This indicates the upper and lower bounds (e.g., 50–200).

[0063] This process enables each edge to gain defense experience against unseen anomalies through the "imagination" capabilities of the cloud, without sharing the original traffic data, thereby demonstrating a higher unknown sample identification rate in collaborative judgment.

[0064] S104: Calculate the minimum prototype distance and minimum memory distance of the common vectors of the aligned samples based on the global alignment matrix. Calculate the comprehensive score of the sample to be judged based on the minimum prototype distance, minimum memory distance, and confidence level. Determine whether the sample to be judged is abnormal traffic based on the comprehensive score of the sample to be judged.

[0065] The minimum prototype distance is the minimum distance between the common vector of the aligned samples and the global prototype set, and the minimum memory distance is the minimum distance between the common vector of the aligned samples and the global dynamic memory. The calculation process for the comprehensive score of the sample to be judged, based on the minimum prototype distance, minimum memory distance, and confidence level, includes: A collaborative closed loop is achieved between real-time rapid judgment at the edge and high-precision re-judgment in the cloud, and the judgment results and learning outputs in the cloud are fed back to the edge in real time to complete online model and threshold updates, thereby ensuring the system's adaptability to emerging threats.

[0066] 1. Local quick judgment The edge is pressed against the new window to obtain the compressed vector z and a local fast determination is performed; when the local confidence level is... Below the local threshold At the time, edge upload (Subject to privacy / bandwidth policies) to the cloud.

[0067] 2. Cloud-based secondary judgment The cloud uses the updated alignment matrix after receiving the data. Will Mapping to the global semantic space: Then calculate the values ​​related to the prototype set respectively. Minimum prototype distance: in, This represents the Euclidean distance between a sample and the closest global prototype, used to measure the similarity between a sample and a known pattern. For the reported common vector Alignment matrix in the cloud The transformed representation; Represents the global prototype set The k-th prototype vector (with the same dimension) ).

[0068] And the minimum memory distance for global memory: in, This represents the Euclidean distance between a sample and the mean of the nearest bucket in global memory; This represents the mean vector of the j-th bucket in the global memory.

[0069] The cloud uses these distances and edge-reported local confidence levels As input, a comprehensive scoring function is used (example format: Where s represents the final overall score; , which is a weighting coefficient (non-negative), used to adjust the relative importance of prototype distance, memory distance, and local confidence to the final score; As a scale parameter, it controls the rate of distance decay: if the typical distance is large, It should be increased accordingly; This represents the maximum confidence level of the local classifier output on the edge side.

[0070] 3. The judgment result and update information are returned to the edge node. Based on score s and cloud threshold and unknown intervals The comparison will be processed by the cloud, and the result will be returned to the edge, where the cloud threshold will be set. It is not a fixed preset, but rather an adaptive result obtained by the cloud based on historical judgment data and statistical results of labeled samples. During the model training and operation phases, the cloud continuously performs statistical analysis on the comprehensive judgment score s corresponding to known normal samples, known abnormal samples, and manually verified samples to form a score distribution set. The cloud then determines the judgment threshold according to preset risk control strategies or statistical rules. It can be set to the high quantile of the abnormal sample score distribution to ensure a high abnormal recall rate; and It can be adaptively determined based on the overlapping or interval regions of the score distributions of normal and abnormal samples, and is used to characterize the decision interval with high model uncertainty. Decision rule: If → Detect an anomaly; if If the sample is deemed "unknown," it is marked as such and triggers manual review or further sampling. If it is determined to be an anomaly, the edge will write the sample into local memory and trigger local fine-tuning or threshold adjustment; if it is determined to be unknown, it will trigger manual review or increase sampling for subsequent annotation.

[0071] This closed loop ensures the complementarity of information and capabilities between the edge and the cloud: the edge completes common judgments with low latency, while the cloud judges difficult samples from a broader historical global perspective and feeds back the learned results to the edge, thereby improving the overall accuracy of the system and the response speed to unknown attacks.

[0072] Specifically, the present invention will be further illustrated below through embodiments: To verify the effectiveness of the proposed "Abnormal Traffic Detection Method for Dynamic Memory Compression Networks Based on Cloud-Edge Collaboration," a full-process experiment was conducted in a cloud-edge collaborative federated environment. This included verification of the quality of low-rank compression features at the edge, verification of the public-private subspace separation effect, verification of cloud-based security prototype alignment (SPA) and clustering accuracy, verification of the long-term abnormal behavior modeling capability brought by federated dynamic memory (FedDyMem), and comprehensive performance testing of the federated generation enhancement and edge-cloud collaborative closed-loop judgment mechanism. The following section presents and analyzes the results of the experiments step by step.

[0073] I. Experimental Environment Configuration 1. Hardware environment: Cloud server: A deep learning server based on Ubuntu 22.04.4 LTS, equipped with an NVIDIA GeForce RTX 4080 GPU, an Intel Core i9-14900KF CPU, and 32 GB of system memory.

[0074] Local workstation: Used for code development, remote server connections, and visualization analysis of experimental results.

[0075] 2. Software environment: Operating system: Ubuntu 22.04.4 LTS (server); Windows 11 (local).

[0076] Deep learning frameworks: PyTorch 1.12.1, Python 3.10.14.

[0077] Key dependencies: data processing and machine learning libraries such as numpy, pandas, and scikit-learn, as well as the Python standard library (os, warnings, random, etc.).

[0078] II. Experimental Dataset 1. Training dataset: The NSL-KDD training set (KDDTrain+.txt) serves as the primary training data source for the federated learning edge; it contains network connection record samples and their attack category annotations. The data preprocessing workflow involves One-Hot encoding of the three discrete fields (protocol_type, service, and flag), Min-Max normalization for the remaining numerical features, and mapping them to five label classes (including normal and multiple attack classes) based on the attack type. The final feature dimension is dynamically determined based on the One-Hot expansion result.

[0079] Federation partitioning method: The training set is divided into 3 parts according to the sample order and distributed to 3 clients for local training to simulate the data holding scenario of non-independent identically distributed (Non-IID).

[0080] 2. Test dataset: The NSL-KDD test set (KDDTest+.txt) serves as the final evaluation data source. It employs the same feature engineering process as the training set (One-Hot encoding + Min-Max normalization + 5-class label mapping) to output classification metrics and evaluate the model's detection and classification capabilities on unseen samples.

[0081] III. Data Preprocessing Use the attack type cluster mapping defined in the patent (one attack class may correspond to multiple original attack names): attack_map = { 'normal': 0, 'neptune': 1, 'teardrop': 1, 'pod': 1, ... 'warezclient': 2, 'guess_passwd': 2, ... 'ipsweep': 3, 'nmap': 3, 'portsweep': 3, ... 'rootkit': 4, 'perl': 4, 'loadmodule': 4 The dozens of attacks were unified into 5 categories, making the subsequent Federated + prototype clustering more stable.

[0082] implement: The `combined = pd.get_dummies(combined, columns=['protocol_type', 'service', 'flag'])` method embeds discrete fields into high-dimensional sparse features and normalizes them before using them as model input.

[0083] Using MinMax normalization ensures that all dimensions fall within [0,1], thus adapting to neural network training.

[0084] We perform class weight calculations to address the extreme class imbalance in NSL-KDD, ensuring that minority classes are not ignored during training.

[0085] IV. Lightweight End-Side Model Training Through structure: Linear → BN → ReLU → Dropout Linear → BN → ReLU The input dimension is dynamically set to the preprocessed dimension (tens to over one hundred dimensions). This enables the extraction of time-series / packet-level comprehensive representations.

[0086] Perform learnable low-rank compression: In the experiment, the compression dimension was set to compress_dim=32.

[0087] Perform public / private projection: , In the experiment, the client only uploaded [something] per round. .

[0088] Differentiated reporting: (1) Raw upload: Number of bytes sent (single report): Where N is the number of samples (the number of samples reported). For public_dim, 4 represents the number of float32 bytes.

[0089] (2) Statistical summary mode: Upload summary ,in In the experiment, the reporting vector s (length k) is set as follows: Traffic volume: (3) Quantization + Noise Reporting: Linear quantization (uniform): Mapping floating-point vectors to integers .

[0090] Given a vector z, find the minimum / maximum value for each dimension (global or local extrema can be used): Inverse quantization: Add Gaussian noise (to simulate differential privacy, not strict dynamic programming): Add noise before reporting or after dequantization: in The noise standard deviation (taken in the experiment) ).

[0091] Communication volume (uint8): If b = 8, then each element is 1 byte.

[0092] V. Construction and Training of Joint Loss Experimental setup: Classification loss (with class weights) improves the ability to identify imbalanced data. Reconstruction loss (preserving original information): Communication loss reduces the energy required to upload data, which helps improve compression quality. Orthogonal constraints ensure that public and private spaces do not interfere with each other, reducing privacy leaks. Total loss: The specific parameters used in the experiment are as follows: lambda_rec = 0.5 lambda_comm = 0.01 lambda_orth = 0.1 V. Cloud Alignment and Clustering Training at different edge nodes There may be affine / linear shifts. To ensure that the common vectors of all nodes after this transformation are as consistent as possible with a reference semantic distribution, an alignment transformation is required. (Can be an orthogonal matrix or an affine matrix): In the experiment, the following command was executed: global_mean = torch.mean(all_uploads) aligned = z_batch - local_mean + global_mean local_mean = the mean of the features uploaded by each client. global_mean = the overall mean of features uploaded by all clients K prototypes are generated from the aligned global vector set using k-means clustering. : The experiment was conducted as follows: kmeans = KMeans(n_clusters=num_classes) 2) kmeans.fit(aligned_features) self.global_prototypes = cluster_centers A global "cluster center" is created as a prototype, which will then be used as a "conditional vector" in subsequent GANs.

[0093] VI. Federated Dynamic Memory Compression Edge devices maintain a dynamic memory structure locally, using hash buckets or small clusters as units. Each unit records the mean, variance, count, and last update time of the embeddings within that bucket. When a low-confidence or suspected anomalous sample appears locally, the compressed vector of that sample is updated into the corresponding bucket, and the statistics are adjusted. The cloud uses an exponential decay fusion strategy to update the global memory for summaries from multiple nodes. That is, to merge in the following forms: In the experiment, the following was executed: cluster_centers = kmeans.cluster_centers_ self.global_prototypes = cluster_centers Each round of clustering in the cloud is based on the latest uploaded data, and the updated cluster centers (global_prototypes) are the "compressed memory".

[0094] VII. Federation Generation Enhancement and Open World Training Generator definition: cond_dim = prototype_dim input_dim = public_dim + prototype_dim G = Generator(input_dim, public_dim) D = Discriminator(public_dim, condition_dim) Input: Random noise (z) + conditional prototype (c), Output: Pseudo-common feature (\tilde{z}) The discriminator distinguishes between real and fake features (real uploaded features vs. GAN generated features), and the experiment is trained for multiple epochs: cloud.train_gan(epochs=10).

[0095] GAN training process: # Training the discriminator loss_D_real = D(x, c) loss_D_fake = D(G(z,c), c) # Training Generator loss_G = BCE(D(G(z,c)), 1) 100 pseudo-feature embeddings are distributed to the edge: pseudo = cloud.generate_pseudo_anomalies(num_samples=100) print("GAN training complete, 100 pseudo-features distributed").

[0096] VIII. Analysis of Experimental Results The method of this invention underwent 15 rounds of federated training on the NSL-KDD traffic dataset. During the experiment, balanced class weights, dynamic learning rate scheduling, public / private subspace decomposition, cloud prototype alignment, and federated generation enhancement mechanisms were employed. The loss continuously decreased during training, indicating that the model has good convergence under cross-client non-independent identically distributed (Non-IID) conditions.

[0097] The final test set results are shown in Table 1 below: Table 2 Test Set Results As shown in Table 1, the overall accuracy rate is 83.03%, and the weighted F1 score reaches 0.8195, indicating that the model performs well in terms of comprehensive classification ability.

[0098] From a category perspective, the normal class (label 0) and the DoS class (label 1) achieved high precision and recall, with the DoS class achieving a recall of 0.9761. This is because this type of attack has obvious common structural characteristics, and after end-side encoding and alignment with the cloud prototype, a stable intra-class distribution can be formed.

[0099] The Probe class (label 3) achieved a high recall rate (0.8879) but a low precision rate (0.5739). This is mainly because the Probe class and some anomaly classes still overlap in the feature space, and it is still difficult to form completely separated feature boundaries in the cloud under the conditions of using prototype alignment and federated dynamic memory compression strategies.

[0100] For R2L / U2R classes (label 2 and label 4) with extremely small sample sizes, the model still maintained a certain level of recognition ability. In particular, for label 4, with only 37 samples, the recall rate reached 0.6405, indicating that the common subspace and prototype enhancement mechanism can enhance inter-class separability under extremely small sample conditions.

[0101] Example 2: To achieve the above objective, such as Figure 5 As shown, based on Embodiment 1, this invention discloses a cloud-edge collaborative dynamic memory compression network abnormal traffic detection system, comprising: The vector processing module 11 is used to obtain the original network traffic of the computing node, perform windowing preprocessing on the original network traffic of the computing node to obtain an intermediate representation vector, and extract and compress the intermediate representation vector to obtain a compressed common vector. The global clustering module 12 is used to securely aggregate compressed common vectors to obtain global statistics, spatially align the global statistics to obtain a global alignment matrix, map the compressed common vectors to a unified semantic space based on the global alignment matrix to obtain aligned common vectors, and cluster the aligned common vectors to obtain a global prototype set. The vector mapping module 13 is used to generate pseudo-anomaly feature vectors based on the global prototype set and the pre-built global dynamic memory, distribute the pseudo-anomaly feature vectors to the computing nodes, obtain the compressed common vectors and local confidence of the samples to be judged within the computing nodes, and map the compressed common vectors of the samples to be judged within the computing nodes to the unified semantic space based on the global alignment matrix to obtain the common vectors of the aligned samples. The anomaly detection module 14 is used to calculate the minimum prototype distance and minimum memory distance of the common vectors of the aligned samples based on the global alignment matrix, calculate the comprehensive score of the sample to be judged based on the minimum prototype distance, minimum memory distance and confidence, and determine whether the sample to be judged is abnormal traffic based on the comprehensive score of the sample to be judged.

[0102] Based on the same inventive concept, this invention also provides a computer device, comprising: one or more processors, and a memory for storing one or more computer programs; the programs include program instructions, and the processor executes the program instructions stored in the memory. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, used to implement one or more instructions, specifically for loading and executing one or more instructions stored in a computer storage medium to implement the above-described method.

[0103] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, performs the above-described method. This storage medium can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0104] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0105] The foregoing has shown and described the basic principles, main features, and advantages of this disclosure. Those skilled in the art should understand that this disclosure is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of this disclosure. Various changes and modifications can be made to this disclosure without departing from its spirit and scope, and all such changes and modifications fall within the scope of this disclosure as claimed.

Claims

1. A method for detecting abnormal traffic in a cloud-edge collaborative dynamic memory compression network, characterized in that, The method includes the following steps: The raw network traffic of the computing node is obtained, and the raw network traffic of the computing node is preprocessed by windowing to obtain an intermediate representation vector. After feature extraction and compression of the intermediate representation vector, a compressed common vector is obtained. The compressed common vectors are securely aggregated to obtain global statistics. The global statistics are spatially aligned to obtain a global alignment matrix. Based on the global alignment matrix, the compressed common vectors are mapped to a unified semantic space to obtain aligned common vectors. The aligned common vectors are clustered to obtain a global prototype set. Based on the global prototype set and the pre-built global dynamic memory, a pseudo-anomaly feature vector is generated. The pseudo-anomaly feature vector is then distributed to the computing nodes to obtain the compressed common vector and local confidence of the samples to be judged within the computing nodes. Based on the global alignment matrix, the compressed common vector of the samples to be judged within the computing nodes is mapped to a unified semantic space to obtain the common vector of the aligned samples. The minimum prototype distance and minimum memory distance of the common vectors of the aligned samples are calculated based on the global alignment matrix. The comprehensive score of the sample to be judged is calculated based on the minimum prototype distance, minimum memory distance and confidence score. The comprehensive score of the sample to be judged is used to determine whether the sample to be judged is abnormal traffic.

2. The abnormal traffic detection method for cloud-edge collaborative dynamic memory compression network according to claim 1, characterized in that, The process of feature extraction and compression of the intermediate representation vector includes: The intermediate representation vector is compressed into an end-side compressed vector by a learnable low-rank mapping matrix, and a joint loss function is used to simultaneously constrain the detection loss, reconstruction loss and communication cost loss. The common subspace projection matrix and the private subspace projection matrix are used to decompose the end-side compressed vector into a compressed common vector.

3. The abnormal traffic detection method for cloud-edge collaborative dynamic memory compression network according to claim 2, characterized in that, The computational process of compressing the intermediate representation vector into an end-side compressed vector using a learnable low-rank mapping matrix includes: Set a learnable low-rank mapping matrix on the end side. Map h to a compressed vector: Where z represents the end-side compression vector; Let represent a learnable low-rank mapping matrix with the number of rows equal to the compression dimension d; The term represents the bias term; h represents the end-side intermediate representation vector. The endpoints use joint loss during the training phase: in, This represents the local classification or anomaly detection loss, with the input being the true label y and the predicted value y. ; For reconstruction loss; Indicates communication cost loss; and These are the weighting coefficients; Introducing two sets of projection matrices P and Q at the end makes z approximately decomposed as follows: in, The common subvectors are used for uploading to the cloud. Private subvectors are preserved locally. Here, P represents the projection matrix of the common subspace, and the output dimension is represented by k, i.e. The dimension of , where Q represents the private subspace projection matrix, and the output dimension is . ; Add the following orthogonal constraints to prevent the overlap of public and private subspaces: The projection matrix can be mapped to a low-rank matrix at the end. They are trained together to ensure effective separation of subspaces, and P and Q are constrained to be approximately orthogonal to reduce information overlap.

4. The cloud-edge collaborative dynamic memory compression network abnormal traffic detection method according to claim 3, characterized in that, The upload method for the compressed common vector includes one of three modes: statistical summary upload, quantized and noisy upload, and encrypted upload. Statistical summary mode: Upload summary ,in Here, n is the number of samples in the current summary window; It is the mean vector; It is the variance vector; This represents the compressed vector of the i-th sample in the common subspace. Quantization and Noise Addition Mode: First, quantize... for Add Laplace or Gaussian noise: in, Indicates the noise variance; Encrypted reporting mode: Report Homomorphic encryption is used to perform secure aggregation and alignment statistics in an encrypted domain in the cloud.

5. The abnormal traffic detection method for cloud-edge collaborative dynamic memory compression network according to claim 1, characterized in that, The global alignment matrix is ​​obtained by constructing an orthogonal Procrustes alignment or a secure multi-square statistical matrix, as shown in the following formula: Where W represents the global alignment transformation matrix to be solved; This represents the i-th common eigenvector; This represents the i-th reference eigenvector; represents the regularization coefficient; N represents the total number of edge nodes.

6. The abnormal traffic detection method for cloud-edge collaborative dynamic memory compression network according to claim 1, characterized in that, Within the pre-built global dynamic memory, the global dynamic memory is organized into hash buckets or small clusters. Each unit records the mean, variance, count, and most recent update time of the embedding within that bucket. When a low-confidence or suspected anomalous sample appears locally, the sample's compressed vector is updated into the corresponding bucket, and the statistics are adjusted. Where j represents the index of the bucket or cluster; Let z be the mean vector of bucket j, with the same dimensions as z. This indicates that bucket j already contains the sample count; represents the covariance matrix or diagonal variance vector of bucket j; z represents the compression vector assigned to the bucket. The global dynamic memory is updated by using an exponential decay fusion strategy to fuse summaries from multiple nodes in the cloud, specifically in the following form: in, It represents the global memory state, which contains several buckets, each with mean, variance, weight, and timestamp; This represents the local summary uploaded by the i-th node in period t, which can be used as a set of several buckets; This indicates the node weight, which can be set according to the amount of data in the node or its trust level. This represents the forgetting factor.

7. The abnormal traffic detection method for cloud-edge collaborative dynamic memory compression network according to claim 1, characterized in that, The generation of pseudo-anomaly feature vectors based on a global prototype set and a pre-built global dynamic memory is processed using a cloud-based conditional generation model, including: The generator for building conditional generative models in the cloud takes a conditional vector c as input, and the conditional vector contains the selected prototype. With memory statistics and random noise The output is the composite embedding. , as pseudo-anomaly feature vector; When training the generator, a feature-matching term is introduced on top of cGAN to make the generator... With noise Generate and synthesize the embedding with the condition vector c. Discriminator Providing discrimination, the generator's objectives during training include adversarial loss and feature-matching constraints: in, Indicates hyperparameters; This represents the intermediate activation of the discriminator or pre-trained feature extractor. To represent diversity constraints, we can write: After training, the cloud allocates a set of samples to each edge node according to the strategy. The edge node adds the samples as unknown or synthetic anomaly labels to the local training set and uses an open-world strategy to fine-tune the local classifier in a short period of time. The cloud allocates resources to the i-th edge node. One synthetic sample: in, Indicates the global allocation coefficient; This represents the computing or storage capacity of edge node i; This indicates the upper and lower bounds.

8. The abnormal traffic detection method for cloud-edge collaborative dynamic memory compression network according to claim 1, characterized in that, The minimum prototype distance is the minimum distance between the common vector of the aligned samples and the global prototype set, and the minimum memory distance is the minimum distance between the common vector of the aligned samples and the global dynamic memory. The cloud uses the updated global alignment matrix after receiving the data. Will Mapping to the global semantic space: Then calculate the values ​​related to the prototype set respectively. Minimum prototype distance: in, This represents the Euclidean distance between the sample and the closest global prototype; For the reported common vector Alignment matrix in the cloud The transformed representation; Represents the global prototype set The k-th prototype vector in; And minimum memory distance: in, This represents the Euclidean distance between a sample and the mean of the nearest bucket in global memory; This represents the mean vector of the j-th bucket in the global memory.

9. The abnormal traffic detection method for cloud-edge collaborative dynamic memory compression network according to claim 1, characterized in that, The calculation process for obtaining the comprehensive score of the sample to be judged based on the minimum prototype distance, minimum memory distance, and confidence level includes: Where s represents the final overall score; These are weighting coefficients used to adjust the relative importance of prototype distance, memory distance, and local confidence to the final score. This is a scale parameter that controls the rate at which distance decays; This represents the maximum confidence level of the local classifier output on the edge side.

10. A cloud-edge collaborative dynamic memory compression network abnormal traffic detection system, employing the cloud-edge collaborative dynamic memory compression network abnormal traffic detection method as described in any one of claims 1 to 9, characterized in that, include: The vector processing module is used to obtain the raw network traffic of the computing node, preprocess the raw network traffic of the computing node by windowing to obtain the intermediate representation vector, and extract and compress the intermediate representation vector to obtain the compressed common vector. The global clustering module is used to securely aggregate compressed common vectors to obtain global statistics, spatially align the global statistics to obtain a global alignment matrix, map the compressed common vectors to a unified semantic space based on the global alignment matrix to obtain aligned common vectors, and cluster the aligned common vectors to obtain a global prototype set. The vector mapping module is used to generate pseudo-anomaly feature vectors based on the global prototype set and the pre-built global dynamic memory library, distribute the pseudo-anomaly feature vectors to the computing nodes, obtain the compressed common vectors and local confidence of the samples to be judged within the computing nodes, and map the compressed common vectors of the samples to be judged within the computing nodes to the unified semantic space based on the global alignment matrix to obtain the common vectors of the aligned samples. The anomaly detection module is used to calculate the minimum prototype distance and minimum memory distance of the common vectors of aligned samples based on the global alignment matrix. Based on the minimum prototype distance, minimum memory distance and confidence level, it calculates the comprehensive score of the sample to be judged and determines whether the sample to be judged is abnormal traffic based on the comprehensive score of the sample to be judged.

Images