Artificial intelligence-based network detection method, system, device and medium

Abstract

The application relates to an artificial intelligence-based network traffic anomaly detection method, system, device and medium, which comprises the following steps: collecting network traffic data and extracting multi-dimensional traffic features, matching business type labels to generate traffic feature vector sequences carrying business semantics; training a global baseline model and a plurality of business semantic baseline models according to the traffic feature vector sequences, and constructing a long-term memory buffer; inputting real-time traffic into a double-branch model for reconstruction, calculating a comprehensive anomaly score fused with business context confidence, calculating a baseline health index and a pollution risk coefficient according to the anomaly score and the reconstruction error, and classifying and marking samples; and performing branch-differentiated incremental updating based on the marking result, wherein the updating of the business semantic baseline model is limited by a pollution risk threshold and is combined with pure samples in the memory buffer for playback constraint. The application can effectively distinguish between business legal mutations and attack traffic, and has baseline pollution detection and defense capabilities.

Description

Technical Field

[0001] This invention relates to the field of network data processing and intelligent analysis technology, specifically to a network detection method, system, device, and medium based on artificial intelligence. Background Technology

[0002] In network security management practice, traffic anomaly detection based on dynamic baselines is an important proactive defense technology. The core idea of ​​this approach is to learn from traffic data under normal network conditions to construct a security baseline that represents "normal" behavior patterns. Real-time traffic is then compared to this baseline to identify suspicious behaviors that deviate from the normal pattern. Compared to traditional detection methods that rely on static thresholds or manually set rules, dynamic baseline methods can automatically adjust detection standards according to changes in the network environment, alleviating to some extent the problem that static rules are difficult to adapt to dynamic traffic changes. With the development of deep learning technology, dynamic baseline construction schemes based on temporal modeling methods such as Long Short-Term Memory (LSTM) networks have been proposed. By modeling the temporal dependencies of traffic data, a more refined characterization of network behavior patterns is achieved.

[0003] However, existing deep learning-based dynamic baseline detection methods still face several unresolved technical shortcomings. First, most methods rely solely on purely data-driven temporal modeling, lacking the ability to perceive the semantics of business context. When legitimate traffic mutations occur in the network due to business operations (such as promotional activities, version releases, CDN strategy adjustments, etc.), these mutations are highly similar to malicious behaviors such as DDoS attacks and port scanning at the traffic characteristic level. Existing methods struggle to effectively distinguish between "business-driven legitimate mutations" and "attack-driven malicious mutations," leading to a large number of false positives. Second, to adapt to the continuous evolution of network behavior, existing methods typically employ online incremental learning mechanisms to continuously update the model. However, when the network suffers from continuous low-intensity scanning attacks or slow penetration attacks, attack traffic will be mixed into the training data at extremely low intensity over a long period. Because the incremental learning mechanism indiscriminately includes recent traffic in model updates, the baseline gradually "adapts" to these abnormal patterns, gradually "normalizing" the attack behavior, leading to irreversible structural degradation of detection capabilities. For this baseline "memory contamination" phenomenon, existing technologies lack effective detection, isolation, and remediation mechanisms, as well as quantitative methods for characterizing the degree of contamination. Summary of the Invention

[0004] Based on this, the purpose of the present invention is to provide an artificial intelligence-based network detection method, system, device and medium that can perceive business semantic context and has baseline contamination detection and defense capabilities.

[0005] The objective of this invention is achieved through the following solution:

[0006] In a first aspect, the present invention provides a network detection method based on artificial intelligence, comprising the following steps:

[0007] S1: Collect network traffic data, perform session reassembly on the network traffic data to extract multi-dimensional traffic features, and match the corresponding service type labels based on the multi-dimensional traffic features to generate a sequence of traffic feature vectors carrying service type labels.

[0008] S2: Based on the traffic feature vector sequence carrying business type labels, train a global baseline model that does not distinguish business type and multiple business semantic baseline models corresponding to each business type label. Select the preset number of samples with the smallest reconstruction error from the traffic feature vectors corresponding to each business type label and store them in the long-term memory buffer of the corresponding business type label to generate a two-branch dynamic baseline model containing a global baseline model, multiple business semantic baseline models and a long-term memory buffer.

[0009] S3: Input the acquired real-time traffic feature vectors into the global baseline model and the business semantic baseline model corresponding to the real-time traffic feature vectors respectively for reconstruction, obtain the global reconstruction error and the business semantic reconstruction error, and perform weighted fusion based on the ratios of the global reconstruction error and the business semantic reconstruction error to the upper bound of the dynamic confidence interval of the global baseline model and the business semantic baseline model respectively, to calculate the comprehensive anomaly score.

[0010] S4: Based on the comprehensive anomaly score, business semantic reconstruction error, and upper bound of the dynamic confidence interval, calculate the baseline health index and pollution risk coefficient of the current business type label, make incremental update decisions on the real-time traffic feature vector based on the baseline health index and pollution risk coefficient, and generate labeling results containing safe and normal samples, high-risk pollution samples, and abnormal samples.

[0011] S5: Based on the labeling results, incrementally update the global baseline model and the business semantic baseline model. For real-time traffic feature vectors labeled as safe and normal samples, if the business semantic reconstruction error does not exceed the upper bound of the dynamic confidence interval and the pollution risk coefficient is lower than the preset pollution risk threshold, the real-time traffic feature vectors are mixed with historical samples sampled from the long-term memory buffer and used to update the corresponding business semantic baseline model. For real-time traffic feature vectors labeled as abnormal samples, skip this incremental update and update the model parameters of the global baseline model and the business semantic baseline model and the corresponding upper bound of the dynamic confidence interval.

[0012] Secondly, the present invention provides an artificial intelligence-based network detection system, which is configured with the following modules:

[0013] The traffic feature construction module is used to collect network traffic data, extract multi-dimensional traffic features after session reassembly of the network traffic data, and match the corresponding service type labels based on the multi-dimensional traffic features to generate a traffic feature vector sequence carrying service type labels.

[0014] The dual-branch model construction module is used to train a global baseline model that does not distinguish between business types and multiple business semantic baseline models corresponding to each business type label based on the traffic feature vector sequence carrying business type labels. It also selects a preset number of samples with the smallest reconstruction error from the traffic feature vectors corresponding to each business type label and stores them in the long-term memory buffer of the corresponding business type label to generate a dual-branch dynamic baseline model containing a global baseline model, multiple business semantic baseline models and a long-term memory buffer.

[0015] The comprehensive anomaly scoring module is used to input the acquired real-time traffic feature vectors into the global baseline model and the business semantic baseline model corresponding to the real-time traffic feature vectors respectively for reconstruction, to obtain the global reconstruction error and the business semantic reconstruction error, and to calculate the comprehensive anomaly score by weighting and fusing the ratios of the global reconstruction error and the business semantic reconstruction error with the upper bound of the dynamic confidence interval of the global baseline model and the business semantic baseline model respectively.

[0016] The sample classification and labeling module is used to calculate the baseline health index and pollution risk coefficient of the current business type label based on the comprehensive anomaly score, business semantic reconstruction error and dynamic confidence interval upper bound. Based on the baseline health index and pollution risk coefficient, it makes incremental update decisions on the real-time traffic feature vector and generates labeling results including safe and normal samples, high-risk pollution samples and abnormal samples.

[0017] The model incremental update module is used to incrementally update the global baseline model and the business semantic baseline model based on the labeling results. For real-time traffic feature vectors labeled as safe and normal samples, if the business semantic reconstruction error does not exceed the upper bound of the dynamic confidence interval and the pollution risk coefficient is lower than the preset pollution risk threshold, the real-time traffic feature vectors are mixed with historical samples sampled from the long-term memory buffer and used to update the corresponding business semantic baseline model. For real-time traffic feature vectors labeled as abnormal samples, the incremental update is skipped, and the model parameters of the global baseline model and the business semantic baseline model and the corresponding upper bound of the dynamic confidence interval are updated.

[0018] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any of the above-mentioned artificial intelligence-based network detection methods.

[0019] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the aforementioned artificial intelligence-based network detection methods.

[0020] In summary, the AI-based network detection method provided in this application constructs a dual-branch dynamic baseline architecture consisting of a global baseline model and multiple business semantic baseline models. By combining this architecture with business type tags to extract semantic features from traffic, it fundamentally decouples legitimate traffic mutations caused by business operations from malicious attack traffic at the semantic level. This overcomes the technical predicament of frequent false alarms in existing pure data-driven methods under business mutation scenarios. Furthermore, by innovatively introducing baseline health indicators and contamination risk coefficients, it achieves real-time monitoring and quantitative characterization of the baseline's own health status. Combined with an incremental update decision mechanism driven by the contamination risk coefficient and a selective memory replay strategy based on a long-term memory buffer, it effectively blocks the gradual contamination path of low-intensity attack traffic on the baseline model, thus solving the technical problem of irreversible degradation of detection capabilities in incremental learning scenarios. Further, through a branch-differentiated incremental update mechanism and a dual-branch collaborative detection and weighted fusion strategy, this invention can achieve adaptive learning of new legitimate business models while effectively preserving learned knowledge. This results in the detection system maintaining long-term stable operation and high-precision detection in dynamic network environments.

[0021] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0022] Figure 1 A flowchart illustrating an artificial intelligence-based network detection method provided in this application embodiment;

[0023] Figure 2 This is a schematic diagram of the structure of an artificial intelligence-based network detection system provided in another embodiment of this application. Detailed Implementation

[0024] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0026] In one embodiment, such as Figure 1 As shown, an artificial intelligence-based network detection method is provided. This embodiment illustrates the method's application to a terminal, but it is understood that the method can also be applied to a server, or to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0027] S1: Collect network traffic data, perform session reassembly on the network traffic data to extract multi-dimensional traffic features, and match the corresponding service type labels based on the multi-dimensional traffic features to generate a sequence of traffic feature vectors carrying service type labels.

[0028] Specifically, in the initial stage of the network detection method, the system acquires raw network traffic data packets from the target network environment through network probes or traffic acquisition devices. These raw traffic data packets exist in the transmission medium in an unordered and discrete form. The system needs to perform session reassembly processing, associating and sorting data packets belonging to the same communication session according to five-tuple information to reconstruct the complete session interaction sequence. The basis for session reassembly includes source IP address, destination IP address, source port, destination port, and transport layer protocol type. Based on these identifiers, the system merges the scattered data packets into the corresponding session streams and arranges them in timestamp order to form structured session data.

[0029] After completing session reassembly, the system extracts multi-dimensional traffic features from each session stream. These multi-dimensional traffic features cover statistical aspects, including session duration, total number of data packets, uplink and downlink traffic bytes, and the mean and variance of data packet arrival intervals; protocol-level features, including transport layer protocol type distribution, application layer protocol identifiers, and flag distribution patterns; and behavioral aspects, including connection establishment frequency, port usage patterns, and destination address dispersion. The system organizes these features according to preset dimensions to form the original feature vector corresponding to each session stream.

[0030] Furthermore, the system inputs the original feature vectors into the business type matching module. This module compares the distribution patterns of the values ​​in each dimension of the feature vectors with a pre-defined business type feature library to determine the business type to which the current session flow belongs. Business types include sudden traffic surges caused by promotional activities, update traffic caused by version releases, scheduling traffic caused by CDN policy adjustments, and regular traffic generated by daily office work and normal business interactions. Based on the matching results, the system attaches a corresponding business type label to each original feature vector, ultimately generating a sequence of traffic feature vectors carrying business type labels. This sequence is arranged in chronological order, preserving the temporal relationship of the traffic data.

[0031] S2: Based on the traffic feature vector sequence carrying business type labels, train a global baseline model that does not distinguish between business types and multiple business semantic baseline models corresponding to each business type label. Select a preset number of samples with the smallest reconstruction error from the traffic feature vectors corresponding to each business type label and store them in the long-term memory buffer of the corresponding business type label to generate a two-branch dynamic baseline model containing a global baseline model, multiple business semantic baseline models and a long-term memory buffer.

[0032] Specifically, after obtaining the traffic feature vector sequence carrying service type labels, the system initiates the construction process of a dual-branch dynamic baseline model. The dual-branch dynamic baseline model consists of three components: a global baseline model, multiple service semantic baseline models, and a long-term memory buffer. The global baseline model is trained using all traffic feature vector sequences as training data, without distinguishing between service type labels carried by the samples. The system inputs all samples into the global baseline model constructed based on a long short-term memory network autoencoder. This model compresses the input sequence into a low-dimensional latent space representation through the encoder, and then reconstructs the latent space representation into an output sequence of the original dimensions through the decoder. The training objective is to minimize the reconstruction error of all samples. After sufficient training, the global baseline model can capture the overall statistical regularity and general temporal patterns of network traffic.

[0033] The training of the business semantic baseline model is performed separately according to the business type label. The system constructs a separate business semantic baseline model for each business type label, and the training data for each model is limited to the traffic feature vector carrying the corresponding business type label. Each business semantic baseline model also adopts a long short-term memory network autoencoder structure, and its training objective is to minimize the reconstruction error of its corresponding business type samples. After training, each business semantic baseline model can capture the unique patterns and temporal dependencies of the corresponding business type at the traffic feature level.

[0034] After model training is complete, the system calculates the reconstruction error of all traffic feature vectors corresponding to each business type label on the corresponding business semantic baseline model, and sorts them in ascending order of reconstruction error, selecting a predetermined number of samples with the smallest reconstruction error. These samples represent the most typical and representative normal behavior patterns in the corresponding business type, and the system stores them in the long-term memory buffer of the corresponding business type label. The long-term memory buffer maintains an independent storage space for each business type to store historical typical samples of that type of business for use in subsequent incremental update phases. Thus, the system generates a dual-branch dynamic baseline model, in which a global baseline model provides a network-wide general behavior reference, multiple business semantic baseline models provide fine-grained business behavior references, and the long-term memory buffer provides historical typical sample references.

[0035] S3: Input the acquired real-time traffic feature vectors into the global baseline model and the business semantic baseline model corresponding to the real-time traffic feature vectors respectively for reconstruction, obtain the global reconstruction error and the business semantic reconstruction error, and calculate the comprehensive anomaly score by weighting and fusing the ratios of the global reconstruction error and the business semantic reconstruction error with the upper bound of the dynamic confidence interval of the global baseline model and the business semantic baseline model respectively.

[0036] Specifically, after the dual-branch dynamic baseline model is built and put into online operation, the system continuously receives real-time network traffic data and performs session reconstruction, multi-dimensional traffic feature extraction, and service type label matching on the real-time traffic data according to the same process in S1, generating a real-time traffic feature vector. The system simultaneously inputs this real-time traffic feature vector into the global baseline model and the service semantic baseline model corresponding to the service type label carried by the real-time traffic feature vector. The global baseline model encodes and decodes the input to reconstruct the model, outputting the reconstructed feature vector. The system calculates the difference between the original input feature vector and the reconstructed output to obtain the global reconstruction error. The service semantic baseline model also reconstructs the input and calculates the reconstruction error to obtain the service semantic reconstruction error. After obtaining the two reconstruction errors, the system also needs to obtain the upper bound of the dynamic confidence interval for each of the global baseline model and the service semantic baseline model. The upper bound of the dynamic confidence interval is calculated based on the statistical distribution of reconstruction errors of each model on the training set during the model training phase, representing the upper limit of the reconstruction error allowed for normal samples. The system calculates the ratio of the global reconstruction error to the upper bound of the dynamic confidence interval of the global baseline model, and the ratio of the business semantic reconstruction error to the upper bound of the dynamic confidence interval of the business semantic baseline model. These two ratios reflect the degree to which the real-time traffic feature vector deviates from the normal pattern at the global level and the business semantic level, respectively.

[0037] The system performs a weighted fusion of these two ratios. During the fusion process, weight coefficients are assigned to both the global reconstruction error ratio and the business semantic reconstruction error ratio. These weight coefficients are configured based on the degree of dependence of the business type on global consistency and business specificity. For business types with relatively uniform global behavioral patterns, the weight of the global reconstruction error ratio can be appropriately increased; for business types with significant individual differences, the weight of the business semantic reconstruction error ratio can be appropriately increased. The resulting weighted fusion value is the comprehensive anomaly score, which integrates the anomaly detection results from both the global and business semantic perspectives to determine whether there are anomalies in the current real-time traffic.

[0038] S4: Based on the comprehensive anomaly score, business semantic reconstruction error, and upper bound of the dynamic confidence interval, calculate the baseline health index and pollution risk coefficient of the current business type label. Based on the baseline health index and pollution risk coefficient, make incremental update decisions on the real-time traffic feature vector to generate a labeling result containing safe and normal samples, high-risk pollution samples, and abnormal samples.

[0039] Specifically, after calculating the comprehensive anomaly score, the system further evaluates the health status of the baseline model corresponding to the current business type label. The baseline health index is calculated based on the trend of the comprehensive anomaly score within a time window and the deviation of the business semantic reconstruction error from the upper bound of the dynamic confidence interval. The system determines whether the baseline model is in a stable operating state by monitoring the temporal evolution pattern of the comprehensive anomaly score. When the comprehensive anomaly score shows a continuous upward trend over a long time window, and the business semantic reconstruction error frequently approaches or exceeds the upper bound of the dynamic confidence interval, it indicates that the baseline model may be at risk of performance degradation or contamination, and the baseline health index decreases accordingly. The calculation of the contamination risk coefficient focuses on assessing the degree of contamination that the current real-time traffic feature vector may cause to the baseline after being included in the model update. The system determines the contamination risk coefficient based on the relative relationship between the business semantic reconstruction error and the upper bound of the dynamic confidence interval, as well as the specific numerical range of the comprehensive anomaly score. When the semantic reconstruction error does not exceed the upper limit of the dynamic confidence interval but is in the high range near the upper limit, it indicates that although the sample has not been judged as abnormal, its pattern has deviated from the distribution range of typical normal samples. If it is included in the model update, it may cause the baseline to shift in a direction that deviates from the normal direction. Such samples have a high risk of contamination.

[0040] The system makes incremental update decisions on real-time traffic feature vectors based on baseline health indicators and contamination risk coefficients, classifying them into one of three categories. The first category is safe and normal samples. These samples have business semantic reconstruction errors below the upper bound of the dynamic confidence interval, and their contamination risk coefficient is below a preset threshold, indicating typical normal behavior patterns and safe for model updates. The second category is high-risk contamination samples. These samples have business semantic reconstruction errors that, while not exceeding the upper bound of the dynamic confidence interval, are in a high range, or their contamination risk coefficient reaches or exceeds the preset threshold, indicating that although they are not currently classified as abnormal, they pose a potential risk of contaminating the baseline and require careful handling. The third category is abnormal samples. These samples have a comprehensive abnormality score exceeding the preset abnormality threshold, or their business semantic reconstruction errors significantly exceed the upper bound of the dynamic confidence interval, indicating that their behavior patterns have clearly deviated from the normal range and should be excluded from model updates.

[0041] S5: Based on the labeling results, incrementally update the global baseline model and the business semantic baseline model. For real-time traffic feature vectors labeled as safe and normal samples, if the business semantic reconstruction error does not exceed the upper bound of the dynamic confidence interval and the pollution risk coefficient is lower than the preset pollution risk threshold, the real-time traffic feature vectors are mixed with historical samples sampled from the long-term memory buffer and used to update the corresponding business semantic baseline model. For real-time traffic feature vectors labeled as abnormal samples, skip this incremental update and update the model parameters of the global baseline model and the business semantic baseline model and the corresponding upper bound of the dynamic confidence interval.

[0042] Specifically, based on the labeling results generated in step S4, incremental update operations are performed on the global baseline model and the business semantic baseline model. For real-time traffic feature vectors labeled as safe and normal samples, the system further determines whether the business semantic reconstruction error of the sample does not exceed the upper bound of the dynamic confidence interval of the corresponding business semantic baseline model and whether the pollution risk coefficient of the corresponding business type label is lower than the preset pollution risk threshold. When both conditions are met, the system includes the real-time traffic feature vector in the incremental update queue. When updating the corresponding business semantic baseline model, the system extracts a preset number of historical samples from the long-term memory buffer of the business type label using a uniform random sampling method. The current real-time traffic feature vector is mixed with the extracted historical samples to form an updated training set, and the parameters of the business semantic baseline model are fine-tuned based on this updated training set.

[0043] The operation of sampling historical samples from the long-term memory buffer and mixing them with current samples for training preserves the characteristics of historical normal patterns during new knowledge learning, preventing drastic shifts in model parameters. For real-time traffic feature vectors marked as anomalous samples, the system skips the current incremental update and does not update the parameters of any baseline model to avoid anomalous patterns being included in the normal baseline. For real-time traffic feature vectors marked as high-risk contamination samples, the system determines whether to include them in the update under the same conditions as for safe and normal samples, but additionally logs the contamination event. After completing the above incremental update operations, the system recalculates the upper bound of the dynamic confidence interval for the global baseline model and the business semantic baseline model based on the reconstruction error distribution of each model on the recent validation sample set, enabling the upper bound of the dynamic confidence interval to adaptively adjust as the normal network traffic patterns evolve.

[0044] In summary, the AI-based network detection method provided in this application constructs a dual-branch dynamic baseline architecture consisting of a global baseline model and multiple business semantic baseline models. By combining this architecture with business type tags to extract semantic features from traffic, it fundamentally decouples legitimate traffic mutations caused by business operations from malicious attack traffic at the semantic level. This overcomes the technical predicament of frequent false alarms in existing pure data-driven methods under business mutation scenarios. Furthermore, by innovatively introducing baseline health indicators and contamination risk coefficients, it achieves real-time monitoring and quantitative characterization of the baseline's own health status. Combined with an incremental update decision mechanism driven by the contamination risk coefficient and a selective memory replay strategy based on a long-term memory buffer, it effectively blocks the gradual contamination path of low-intensity attack traffic on the baseline model, thus solving the technical problem of irreversible degradation of detection capabilities in incremental learning scenarios. Further, through a branch-differentiated incremental update mechanism and a dual-branch collaborative detection and weighted fusion strategy, this invention can achieve adaptive learning of new legitimate business models while effectively preserving learned knowledge. This results in the detection system maintaining long-term stable operation and high-precision detection in dynamic network environments.

[0045] In one embodiment, S1 of the artificial intelligence-based network detection method provided by the present invention specifically includes the following steps:

[0046] S11: Collect network traffic data, perform session reassembly processing on the network traffic data according to the five-tuple, aggregate the unordered data packets into session streams according to five dimensions: source IP address, destination IP address, source port, destination port and protocol type, and sort the data packets in each session stream according to the timestamp of each data packet to generate session stream data containing timestamp sequence, data packet size sequence, protocol type and direction flag;

[0047] S12: Based on the sliding time window segmentation technique, the session flow data is segmented and processed. The continuous network traffic is divided into several traffic segments according to the preset time window, and the session flow in each traffic segment is statistically aggregated to generate network layer statistical features and transport layer timing features in each time window.

[0048] S13: Based on the pre-built business semantic feature library, perform business pattern matching processing on multi-dimensional traffic features. Compare the multi-dimensional traffic features of each traffic segment with the feature fingerprints of each business pattern in the business semantic feature library. Assign corresponding business type labels to each traffic segment according to the business pattern with the highest matching score, and generate a traffic feature vector sequence carrying business type labels.

[0049] Specifically, the system deploys traffic acquisition probes at network egress points or core switching nodes to acquire raw network data packets via port mirroring or optical splitting. The raw data packets arrive at the acquisition interface in chronological order, and there is no session association between them. The system needs to reorganize the packets based on their header fields. The system extracts the source IP address, destination IP address, source port, destination port, and transport layer protocol type from each packet, concatenating these five fields into a five-tuple identifier. Packets with the same five-tuple identifier are grouped into the same session. The system iterates through all data packets within the acquisition period, grouping and aggregating them according to the five-tuple identifier to form several packet sets, each set corresponding to a complete communication session.

[0050] For each session's data packet set, the system extracts the timestamp field from the header of each data packet and sorts the data packets within the set according to their timestamp values ​​from smallest to largest, ensuring that the data packets within each session are presented in chronological order. The sorted session data contains a timestamp sequence. ,in This represents the total number of packets within the session. This is the timestamp of the i-th data packet, with the unit consistent with the system time of the acquisition device; it also includes a sequence of data packet sizes. , } represents the byte length of the i-th data packet; it also records the transport layer protocol type and direction flag relative to the session initiator for each data packet, with the direction flag taking the value of uplink or downlink. The system further calculates the duration of the session. This physical quantity represents the duration of a session from its inception to its termination, measured in seconds, and serves as the foundational statistical measure for subsequent feature extraction. Through the above processing, the system transforms discrete data packet streams into structured session stream data, providing ordered input data for subsequent time window segmentation and feature aggregation.

[0051] The system is based on a preset fixed time window length. Split the session stream data along a timeline. The unit of measurement is seconds, and its value is set based on the periodic variation of network traffic. The system divides the time axis into non-overlapping continuous intervals starting from the start of data collection. ,in For window number, To capture the start time, each window contains data packets from all sessions within that time period. For each time window, the system performs statistical aggregation on all session flows, generating network layer statistical features and transport layer timing features. The network layer statistical features include the total number of data packets within the window. ,in Iterate through all sessions within this window. The number of data packets in the j-th session; the total number of bytes in the window. , The length in bytes of the i-th data packet in the j-th session; the number of uplink bytes. Downlink byte count The average data packet size is obtained by summing the values ​​based on the direction flags. Transport layer timing characteristics include average session duration. Where M is the total number of sessions within the window. For the first Duration of each session; mean packet arrival interval ,in The k-th timestamp of all data packets within the window, sorted by global time; and the standard deviation of the arrival interval is calculated. ,in All of the above statistics have clearly defined physical dimensions. and For dimensionless counting, The unit of measurement is bytes. and The unit of measurement is seconds. The system organizes the above features into fixed-dimensional vectors, with each time window corresponding to a multi-dimensional traffic feature vector, which represents the overall statistical behavior of network traffic within that time period.

[0052] The system pre-constructs a business semantic feature library, which contains several business pattern entries. Each business pattern corresponds to a type of network business activity, including promotional activity traffic, version release traffic, CDN scheduling traffic, and daily office traffic. Each business pattern entry stores a feature fingerprint vector, which is obtained by the system in the offline phase through feature extraction and clustering of historical typical traffic segments of this type of business. Its dimension is consistent with the multi-dimensional traffic feature vector generated in S12. For each time window corresponding to the multi-dimensional traffic feature vector generated in S12... ,in The system calculates the feature dimension. Compared with the feature fingerprints of various business models in the feature library The matching score is calculated using cosine similarity, and its formula is: ,in The dot product of two vectors. and These are the Euclidean norms of the two vectors, respectively. The score ranges from [value missing]. The dimensionless value represents the degree of similarity in direction between the current traffic feature vector and the fingerprints of various business models. A higher score indicates that the behavior pattern of the traffic segment is more similar to the corresponding business type.

[0053] The system iterates through all business patterns in the feature library, calculates the matching score for each pattern, and selects the business pattern with the highest score as the basis for assigning a business type label to that traffic segment. When the highest score is lower than a preset matching acceptance threshold, the system marks the traffic segment as an unknown business type and stores its feature vector in a queue awaiting manual review for future expansion of the feature library. The system attaches a corresponding business type label to each traffic segment and organizes them into a traffic feature vector sequence according to the chronological order of the time window. Each vector is labeled ,in This is a set of business type labels. This sequence retains both temporal order and business semantic information, providing input data for the subsequent training of the dual-branch dynamic baseline model.

[0054] In one embodiment, S2 of the artificial intelligence-based network detection method provided by the present invention specifically includes the following steps:

[0055] S21: Based on the traffic feature vector sequence carrying business type labels, all traffic feature vectors are used as training samples in a way that does not distinguish between business types. The LSTM autoencoder is trained and processed. The encoder compresses the input sequence into a hidden state vector and then reconstructs it into the original sequence through the decoder. The training objective is to minimize the reconstruction error and generate a global baseline model that does not distinguish between business types.

[0056] S22: Based on each business type label in the traffic feature vector sequence carrying business type labels, train independent LSTM autoencoders for each subset of traffic feature vectors corresponding to each business type label. The LSTM autoencoder corresponding to each business type label only uses the traffic feature vector under that business type label as the training sample, and takes minimizing the reconstruction error on their respective business subsets as the training objective to generate multiple business semantic baseline models corresponding to each business type label.

[0057] S23: Based on the subset of traffic feature vectors corresponding to each business type label and their corresponding business semantic baseline models, calculate the reconstruction error of each traffic feature vector on each business semantic baseline model, sort them in ascending order of reconstruction error, select the preset number of samples with the smallest reconstruction error, store them in the long-term memory buffer of the corresponding business type label, and generate a dual-branch dynamic baseline model containing a global baseline model, multiple business semantic baseline models and a long-term memory buffer.

[0058] Specifically, after acquiring the traffic feature vector sequence carrying service type labels, the system organizes all traffic feature vectors in the sequence into a training dataset in chronological order. During training, the service type labels carried by each sample are not used for sample selection or grouping; all samples are treated equally. The system constructs an autoencoder structure based on a Long Short-Term Memory (LSTM) network, which consists of a cascaded encoding network and a decoding network. The encoding network contains several LSTM layers, each containing several LSTM units. Each LSTM unit maintains a forget gate, an input gate, an output gate, and a unit state. The gating mechanism is calculated as follows: Forget Gate... Input gate Candidate cell status , unit state Output gate Hidden state ,in For a moment The input vector, This is the hidden state from the previous moment. This represents the cell state at the previous moment. For each gating unit, there is a trainable weight matrix. For the corresponding bias vector, It is the Sigmoid activation function. The hyperbolic tangent activation function is used. This represents element-wise product. The encoding network reads the input sequence sequentially step by step. Where T is the sequence length, and each Given a d-dimensional flow feature vector, the encoding network processes the complete sequence and outputs the final hidden state vector. ,in The hidden state dimension is represented by this vector, which serves as a compressed representation of the input sequence.

[0059] Furthermore, the decoding network also consists of several LSTM layers, and its network structure is symmetrical to that of the encoding network. The decoding network uses the hidden state vector output by the encoding network as its initial state and gradually reconstructs the output sequence. each

[0060] This represents the reconstructed value of the input vector at the corresponding time step. The system's training objective is to minimize the reconstruction error, employing the mean squared error loss function. ,in The loss function is calculated as the square of the Euclidean distance, and its dimensions are the sum of the squares of each dimension of the feature vector. The system employs a time-based backpropagation algorithm for parameter optimization. The weight matrix is ​​initialized using a Xavier uniform distribution, the bias vector is initialized to zero, and the optimizer uses adaptive moment estimation. During training, the system divides the dataset into several batches, each containing multiple sequence samples, which are sequentially fed into the network for forward propagation and gradient calculation. After multiple iterations until the loss function converges, the converged network becomes the global baseline model.

[0061] The system uses a set of business type tags. Each tag in Select the tags with the following labels from the traffic feature vector sequence carrying business type labels. All samples are used to form the training subset for that business type. The training subsets for each business type may differ in the number of samples; the system trains an independent LSTM autoencoder for each business type. For each business type label... The system constructs a dedicated LSTM autoencoder. The encoding and decoding network structures of this autoencoder are consistent with the global baseline model in S21, both containing multiple LSTM layers and corresponding LSTM units. However, each business semantic baseline model only uses the training subset corresponding to the business type label for parameter learning, and there is no parameter sharing between different business types.

[0062] For each business type label The samples in its training subset are organized into several sequences in chronological order, each sequence Each This is a traffic feature vector carrying label l. The system inputs this sequence into an LSTM autoencoder for this service type. The encoding network reads the sequence step by step and outputs the hidden state vector. The decoding network reconstructs the sequence using the hidden state as the initial state. The training objective of each business semantic baseline model is to minimize the reconstruction error on the training subset of that business type, and the loss function is in the form of: The loss function is independent across different business types. The system uses the same Xavier uniform distribution initialization strategy for the weight matrices of the LSTM autoencoders for each business type, and initializes the bias vectors to zero. Each model uses a time-based backpropagation algorithm and an adaptive moment estimation optimizer for parameter updates. Each business semantic baseline model undergoes independent multi-round iterative training until its respective loss function converges.

[0063] After training, the system labels each business type. Save the corresponding business semantic baseline model. This model has low reconstruction error characteristics only for the traffic feature vector under this label. For inputs of other business types, it will produce large reconstruction errors due to distribution differences, thereby realizing behavioral modeling at the business semantic level.

[0064] After the system completes the training of the business semantic baseline model corresponding to each business type, it labels each business type. For each traffic feature vector in the training subset of this business type The corresponding business semantic baseline model is input and forward propagation is performed. The specific process is as follows: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error.] As a sequence input of length 1, the encoding network outputs the corresponding hidden state vector. The decoding network reconstructs the input using the hidden state as the initial state, and obtains the reconstructed vector. The system calculates the reconstruction error for this sample. The error is a scalar, and its dimension is the sum of squares of the units of each dimension of the eigenvector.

[0065] The system iterates through all samples in the training subset for this business type, calculates the reconstruction error of each sample on the corresponding business semantic baseline model, and obtains the error set. ,in For business type The total number of training samples. The system sorts the error set in ascending order of numerical value. The smaller the reconstruction error, the higher the reconstructability of the sample on the business semantic baseline model, that is, the more consistent the behavior pattern of the sample is with the typical pattern of this business type. The system selects the top samples with the smallest reconstruction errors from the sorted results. 1 sample, of which This is the preset long-term memory buffer capacity, which is set separately for each service type. The system will use this... The traffic feature vector of each sample and its corresponding business type label are stored in the long-term memory buffer of that business type.

[0066] The long-term memory buffers for each service type are independent and do not interfere with each other. Each buffer is used to store the most representative historical samples of that service type. The system organizes the global baseline model trained in S21, the semantic baseline models for each service trained in S22, and the long-term memory buffers for each service type into a dual-branch dynamic baseline model. In the subsequent online detection phase, this model uses the global baseline model and the service semantic baseline model in parallel to reconstruct real-time traffic, while using typical historical samples from the long-term memory buffers for incremental updates to maintain the model's memory of historical normal patterns.

[0067] In one embodiment, S3 of the artificial intelligence-based network detection method provided by the present invention specifically includes the following steps:

[0068] S31: Perform business type label matching processing on the acquired real-time traffic feature vector, determine the business type label corresponding to the real-time traffic feature vector according to the pre-built business semantic feature library, and input the real-time traffic feature vector into the global baseline model and the business semantic baseline model corresponding to the real-time traffic feature vector for reconstruction processing, and obtain the global reconstruction error and the business semantic reconstruction error respectively.

[0069] S32: Based on the reconstruction error sequence of the most recent samples judged as normal within the sliding window, perform real-time sliding update processing on the upper bound of the confidence interval of the global baseline model and the upper bound of the confidence interval of the business semantic baseline model, respectively, and calculate the upper bound of the global dynamic confidence interval and the upper bound of the business semantic dynamic confidence interval at the current moment.

[0070] S33: Calculate the first deviation ratio based on the business semantic reconstruction error and the upper bound of the business semantic dynamic confidence interval, calculate the second deviation ratio based on the global reconstruction error and the upper bound of the global dynamic confidence interval, and perform a weighted summation of the first deviation ratio and the second deviation ratio using the matching confidence of the business semantic baseline model as the weight to generate a comprehensive anomaly score.

[0071] Specifically, the system continuously receives real-time network traffic data, performs session reassembly, time window segmentation, and statistical feature extraction on the real-time data, and generates a real-time traffic feature vector for the current moment. ,in This serves as the feature dimension. The system compares this feature vector with the feature fingerprints of each business mode in a pre-built business semantic feature library, calculates the matching score using cosine similarity, and selects the business mode with the highest score as the business type label for this real-time traffic feature vector. .

[0072] The system simultaneously acquires the global baseline model and the corresponding labels. The business semantic baseline models consist of two pre-trained LSTM autoencoders, each containing an encoder and a decoder network. The encoder network comprises multiple stacked LSTM layers, each containing several LSTM units. The decoder network structure is symmetrical to the encoder network. The system will... The input sequence, each of length 1, is input into the encoding networks of the two models. The encoding networks output the corresponding hidden state vectors, and the decoding networks reconstruct the input sequence using these hidden states as their initial states, outputting the reconstructed vectors respectively. and .

[0073] The system calculates the global reconstruction error. ,in This error is the square of the Euclidean distance, and its dimension is the sum of the squares of each dimension of the feature vector, reflecting the degree to which the current sample is unreconstructable in the global model. The system also calculates the business semantic reconstruction error. The dimensions of this error are the same as The same error reflects the degree to which the current sample cannot be reconstructed in the corresponding business semantic model. The two errors quantify the degree of deviation between the current traffic pattern and the normal baseline from the perspectives of global general behavior and business-specific behavior, respectively.

[0074] The system maintains a fixed-length sliding window for both the global baseline model and each business semantic baseline model. This window only stores the reconstruction error values ​​of samples marked as normal on the corresponding model; reconstruction errors of abnormal samples are not included in the window. For the global baseline model, the system maintains a window of [missing information]. Store the most recent Global reconstruction error of a normal sample ,in For the current moment, This is the window length. For each business type label l, the system maintains the corresponding window. Store recent The semantic reconstruction error of a normal sample with label l }

[0075] The system calculates the mean of the error sequence within each window at each time step. and standard deviation , where the mean The dimensions of the standard deviation are consistent with those of the reconstruction error.

[0076] Based on the above statistics, the system calculates the upper bound of the global dynamic confidence interval. ,in This is a preset confidence fold coefficient, a dimensionless scalar whose physical meaning is to control the degree to which the confidence interval covers the proportion of normal samples. A larger value indicates a higher upper bound in the interval, and a wider tolerance range for normal samples. Similarly, the system labels each business type. Calculate the upper bound of the dynamic confidence interval for this business type. ,in )and From windows respectively The mean and standard deviation of the internal error. Through real-time sliding updates, the system dynamically adjusts the upper bound of the confidence interval to follow the normal evolution trend of network traffic, avoiding false alarms or missed alarms caused by static thresholds.

[0077] The system obtains the global reconstruction error at the current moment. Business semantic reconstruction error Upper bound of global dynamic confidence interval Upper bound of dynamic confidence interval for business semantics Next, two deviation ratios are calculated. The first deviation ratio is the ratio of the business semantic reconstruction error to the upper bound of its dynamic confidence interval. The second deviation ratio is the ratio of the global reconstruction error to the upper bound of its dynamic confidence interval. Both ratios are dimensionless scalars, physically representing the relative magnitude of the reconstruction error of the current sample on each model compared to the upper limit of the normal reconstruction error tolerance. A ratio greater than 1 indicates that the deviation of the current sample has exceeded the confidence interval. The system calculates the matching confidence of the business semantic baseline model. This confidence level is defined based on the stability of the upper bound of the dynamic confidence interval for the current business type within a historical window. The specific calculation method is as follows: ,in The upper bound of the dynamic confidence interval for business semantics at recent time points. The coefficient of variation, i.e. This coefficient is dimensionless and reflects the degree of fluctuation of the upper bound of the confidence interval within the recent time window. The more drastic the fluctuation of the upper bound of the confidence interval, the lower the matching confidence, indicating that the current business semantic baseline model is not stable enough in representing the normal pattern.

[0078] The system is based on The first deviation ratio and the second deviation ratio are weighted and summed to generate a comprehensive anomaly score. The physical meaning of this weighted summation formula is as follows: when the business semantic baseline model performs stably in the near term, the comprehensive anomaly score depends more on the degree of deviation at the business semantic level, thus reflecting business-specific anomalies more precisely; when the business semantic baseline model is unstable, the system shifts its detection focus to the global baseline model to ensure the reliability of the anomaly detection benchmark. Comprehensive Anomaly Score It is a dimensionless scalar, and the larger its value, the greater the deviation of the current real-time traffic feature vector from the normal behavior pattern.

[0079] In one embodiment, S4 of the artificial intelligence-based network detection method provided by the present invention specifically includes the following steps:

[0080] S41: Calculate the baseline health index of the current business type label based on the proportion of the number of samples judged as abnormal in the sliding window to the total number of samples in the sliding window. When the baseline health index is continuously lower than the preset health lower bound, it is determined that the baseline model of the business type label is at risk of contamination.

[0081] S42: When the business semantic reconstruction error exceeds the upper limit of the business semantic dynamic confidence interval and the comprehensive anomaly score does not exceed the preset anomaly judgment threshold, a potential pollution alarm is triggered. Based on the triggering frequency of the potential pollution alarm and the preset attenuation coefficient, the pollution risk coefficient of the current business type label is calculated using an incremental method.

[0082] Among them, the potential pollution alarm is used to indicate the pollution state in which the business semantic baseline model has been gradually eroded by abnormal traffic, its reconstruction error of abnormal samples tends to decrease, and the upper bound of the confidence interval is continuously raised.

[0083] S43: Based on the comparison results of the comprehensive anomaly score and the preset anomaly judgment threshold, and the comparison results of the pollution risk coefficient and the preset pollution risk threshold, perform incremental update decision processing on the real-time traffic feature vector. Mark the samples whose comprehensive anomaly score does not exceed the anomaly judgment threshold and whose pollution risk coefficient is lower than the pollution risk threshold as safe and normal samples. Mark the samples whose pollution risk coefficient is not lower than the pollution risk threshold as high-risk pollution samples and store them in the isolation buffer. Mark the samples whose comprehensive anomaly score exceeds the anomaly judgment threshold as abnormal samples. Generate a marking result containing safe and normal samples, high-risk pollution samples, and abnormal samples.

[0084] Specifically, the system maintains a sliding window with a fixed time span for each service type label. This window slides forward along the time axis, storing the anomaly labeling status of each sample within the most recent collection period after comprehensive anomaly scoring. The system counts the number of samples marked as anomaly within this sliding window and calculates the ratio of this number to the total number of samples within the sliding window. This ratio reflects the frequency of abnormal behavior in recent traffic. Based on this ratio, the system defines a baseline health index, which is calculated as follows:

[0085]

[0086] in, For the current moment, This is the tag for the current business type. This represents the number of abnormal samples belonging to business type l within the sliding window. The sliding window contains business types The total sample size. This indicator This is a dimensionless proportion, which physically represents the proportion of samples in recent traffic that have not been identified as abnormal. A higher value indicates that the traffic pattern of the current business type is more stable and consistent with the representation range of the baseline model. A lower value indicates that the proportion of abnormal samples in recent traffic is increasing, and the baseline model's ability to characterize the normal behavior of this business type may deteriorate.

[0087] The system will Compared with the preset lower bound of health, when If the baseline model for a service type label falls below this lower bound for multiple consecutive sliding window periods, the system determines that the baseline model is at risk of contamination. This multi-period judgment condition filters out temporary health drops caused by sudden abnormal traffic spikes, preventing occasional fluctuations from being misjudged as baseline contamination. This judgment result serves as one of the triggering conditions for subsequent contamination risk coefficient calculations, and the system adjusts its trust level in the baseline model for that service type accordingly.

[0088] The system calculates the business semantic reconstruction error based on the current real-time traffic feature vector at each moment. and comprehensive abnormality score , where l is the business type label carried by the current sample. The system also obtains the upper bound of the dynamic confidence interval for the business semantics corresponding to that business type. This upper bound is obtained by statistically analyzing the reconstruction error distribution of the business semantic baseline model on the corresponding business type training set during the model training phase. Its calculation method is as follows:

[0089]

[0090] in, This represents the mean reconstruction error of the training samples for this business type. The standard deviation of the reconstruction error of the training samples for this business type. This is a preset fold factor used to control the proportion of normal samples covered by the confidence interval. When the system detects... and At the time of their establishment, among them To pre-determine the anomaly threshold, the system triggers a potential contamination alarm. The physical meaning of this alarm is that the reconstruction deviation of the current sample at the business semantic level has exceeded the normal confidence range, but its overall anomaly score has not yet reached the anomaly threshold. This indicates that this type of sample is in a transitional region between normal and anomalous distributions. If the model continues to be updated with this type of sample, the reconstruction error distribution of the business semantic baseline model will gradually shift towards an anomalous pattern, leading to… and Increase The resulting rise in elevation eventually reduces the reconstruction error of previously detectable abnormal samples, leading to a decrease in detection sensitivity. The system calculates the pollution risk coefficient based on the trigger history of potential pollution alarms. Its recursive calculation formula is:

[0091]

[0092] in, For the current moment, Let l be the pollution risk coefficient for business type l at the current moment. The pollution risk coefficient at the previous moment. To preset the pollution increment coefficient, For the preset attenuation coefficient, This is an indicator function that takes the value 1 when the condition within the parentheses is true, and otherwise takes the value _____. The recursive structure of this formula consists of two terms: the first term is a positive increment term, which is applied only when a potential pollution alarm is triggered, with a step size... Increased pollution risk; the second term is the attenuation term, expressed as an attenuation coefficient. The product of the pollution risk value at the previous moment continuously reduces the pollution risk at each moment. The dimensions of ) are from and Together, when both are dimensionless coefficients, The value is dimensionless and its physical meaning is a relative estimate of the degree of cumulative contamination suffered by the current business semantic baseline model. The higher the value, the greater the risk of the baseline being eroded by abnormal patterns. The design of the attenuation term ensures that the contamination risk coefficient decays exponentially over time after potential contamination alarms stop being triggered, reflecting the mechanism by which the system gradually restores trust in the baseline after continuously receiving normal samples.

[0093] The system performs a comprehensive anomaly score on each real-time traffic feature vector. Calculation and pollution risk coefficient After the update, an incremental update decision is executed. The system first compares... Compared with the preset anomaly detection threshold The size relationship, and comparison at the same time With respect to the preset pollution risk threshold The size relationship. When and When the sample is marked as a safe and normal sample, the deviation of such samples from both the global and business semantic perspectives is within an acceptable range, and the contamination risk coefficient is below the risk threshold, indicating that it belongs to the typical normal mode of the current baseline model’s credible representation and can be safely used for subsequent incremental updates.

[0094] when and When a sample is identified as a high-risk contamination sample, the system marks it as such. Although the overall anomaly score of such samples does not exceed the anomaly judgment threshold, the contamination risk coefficient has reached or exceeded the risk threshold, indicating that the sample is in the edge region of normal distribution. Its pattern characteristics have structural differences from the typical training samples of the baseline model. If it is directly included in the incremental update, it may cause the parameters of the baseline model to be adjusted in a direction that deviates from the normal center. The system stores such samples in the isolation buffer so that they do not participate in any incremental training of the model for the time being. The system will decide whether to release or permanently isolate them after observing their temporal correlation with subsequent samples.

[0095] when When this happens, the system marks the sample as an anomalous. Such samples have a comprehensive anomalous score exceeding the anomalous judgment threshold, indicating that their deviation from the normal range is significantly greater than normal at the global or business semantic level. The system skips incremental update processing for these samples and does not include them in the training data of any model. (Comprehensive annomalous score) The unit of measurement is a dimensionless ratio, which physically represents the weighted sum of the normalized deviations of the reconstruction errors of the current sample from the global baseline model and the business semantic baseline model relative to the upper bound of their respective dynamic confidence intervals. A higher score indicates a greater difference between the sample's behavior pattern and the normal baseline. Through the above triple classification mechanism, the system generates labeling results containing three categories: safe and normal samples, high-risk contaminated samples, and abnormal samples. These labeling results provide a clear basis for sample selection in subsequent incremental update steps.

[0096] In one embodiment, S5 of the artificial intelligence-based network detection method provided by the present invention specifically includes the following steps:

[0097] S51: Based on the labeling results, for real-time traffic feature vectors labeled as safe and normal samples, determine whether the business semantic reconstruction error of the real-time traffic feature vector exceeds the upper bound of the dynamic confidence interval of the business semantics and whether the pollution risk coefficient of the real-time traffic feature vector is lower than the preset pollution risk threshold. If the business semantic reconstruction error does not exceed the upper bound of the dynamic confidence interval of the business semantics and the pollution risk coefficient is lower than the preset pollution risk threshold, then the real-time traffic feature vector is determined to be a valid sample that can be used for updating the business semantic baseline model.

[0098] S52: Randomly sample a preset number of historical samples from the long-term memory buffer, mix the preset number of historical samples with the effective samples, construct a training batch containing historical samples and current effective samples, use the training batch to incrementally update the business semantic baseline model corresponding to the real-time traffic feature vector, and constrain the direction of incremental update by training the current effective samples and historical clean samples together.

[0099] S53: Based on the labeling results, for real-time traffic feature vectors labeled as safe and normal samples, the real-time traffic feature vectors are used to update the global baseline model. For real-time traffic feature vectors labeled as abnormal samples, the incremental update is skipped. After the incremental update is completed, the model parameters of the global baseline model and the business semantic baseline model and the corresponding upper bound of the dynamic confidence interval are updated synchronously.

[0100] Specifically, the system performs an incremental update and filtering process on the business semantic baseline model based on the labeling results. This applies to real-time traffic feature vectors labeled as safe and normal samples. The system extracts its business semantic reconstruction error. and pollution risk coefficient , where l is the business type label corresponding to the sample. The system will The upper bound of the current dynamic confidence interval for this business type Compare, and at the same time With respect to the preset pollution risk threshold Comparison. When and When both conditions are met, the system determines that the sample is a valid sample that can be used to update the business semantic baseline model. The physical meaning of this determination condition is that: the reconstruction error of the sample at the business semantic level is within the normal fluctuation range, indicating that its behavior pattern is consistent with the normal distribution of the current baseline representation; at the same time, the contamination risk coefficient is lower than the threshold, indicating that the sample does not have the risk attribute that would cause the baseline parameters to shift in an abnormal direction. The dimension of is the sum of squares of the units of each dimension of the eigenvector. Having the same dimensions, a direct comparison between the two demonstrates physical consistency. For a dimensionless scalar, its recursive formula is: ,in and Dimensionless preset coefficients are used to control the pollution increment and decay rate, respectively. The system only includes samples that simultaneously meet both of the above conditions in subsequent incremental training, in order to exclude ambiguous samples in the boundary region between normal and abnormal, and avoid introducing unnecessary parameter perturbations.

[0101] After obtaining valid samples, the system performs a random sampling operation from the long-term memory buffer corresponding to the business type, with the sampling quantity being the preset batch capacity. The long-term memory buffer stores typical historical samples with the smallest reconstruction error during the training phase for this business type; its traffic feature vector is denoted as... Each sample carries the same business type label. The system will sample Historical samples and current valid samples The mixture is processed to form a mixture containing A training batch of samples. The training batch is constructed as follows: The system performs incremental updates on the business semantic baseline model corresponding to business type l using this training batch. The business semantic baseline model is a multi-layer LSTM autoencoder, whose encoding network contains several LSTM layers, with each LSTM unit having a hidden state dimension of p. The decoding network structure is symmetrical to the encoding network. The loss function for incremental updates adopts the mean squared error form. ,in For the model to input The reconstructed output loss function is averaged over all samples in the batch, with dimensions consistent with the reconstruction error. The system trains with both current valid samples and historical clean samples, ensuring that the model parameter updates are constrained by typical historical patterns, preventing the model from forgetting historical normal behavior due to over-adaptation to recent traffic distribution. The system uses an adaptive moment estimation optimizer for parameter updates, with the learning rate decaying with iterations according to a preset strategy. The gradient of the weight matrix is ​​calculated using a time-based backpropagation algorithm.

[0102] The system simultaneously performs incremental updates to the global baseline model. For real-time traffic feature vectors labeled as safe and normal samples, the system does not filter them using contamination risk coefficients and upper bounds of confidence intervals; instead, it directly uses all real-time traffic feature vectors labeled as safe and normal samples to update the global baseline model. The structure of the global baseline model is the same as that of the business semantic baseline model, both being multi-layer LSTM autoencoders. The number of layers and unit dimensions of the encoding and decoding networks are consistent with those of the business semantic baseline model. The system organizes the accumulated safe and normal samples from multiple time points into training batches, with the batch size set according to a preset value. The loss function also uses mean squared error. ,in This represents the number of samples in the global update batch. For real-time traffic feature vectors marked as anomalous samples, the system skips this incremental update and does not include them in the training data of the global baseline model or any business semantic baseline model. After completing all incremental update operations, the system recalculates the reconstruction error distribution of each model on the current normal sample set, and updates the upper bound of the global dynamic confidence interval accordingly. and the upper bound of the dynamic confidence interval for each business semantic ,in )and ) represents the mean and standard deviation of the reconstruction error of the global baseline model on the most recent normal samples. )and The mean and standard deviation of the reconstruction error of the corresponding business semantic baseline model on the most recent normal samples are given, where k is a preset confidence factor, which is dimensionless. Through the above process, the system simultaneously achieves the model's adaptability to network evolution and its resistance to baseline contamination during the incremental update phase.

[0103] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0104] Based on the same inventive concept, this application also provides an AI-based network detection system for implementing the AI-based network detection method described above. The solution provided by this system is similar to the implementation described in the above method; therefore, the specific limitations in one or more AI-based network detection system embodiments provided below can be found in the limitations of the AI-based network detection method described above, and will not be repeated here.

[0105] Preferably, such as Figure 2 As shown, the present invention provides a network detection system 600 based on artificial intelligence, which is configured with the following modules:

[0106] The traffic feature construction module 610 is used to collect network traffic data, extract multi-dimensional traffic features after session reassembly of the network traffic data, and match the corresponding service type labels according to the multi-dimensional traffic features to generate a traffic feature vector sequence carrying the service type labels.

[0107] The dual-branch model construction module 620 is used to train a global baseline model that does not distinguish between business types and multiple business semantic baseline models corresponding to each business type label based on the traffic feature vector sequence carrying business type labels. It also selects a preset number of samples with the smallest reconstruction error from the traffic feature vectors corresponding to each business type label and stores them in the long-term memory buffer of the corresponding business type label to generate a dual-branch dynamic baseline model containing a global baseline model, multiple business semantic baseline models and a long-term memory buffer.

[0108] The comprehensive anomaly scoring module 630 is used to input the acquired real-time traffic feature vector into the global baseline model and the business semantic baseline model corresponding to the real-time traffic feature vector respectively for reconstruction, to obtain the global reconstruction error and the business semantic reconstruction error, and to calculate the comprehensive anomaly score by weighting and fusing the ratios of the global reconstruction error and the business semantic reconstruction error with the upper bound of the dynamic confidence interval of the global baseline model and the business semantic baseline model respectively.

[0109] The sample classification and labeling module 640 is used to calculate the baseline health index and pollution risk coefficient of the current business type label based on the comprehensive anomaly score, business semantic reconstruction error and upper bound of dynamic confidence interval, and to make incremental update decisions on the real-time traffic feature vector based on the baseline health index and pollution risk coefficient, and generate labeling results including safe and normal samples, high-risk pollution samples and abnormal samples.

[0110] The model incremental update module 650 is used to incrementally update the global baseline model and the business semantic baseline model based on the labeling results. For real-time traffic feature vectors labeled as safe and normal samples, if the business semantic reconstruction error does not exceed the upper bound of the dynamic confidence interval and the pollution risk coefficient is lower than the preset pollution risk threshold, the real-time traffic feature vectors are mixed with historical samples sampled from the long-term memory buffer and used to update the corresponding business semantic baseline model. For real-time traffic feature vectors labeled as abnormal samples, the incremental update is skipped, and the model parameters of the global baseline model and the business semantic baseline model and the corresponding upper bound of the dynamic confidence interval are updated.

[0111] In one embodiment, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described artificial intelligence-based network detection method.

[0112] In one embodiment, this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described artificial intelligence-based network detection method.

[0113] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0114] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0115] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A network detection method based on artificial intelligence, characterized in that, Includes the following steps: S1: Collect network traffic data, perform session reassembly on the network traffic data to extract multi-dimensional traffic features, and match the corresponding service type tags according to the multi-dimensional traffic features to generate a traffic feature vector sequence carrying service type tags; S2: Based on the traffic feature vector sequence carrying the business type label, train a global baseline model that does not distinguish between business types and multiple business semantic baseline models corresponding to each business type label, and select a preset number of samples with the smallest reconstruction error from the traffic feature vectors corresponding to each business type label and store them in the long-term memory buffer of the corresponding business type label to generate a dual-branch dynamic baseline model containing the global baseline model, the multiple business semantic baseline models and the long-term memory buffer. S3: Input the acquired real-time traffic feature vector into the global baseline model and the business semantic baseline model corresponding to the real-time traffic feature vector respectively for reconstruction, obtain the global reconstruction error and the business semantic reconstruction error, and calculate the comprehensive anomaly score by weighting and fusing the ratios of the global reconstruction error and the business semantic reconstruction error to the upper bound of the dynamic confidence interval of the global baseline model and the business semantic baseline model respectively. S4: Based on the comprehensive anomaly score, the business semantic reconstruction error, and the upper bound of the dynamic confidence interval, calculate the baseline health index and pollution risk coefficient of the current business type label, and make incremental update decisions on the real-time traffic feature vector based on the baseline health index and the pollution risk coefficient to generate a labeling result containing safe and normal samples, high-risk pollution samples, and abnormal samples. S5: Based on the labeling results, incrementally update the global baseline model and the business semantic baseline model. For real-time traffic feature vectors labeled as safe and normal samples, if the business semantic reconstruction error does not exceed the upper bound of the dynamic confidence interval and the pollution risk coefficient is lower than the preset pollution risk threshold, the real-time traffic feature vectors are mixed with historical samples sampled from the long-term memory buffer and used to update the corresponding business semantic baseline model. For real-time traffic feature vectors labeled as abnormal samples, skip this incremental update and update the model parameters of the global baseline model and the business semantic baseline model and the corresponding upper bound of the dynamic confidence interval.

2. The method according to claim 1, characterized in that, S1 includes: S11: Collect network traffic data, perform session reassembly processing on the network traffic data according to the five-tuple, aggregate the disordered data packets into session streams according to the five dimensions of source IP address, destination IP address, source port, destination port and protocol type, and sort the data packets in each session stream according to the timestamp of each data packet in time sequence to generate session stream data containing timestamp sequence, data packet size sequence, protocol type and direction flag; S12: The session flow data is segmented based on the sliding time window partitioning technique. The continuous network traffic is divided into several traffic segments according to a preset time window, and the session flow in each traffic segment is statistically aggregated to generate network layer statistical features and transport layer timing features in each time window. S13: Based on the pre-built business semantic feature library, perform business pattern matching processing on the multi-dimensional traffic features, compare the multi-dimensional traffic features of each traffic segment with the feature fingerprints of each business pattern in the business semantic feature library, assign corresponding business type labels to each traffic segment according to the business pattern with the highest matching score, and generate a traffic feature vector sequence carrying business type labels.

3. The method according to claim 1, characterized in that, S2 includes: S21: Based on the traffic feature vector sequence carrying the business type label, all traffic feature vectors are used as training samples in a way that does not distinguish between business types. The LSTM autoencoder is trained and processed. The encoder compresses the input sequence into a hidden state vector and then reconstructs it into the original sequence through the decoder. The training objective is to minimize the reconstruction error and generate a global baseline model that does not distinguish between business types. S22: Based on each service type label in the traffic feature vector sequence carrying service type labels, train independent LSTM autoencoders for each subset of traffic feature vectors corresponding to each service type label. The LSTM autoencoder corresponding to each service type label uses only the traffic feature vector under that service type label as the training sample, and takes minimizing the reconstruction error on each service subset as the training objective to generate multiple service semantic baseline models corresponding to each service type label. S23: Based on the subset of traffic feature vectors corresponding to each business type label and their corresponding business semantic baseline models, calculate the reconstruction error of each traffic feature vector on each business semantic baseline model, sort them in ascending order of reconstruction error, select a preset number of samples with the smallest reconstruction error, store them in the long-term memory buffer of the corresponding business type label, and generate a dual-branch dynamic baseline model that includes the global baseline model, the multiple business semantic baseline models and the long-term memory buffer.

4. The method according to claim 1, characterized in that, S3 includes: S31: Perform business type label matching processing on the acquired real-time traffic feature vector, determine the business type label corresponding to the real-time traffic feature vector according to the pre-built business semantic feature library, and input the real-time traffic feature vector into the global baseline model and the business semantic baseline model corresponding to the real-time traffic feature vector for reconstruction processing, and obtain the global reconstruction error and the business semantic reconstruction error respectively. S32: Based on the reconstruction error sequence of the most recent samples judged as normal within the sliding window, perform real-time sliding update processing on the upper bound of the confidence interval of the global baseline model and the upper bound of the confidence interval of the business semantic baseline model, respectively, and calculate the upper bound of the global dynamic confidence interval and the upper bound of the business semantic dynamic confidence interval at the current moment. S33: Calculate a first deviation ratio based on the business semantic reconstruction error and the upper bound of the business semantic dynamic confidence interval, calculate a second deviation ratio based on the global reconstruction error and the upper bound of the global dynamic confidence interval, and perform a weighted summation of the first deviation ratio and the second deviation ratio using the matching confidence of the business semantic baseline model as the weight to generate a comprehensive anomaly score.

5. The method according to claim 1, characterized in that, S4 includes: S41: Calculate the baseline health index of the current business type label based on the proportion of the number of samples judged as abnormal in the sliding window to the total number of samples in the sliding window. When the baseline health index is continuously lower than the preset health lower bound, it is determined that the baseline model of the business type label has a risk of contamination. S42: When the business semantic reconstruction error exceeds the upper bound of the business semantic dynamic confidence interval and the comprehensive anomaly score does not exceed the preset anomaly judgment threshold, a potential pollution alarm is triggered. Based on the triggering frequency of the potential pollution alarm and the preset attenuation coefficient, the pollution risk coefficient of the current business type label is calculated using an incremental method. The potential contamination alarm is used to indicate that the business semantic baseline model has been gradually eroded by abnormal traffic, its reconstruction error for abnormal samples tends to decrease, and the upper bound of the confidence interval is continuously raised, indicating a contamination state. S43: Based on the comparison results of the comprehensive anomaly score and the preset anomaly judgment threshold, and the comparison results of the pollution risk coefficient and the preset pollution risk threshold, perform incremental update decision processing on the real-time traffic feature vector, mark samples whose comprehensive anomaly score does not exceed the anomaly judgment threshold and whose pollution risk coefficient is lower than the pollution risk threshold as safe and normal samples, mark samples whose pollution risk coefficient is not lower than the pollution risk threshold as high-risk pollution samples and store them in the isolation buffer, and mark samples whose comprehensive anomaly score exceeds the anomaly judgment threshold as abnormal samples, and generate a marking result containing safe and normal samples, high-risk pollution samples and abnormal samples.

6. The method according to claim 5, characterized in that, The formula for calculating the pollution risk coefficient is as follows: Where t represents the current time and l represents the current business type label. The pollution risk coefficient at the current moment. The pollution risk coefficient at the previous moment. The error is the reconstruction error of the business semantics. This is the upper bound of the dynamic confidence interval for the business semantics. For the comprehensive anomaly score, The preset anomaly detection threshold is... The preset pollution increment coefficient, The preset attenuation coefficient, This is an indicator function that takes the value 1 when the condition inside the parentheses is true and 0 otherwise.

7. The method according to any one of claims 1-6, characterized in that, S5 includes: S51: Based on the labeling results, for real-time traffic feature vectors labeled as safe and normal samples, determine whether the business semantic reconstruction error of the real-time traffic feature vector exceeds the upper bound of the dynamic confidence interval of the business semantics and whether the pollution risk coefficient of the real-time traffic feature vector is lower than the preset pollution risk threshold. If the business semantic reconstruction error does not exceed the upper bound of the dynamic confidence interval of the business semantics and the pollution risk coefficient is lower than the preset pollution risk threshold, then the real-time traffic feature vector is determined to be an effective sample that can be used for updating the business semantic baseline model. S52: Randomly sample a preset number of historical samples from the long-term memory buffer, mix the preset number of historical samples with the effective samples, construct a training batch containing historical samples and current effective samples, and incrementally update the business semantic baseline model corresponding to the real-time traffic feature vector using the training batch, thereby constraining the incremental update direction by training the current effective samples and historical clean samples together. S53: Based on the labeling results, for real-time traffic feature vectors labeled as safe and normal samples, the real-time traffic feature vectors are used to update the global baseline model. For real-time traffic feature vectors labeled as abnormal samples, the incremental update is skipped. After the incremental update is completed, the model parameters of the global baseline model and the business semantic baseline model and the corresponding upper bound of the dynamic confidence interval are updated synchronously.

8. A network detection system based on artificial intelligence, characterized in that, The system includes: The traffic feature construction module is used to collect network traffic data, extract multi-dimensional traffic features after session reassembly of the network traffic data, and match the corresponding service type labels according to the multi-dimensional traffic features to generate a traffic feature vector sequence carrying service type labels. The dual-branch model construction module is used to train a global baseline model that does not distinguish between business types and multiple business semantic baseline models corresponding to each business type label based on the traffic feature vector sequence carrying business type labels. It also selects a preset number of samples with the smallest reconstruction error from the traffic feature vectors corresponding to each business type label and stores them in the long-term memory buffer of the corresponding business type label to generate a dual-branch dynamic baseline model that includes the global baseline model, the multiple business semantic baseline models and the long-term memory buffer. The comprehensive anomaly scoring module is used to input the acquired real-time traffic feature vector into the global baseline model and the business semantic baseline model corresponding to the real-time traffic feature vector respectively for reconstruction, to obtain the global reconstruction error and the business semantic reconstruction error, and to calculate the comprehensive anomaly score by weighting and fusing the ratios of the global reconstruction error and the business semantic reconstruction error to the upper bound of the dynamic confidence interval of the global baseline model and the business semantic baseline model respectively. The sample classification and labeling module is used to calculate the baseline health index and pollution risk coefficient of the current business type label based on the comprehensive anomaly score, the business semantic reconstruction error and the upper bound of the dynamic confidence interval, and to make incremental update decisions on the real-time traffic feature vector based on the baseline health index and the pollution risk coefficient, thereby generating a labeling result that includes safe and normal samples, high-risk pollution samples and abnormal samples. The model incremental update module is used to incrementally update the global baseline model and the business semantic baseline model based on the labeling results. For real-time traffic feature vectors labeled as safe and normal samples, if the business semantic reconstruction error does not exceed the upper bound of the dynamic confidence interval and the pollution risk coefficient is lower than the preset pollution risk threshold, the real-time traffic feature vectors are mixed with historical samples sampled from the long-term memory buffer and used to update the corresponding business semantic baseline model. For real-time traffic feature vectors labeled as abnormal samples, the incremental update is skipped, and the model parameters of the global baseline model and the business semantic baseline model and the corresponding upper bound of the dynamic confidence interval are updated.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images