A network information operation and maintenance system based on AI multimodality

Abstract

This invention provides an AI-based multimodal network information operation and maintenance system, belonging to the field of network information operation and maintenance technology. The system collects heterogeneous operation and maintenance data such as network traffic, device status, audio alarms, and thermal imaging through a data acquisition unit. It utilizes a multimodal feature extraction module to extract various features in parallel, forming a unified multidimensional feature vector. A hierarchical anomaly detection architecture is established, deploying lightweight and deep anomaly detection models at the edge and cloud respectively. A multimodal feature fusion model intelligently fuses different modal features through an attention mechanism. An adaptive threshold dynamically adjusts the Bayesian algorithm to optimize the detection threshold in real time based on network status. A multimodal fusion decision engine is constructed to weight and fuse anomaly detection results and dynamically adjust modal weights, thus solving the technical problem of insufficient accuracy in multimodal operation and maintenance data fusion processing.

Description

Technical Field

[0001] This invention belongs to the field of network information operation and maintenance technology, and more specifically, relates to a network information operation and maintenance system based on AI multimodality. Background Technology

[0002] Network information operation and maintenance (O&M) systems are a key technology for ensuring the stable operation of large-scale network infrastructure. Traditional O&M systems mainly rely on single-modal monitoring data for anomaly detection. They collect O&M information by deploying network traffic monitoring equipment, log analysis tools, and status monitoring sensors, and then use rule-based anomaly detection algorithms or simple machine learning models for fault prediction and diagnosis. These systems are widely used in enterprise data centers, cloud computing platforms, and communication network O&M. However, traditional single-modal O&M systems have significant shortcomings when dealing with complex network environments. Because they rely on only a single type of monitoring data, they cannot comprehensively reflect the true operating status of network devices, resulting in limited anomaly detection accuracy and a high false alarm rate. Furthermore, they lack effective multi-source information fusion mechanisms, making it difficult to identify complex fault modes across devices and levels. In today's environment of ever-expanding network scale and increasing business complexity, the lack of intelligent multi-modal data fusion processing capabilities makes it difficult for traditional O&M systems to accurately identify and predict complex network anomalies. In other words, existing technologies suffer from insufficient accuracy in multi-modal O&M data fusion processing. Summary of the Invention

[0003] In view of this, the present invention provides a network information operation and maintenance system based on AI multimodality, which can solve the technical problem of insufficient accuracy in multimodal operation and maintenance data fusion and processing in the prior art.

[0004] This invention is implemented as follows: It provides an AI-based multimodal network information operation and maintenance system. A multimodal data acquisition unit is deployed on network devices to collect multimodal operation and maintenance data, including network traffic data, device status logs, audio alarm signals, and device thermal imaging images. Preliminary preprocessing is performed through edge computing nodes. The collected multimodal operation and maintenance data is input into a multimodal feature extraction module, which extracts network traffic temporal features, text semantic features, audio spectrum features, and image visual features through a parallel processing architecture, forming a unified multidimensional feature vector. A hierarchical anomaly detection architecture is established, deploying a lightweight anomaly detection model on the edge for real-time preliminary judgment, and deploying a deep operation and maintenance anomaly identification model in the cloud for in-depth analysis. A multimodal feature fusion model is used to process the unified multidimensional feature vector. In the fusion process, the attention weight allocation parameters in the multimodal feature fusion model are determined based on network latency parameters, anomaly confidence parameters, and modality data quality parameters, outputting a fused feature vector. An adaptive threshold dynamic adjustment Bayesian algorithm is used to detect anomalies in the fused feature vector, and the anomaly detection threshold is dynamically adjusted through a Bayesian inference mechanism. A multimodal fusion decision engine is established, which performs weighted fusion of anomaly detection results from different modalities through an attention weight allocation mechanism. When the anomaly amplitude value of a single modality is within the range of [0.3, 0.6], the multimodal fusion continues to operate normally. When the anomaly amplitude value of a single modality is within the range of (0.6, 0.8], the weight of that single modality is reduced to 60% of the rated weight. When the anomaly amplitude value of a single modality exceeds 0.8, the single modality immediately stops participating in the fusion process.

[0005] Specifically, the multimodal data acquisition unit includes a network traffic monitoring sensor, a device status log collector, an audio signal collector, and a thermal imaging camera, used to collect heterogeneous data sources in the network operation and maintenance environment in real time.

[0006] The edge computing node is specifically a lightweight computing device deployed at the network edge, used to preprocess and perform preliminary analysis on raw multimodal operation and maintenance data, reducing data transmission latency and bandwidth consumption.

[0007] The multimodal feature extraction module adopts a parallel processing architecture, which processes different types of input data simultaneously and extracts key feature information of their respective domains.

[0008] The network traffic time-series characteristics refer to the time-series characteristics extracted from the network data packet transmission pattern, including traffic fluctuation period, peak distribution, and transmission delay pattern.

[0009] The text semantic features refer to the semantic information extracted from the device status log text, including keyword weights, semantic relevance, and abnormal event descriptions.

[0010] The audio spectrum features refer to the frequency domain features extracted from the audio of the device in operation, including spectral energy distribution, harmonic components, and noise features.

[0011] The image visual features refer to the visual features extracted from the thermal imaging images of the device, including temperature distribution patterns, hot spots, and temperature change trends.

[0012] The unified multidimensional feature vector refers to a high-dimensional vector representation formed by concatenating different modal features after standardization.

[0013] The hierarchical anomaly detection architecture adopts an edge-cloud collaborative deployment mode, deploying a lightweight anomaly detection model with low computational complexity on the edge side and a high-precision deep operation and maintenance anomaly identification model on the cloud.

[0014] The deep operation and maintenance anomaly identification model uses a multi-level attention mechanism to process multimodal operation and maintenance data. The beam search parameters in the deep operation and maintenance anomaly identification model are determined based on network latency parameters and anomaly confidence parameters.

[0015] The multimodal feature fusion model is used to intelligently fuse feature vectors from different modalities, thereby improving the accuracy and robustness of anomaly detection.

[0016] The adaptive threshold dynamic adjustment Bayesian algorithm refers to a threshold adaptive adjustment mechanism based on Bayesian inference, which dynamically optimizes the anomaly detection threshold according to historical data and the current network state.

[0017] The multimodal fusion decision engine uses an attention mechanism to adaptively weight the detection results of different modalities, and dynamically allocates fusion weights based on the reliability and relevance of each modal data.

[0018] The attention weight allocation mechanism automatically adjusts the contribution weight of different modalities in the final decision by calculating the correlation score between each modality feature and the abnormal pattern.

[0019] The process includes establishing an anomaly root cause analysis module after the multimodal fusion decision engine step. This module uses a causal reasoning algorithm to perform in-depth analysis on the anomaly detection results output by the multimodal fusion decision engine. By constructing a device dependency graph and an event time-series correlation matrix, the module identifies and outputs the root causes and propagation paths of the abnormal events.

[0020] This invention addresses the technical problem of insufficient accuracy in multimodal operation and maintenance data fusion processing by constructing a multimodal feature extraction module, a hierarchical anomaly detection architecture, and a multimodal fusion decision engine. The invention employs a parallel processing architecture to simultaneously extract network traffic temporal features, text semantic features, audio spectral features, and image visual features. Through a unified multidimensional feature vector representation and attention weight allocation mechanism, it achieves intelligent fusion of data from different modalities, effectively overcoming the shortcomings of incomplete information acquisition in traditional single-modal systems and significantly improving the accuracy and robustness of anomaly detection. The adaptive threshold dynamic adjustment Bayesian algorithm established in this invention can automatically optimize detection parameters based on changes in network state. Combined with an edge-cloud collaborative hierarchical anomaly detection architecture, it achieves a balance between real-time performance and accuracy, thus solving the core technical problem of insufficient accuracy in multimodal operation and maintenance data fusion processing. Attached Figure Description

[0021] Figure 1 This is a flowchart of the system execution steps of the present invention.

[0022] Figure 2 This is a schematic diagram of the adaptive threshold dynamic adjustment process in the embodiment.

[0023] Figure 3 This is a multimodal fusion decision weight allocation diagram in the embodiment. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0025] like Figure 1 The diagram shown is a flowchart of a network information operation and maintenance system based on AI multimodality provided by this invention. The specific steps performed by this system are as follows:

[0026] S01. Deploy multimodal data acquisition units on network devices to collect multimodal operation and maintenance data, including network traffic data, device status logs, audio alarm signals, and device thermal imaging images, and perform preliminary preprocessing through edge computing nodes;

[0027] S02. Input the collected multimodal operation and maintenance data into the multimodal feature extraction module, and extract network traffic temporal features, text semantic features, audio spectrum features and image visual features through a parallel processing architecture to form a unified multidimensional feature vector;

[0028] S03. Establish a hierarchical anomaly detection architecture. Deploy a lightweight anomaly detection model on the edge for real-time preliminary judgment, and deploy a deep operation and maintenance anomaly identification model on the cloud for in-depth analysis. When the network latency parameter value is in the range [10, 50], reduce the cloud processing frequency to 70% of the original frequency; when the network latency parameter value is in the range (50, 100], maintain the current cloud processing frequency; when the network latency parameter value is in the range (100, 200], increase the edge processing frequency to 150% of the original frequency.

[0029] S04. A multimodal feature fusion model is used to fuse the unified multidimensional feature vector. The attention weight allocation parameter in the multimodal feature fusion model is determined based on the network latency parameter, the anomaly confidence parameter, and the modal data quality parameter, and the fused feature vector is output.

[0030] S05. An adaptive threshold dynamic adjustment Bayesian algorithm is used to detect anomalies in the fused feature vector. The anomaly detection threshold is dynamically adjusted through the Bayesian inference mechanism. When the anomaly confidence change rate exceeds 15% / second and the duration is greater than 3 seconds, a fast response adjustment is immediately executed; if the anomaly confidence change rate exceeds 15% / second but the duration is less than 3 seconds, a slow response adjustment is executed.

[0031] S06. Establish a multimodal fusion decision engine, and perform weighted fusion of anomaly detection results of different modalities through an attention weight allocation mechanism. When the single-modal anomaly amplitude value is within the range of [0.3, 0.6], maintain the normal operation of multimodal fusion; when the single-modal anomaly amplitude value is within the range of (0.6, 0.8], reduce the single-modal weight to 60% of the rated weight; when the single-modal anomaly amplitude value exceeds 0.8, immediately stop the single-modal from participating in the fusion operation.

[0032] S07. Optionally, it also includes establishing an anomaly root cause tracing analysis module, using a causal relationship reasoning algorithm to perform in-depth analysis on the anomaly detection results output by the multimodal fusion decision engine, and identifying and outputting the root cause and propagation path of the abnormal event by constructing a device dependency graph and an event time sequence correlation matrix.

[0033] The multimodal data acquisition unit includes a network traffic monitoring sensor, a device status log collector, an audio signal acquisition device, and a thermal imaging camera, used to collect heterogeneous data sources in the network operation and maintenance environment in real time. The edge computing node is a lightweight computing device deployed at the network edge, used to preprocess and perform preliminary analysis on the raw multimodal operation and maintenance data, reducing data transmission latency and bandwidth consumption. The multimodal feature extraction module adopts a parallel processing architecture, simultaneously processing different types of input data and extracting key feature information from their respective domains. The network traffic time-series features are time-series features extracted from network data packet transmission patterns, including traffic fluctuation periods, peak distribution, and transmission delay patterns. The text semantic features are semantic information extracted from device status log text, including keyword weights, semantic relevance, and abnormal event descriptions. The audio spectrum features are frequency domain features extracted from device operating audio, including spectral energy distribution, harmonic components, and noise features. The image visual features are visual features extracted from device thermal imaging images, including temperature distribution patterns, hotspot areas, and temperature change trends. The unified multidimensional feature vector is a high-dimensional vector representation formed by concatenating different modal features after standardization. The hierarchical anomaly detection architecture adopts an edge-cloud collaborative deployment mode, deploying a lightweight anomaly detection model with low computational complexity on the edge side and a high-precision deep operation and maintenance anomaly identification model on the cloud.

[0034] The deep O&M anomaly detection model employs a multi-level attention mechanism to process multimodal O&M data. The beam search parameters in this model are determined based on network latency and anomaly confidence parameters. The model's structure is a Transformer-based multimodal anomaly detection network, comprising a multimodal encoding layer, a cross-modal attention computation layer, a temporal modeling layer, and an anomaly classification output layer. The multimodal encoding layer independently encodes network traffic temporal features, text semantic features, audio spectral features, and image visual features. The cross-modal attention computation layer calculates the correlation between different modalities and performs feature alignment using a self-attention mechanism. The temporal modeling layer uses a temporal convolutional network to capture the temporal dependencies of O&M data. The anomaly classification output layer maps the fused features to anomaly detection results and confidence scores. The entire model supports dynamic adjustment of the beam search parameters to balance detection accuracy and inference speed. The steps for establishing the training dataset for the deep operation and maintenance anomaly identification model include: collecting multimodal operation and maintenance data samples containing normal operation status and various abnormal statuses from a large-scale network operation and maintenance environment; performing data cleaning and format standardization on the collected raw multimodal operation and maintenance data to ensure data quality consistency; establishing a multi-level supervised learning label system by finely classifying and labeling the samples according to the severity and impact range of the anomaly type; expanding the number of training samples and balancing the sample distribution of different anomaly categories using data augmentation techniques such as time window sliding, data interpolation, and noise injection; and dividing the processed multimodal operation and maintenance data into training set, validation set, and test set according to both time order and anomaly type to ensure the temporal integrity and category representativeness of the dataset. The training steps of the deep operation and maintenance anomaly detection model include: initializing the parameter weights of the multimodal coding layer using a Transformer encoder pre-trained on large-scale network data; training the entire anomaly detection network using a composite loss function including classification loss and attention regularization terms; updating parameters using the Adam optimization algorithm; employing early stopping mechanism, learning rate cosine annealing scheduling, and gradient pruning techniques during training to prevent model overfitting and improve training stability; evaluating model performance using anomaly detection precision, recall, and F1 score on the validation set; fine-tuning the hyperparameter configuration of the beam search parameters and attention mechanism using Bayesian optimization; and finally validating the model's generalization ability and deployment effect in actual operation and maintenance environments on a test set containing real network fault scenarios.

[0035] The multimodal feature fusion model is used to intelligently fuse feature vectors from different modalities, improving the accuracy and robustness of anomaly detection. The model is structured as a multi-level attention fusion network, including an intra-modal feature encoding layer, a cross-modal attention computation layer, and a fusion decision output layer. The intra-modal feature encoding layer performs deep encoding and dimensionality reduction on the original features of each modality. The cross-modal attention computation layer calculates the correlation weights between different modalities and performs feature alignment through a self-attention mechanism. The fusion decision output layer maps the weighted fused multimodal features into a fused feature vector. The entire model employs an end-to-end training approach and supports dynamic adjustment of attention weight allocation parameters during the inference phase. The steps for establishing the training dataset for the multimodal feature fusion model include: collecting multimodal operation and maintenance data samples containing normal and abnormal states from a real network operation and maintenance environment; cleaning and standardizing the collected raw data to ensure data quality and consistency; classifying and labeling samples according to the anomaly type and severity to establish supervised learning labels; using data augmentation techniques to expand the number of training samples and balance the distribution of samples from different categories; dividing the processed data into training, validation, and test sets according to time order and ensuring the temporal integrity of the data. The steps for training the multimodal feature fusion model include: initializing the parameter weights of each modality coding layer using a pre-trained single-modality feature extractor; training the entire fusion network using a cross-entropy loss function with a regularization term and optimizing the model parameters using a gradient descent algorithm; employing an early stopping mechanism and learning rate scheduling strategy during training to prevent overfitting and improve convergence speed; evaluating model performance using a validation set and tuning hyperparameter configuration using a grid search method; and finally verifying the model's generalization ability and applicability to real-world operation and maintenance scenarios on the test set.

[0036] The adaptive threshold dynamic adjustment Bayesian algorithm is a threshold adaptive adjustment mechanism based on Bayesian inference. It dynamically optimizes the anomaly detection threshold according to historical data and the current network state. The algorithm maintains the prior distribution of parameters, continuously updates the posterior probability through observed data, uses Bayesian factors to evaluate anomaly confidence, and combines a sliding window mechanism to handle time-series characteristics, achieving intelligent threshold adjustment and continuous improvement of anomaly detection accuracy. In network information operation and maintenance systems, this adaptive threshold dynamic adjustment Bayesian algorithm effectively solves the problem that traditional fixed thresholds cannot adapt to dynamic changes in the network environment. Through a continuous learning mechanism, it automatically adjusts the sensitivity of anomaly detection, significantly improving the accuracy of anomaly detection and reducing the false alarm rate. By maintaining the Bayesian prior distribution of network state parameters and updating the posterior probability distribution based on new observed data, the algorithm can accurately capture periodic changes in network operation modes and sudden anomaly events. It uses Bayesian factors to quantify anomaly confidence and combines a sliding window mechanism to handle the correlation of time-series data, ensuring that the threshold adjustment process maintains statistical rigor while possessing real-time response capabilities. Therefore, it provides a theoretically sound and highly practical solution for anomaly detection in multimodal operation and maintenance data. The algorithm takes network delay distribution parameters, traffic fluctuation variance parameters, equipment temperature statistics parameters, and historical anomaly frequency parameters as inputs, calculates the posterior probability distribution using Bayes' theorem, obtains the optimal threshold parameters through maximum a posteriori estimation, and outputs dynamically adjusted anomaly detection thresholds and anomaly confidence parameters.

[0037] The multimodal fusion decision engine employs an attention mechanism to adaptively weight the detection results of different modalities, dynamically allocating fusion weights based on the reliability and relevance of each modality's data. This attention weight allocation mechanism automatically adjusts the contribution ratio of different modalities in the final decision by calculating the correlation score between each modality's features and the abnormal pattern. The distributed storage architecture combines the efficient time-series processing capabilities of time-series databases, the large-capacity file management capabilities of object storage, and the relational query capabilities of graph databases, forming a unified management platform for heterogeneous data. The data layered compression technology performs layered storage based on data access frequency and importance; hot data uses high-speed storage media, while cold data uses compressed storage, achieving a balance between storage cost and access performance. The causal relationship inference algorithm uses a directed acyclic graph to model the dependencies between devices, determining causal links through event correlation analysis within a time window. The device dependency graph is a directed graph model built based on network topology and business flow, where nodes represent network devices and edges represent dependency strength. The event temporal correlation matrix records the co-occurrence frequency and temporal relationship of various abnormal events within different time windows.

[0038] The anomaly detection threshold calculation equation set includes a Bayesian update equation, a confidence assessment equation, and a threshold optimization equation. The Bayesian update equation updates the posterior distribution of parameters based on new observation data. Inputs include prior distribution parameters, observed data samples, likelihood function parameters, time weighting factors, and historical data window size; output is the updated posterior distribution parameters. The confidence assessment equation calculates the confidence level for anomaly detection. Inputs include the posterior distribution mean, posterior distribution variance, current observation value, statistical significance level, and Bayesian factor threshold; output is the anomaly confidence parameter. The threshold optimization equation determines the optimal anomaly detection threshold. Inputs include the anomaly confidence parameter, historical false alarm rate, detection latency requirements, system resource constraints, and business importance weights; output is the optimized anomaly detection threshold.

[0039] The beam search parameter adjustment function is used to dynamically adjust the beam search parameters of the deep operation and maintenance anomaly identification model according to the network status. The beam search parameter adjustment function calculates the beam search adjustment coefficient based on the network latency parameter and the anomaly confidence parameter. When the beam search adjustment coefficient b∈[0.2, 0.5), the beam search parameter is adjusted to 0.6 times the original parameter in a linear decreasing manner to accelerate the inference process. When the beam search adjustment coefficient b∈[0.5, 0.8), the current beam search parameter is kept unchanged to maintain detection accuracy. When the beam search adjustment coefficient b∈[0.8, 1.0], the beam search parameter is increased to 1.4 times the original parameter in an exponential growth manner to improve detection accuracy.

[0040] The attention weight allocation parameter adjustment function is used to dynamically adjust the attention weight allocation parameters of the multimodal feature fusion model according to the network state. The attention weight allocation parameter adjustment function calculates the attention adjustment coefficient based on the network latency parameter, the anomaly confidence parameter and the modal data quality parameter. When the attention adjustment coefficient a∈[0.3, 0.7), the attention weight allocation parameter is adjusted to 0.8 times the original parameter using linear interpolation. When the attention adjustment coefficient a∈[0.7, 0.9), the current attention weight allocation parameter remains unchanged. When the attention adjustment coefficient a∈[0.9, 1.0], the attention weight allocation parameter is enhanced to 1.2 times the original parameter using exponential decay.

[0041] The specific implementation methods of the above steps are described in detail below.

[0042] The specific implementation of step S01 involves deploying a multimodal data acquisition unit at key nodes of the network device. This unit comprises four types of sensing devices: a network traffic monitoring sensor, a device status log collector, an audio signal collector, and a thermal imaging camera. The network traffic monitoring sensor uses deep packet inspection technology to capture network data packets in real time, with a sampling frequency of 1000 times / second and a data cache capacity of 512MB. Data transmission is triggered when the cache utilization exceeds 80%. The device status log collector periodically collects device operation logs based on a system call interception mechanism, with a sampling interval of 10 seconds and a log file size threshold of 100MB. The audio signal collector captures device operation audio through a high-precision microphone array, with a sampling frequency of 48kHz, a quantization bit depth of 16 bits, and a frequency response range of 20Hz to 20kHz. The thermal imaging camera uses an uncooled micro-calorimeter detector to acquire thermal images of the device, with a resolution of 640×480 pixels, a temperature measurement accuracy of ±2℃, and a frame rate of 30fps. Lightweight data preprocessing algorithms are deployed on edge computing nodes. A sliding window average filtering algorithm is used to remove data noise, with the window size set to 50 sample points. At the same time, a data compression algorithm is used to compress the original data to 30% of its original volume to reduce transmission bandwidth.

[0043] The specific implementation of step S02 involves establishing a multimodal feature extraction module, employing a parallel processing architecture to simultaneously process four different types of input data. Network traffic temporal feature extraction utilizes a Long Short-Term Memory (LSTM) network algorithm. The network structure contains 128 hidden units, with a time step of 50 and a learning rate of 0.001. Temporal features such as traffic fluctuation cycles, peak distributions, and transmission delay patterns are extracted by analyzing the transmission patterns of network data packets. Text semantic feature extraction is based on a pre-trained Chinese language model with 768-dimensional word vectors and a maximum sequence length of 512 characters. An attention mechanism is used to calculate keyword importance, and semantic information, including keyword weights, semantic relevance, and abnormal event descriptions, is extracted from device status logs. Audio spectrum feature extraction employs a Fast Fourier Transform (FFT) algorithm with 2048 transform points. A Hamming window is used as the window function, with an overlap rate of 50%, extracting frequency domain information such as spectral energy distribution, harmonic components, and noise characteristics. Image visual feature extraction is based on a deep convolutional neural network with a depth of 50 layers, a kernel size of 3×3, and a pooling window size of 2×2. Visual features such as temperature distribution patterns, hotspot regions, and temperature change trends are extracted through a multi-scale feature pyramid. All feature vectors are standardized and then concatenated to form a unified multi-dimensional feature vector, with a final feature vector dimension of 2048.

[0044] The specific implementation of step S03 involves constructing a hierarchical anomaly detection architecture, deploying a lightweight anomaly detection model on the edge side, and deploying a deep operational anomaly identification model on the cloud side. The lightweight edge model uses a support vector machine algorithm, with a radial basis function kernel, a regularization parameter C set to 1.0, a kernel parameter γ set to 0.001, a model size controlled within 10MB, and an inference time controlled within 100 milliseconds. The cloud-based deep model is based on a transformer architecture, with 12 encoder layers, 8 attention heads, a 768-dimensional hidden unit dimension, and a 3072-dimensional feedforward network dimension, resulting in approximately 110 million parameters. Network latency parameter monitoring uses a round-trip time (RTD) measurement algorithm, with a measurement interval of 1 second and a statistical window size of 60 seconds. When the network latency parameter value is between 10 and 50 milliseconds, the cloud processing frequency is reduced from 10 times per second to 7 times per second, a reduction of 30%. When the network latency parameter value is between 50 and 100 milliseconds, the cloud processing frequency remains unchanged. When the network latency parameter value is between 100 milliseconds and 200 milliseconds, the edge processing frequency increases from the original 5 times per second to 7.5 times per second, an increase of 50%. When the latency parameter exceeds 200 milliseconds, pure edge mode is activated, and cloud collaborative processing is suspended.

[0045] The specific implementation of step S04 involves using a multimodal feature fusion model to fuse a unified multidimensional feature vector. This model is based on a multi-level attention fusion network architecture. The intramodal feature encoding layer employs a fully connected neural network, with each modality corresponding to an independent encoder. The network has three layers with 1024, 512, and 256 hidden units respectively. The activation function is a modified linear unit function, and the dropout rate is set to 0.2. The cross-modal attention calculation layer is based on a self-attention mechanism. The query matrix, key matrix, and value matrix are all 256-dimensional, and there are eight attention heads. Feature alignment is achieved by calculating the correlation weights between different modalities. The fusion decision output layer maps the weighted fused multimodal features into a 512-dimensional fused feature vector. The attention weight allocation parameters are dynamically determined based on network latency parameters, anomaly confidence parameters, and modal data quality parameters, with a weight update frequency of once every 10 seconds. The network latency parameter accounts for 40% of the weight, the anomaly confidence parameter accounts for 35%, and the modal data quality parameter accounts for 25%. When the data quality score of a certain modality is lower than 0.6, the weight of that modality is automatically reduced to 50% of its original weight.

[0046] The specific implementation of step S05 involves using an adaptive threshold dynamic adjustment Bayesian algorithm to detect anomalies in the fused feature vector. This algorithm dynamically adjusts the anomaly detection threshold based on a Bayesian inference mechanism. The algorithm maintains the prior distribution of network state parameters, using a β distribution as the prior, with initial values ​​for shape parameter α set to 2.0 and shape parameter β set to 5.0. The posterior probability distribution is continuously updated using observed data, and the property of conjugate priors simplifies computational complexity. The Bayesian factor is calculated using the likelihood ratio test method, with the null hypothesis being the normal network state and the alternative hypothesis being the abnormal network state. The Bayesian factor threshold is set to 3.0. The sliding window mechanism has a time window size of 300 seconds and a sliding step size of 30 seconds. Historical data weights are calculated using an exponential decay method, with a decay factor of 0.95. The anomaly confidence change rate monitoring uses a differential calculation method. When the change rate exceeds 15% / second and the duration is greater than 3 seconds, a fast response adjustment mechanism is triggered, increasing the threshold adjustment step size to twice the original step size. When the rate of change exceeds 15% / second but the duration is less than 3 seconds, the slow response adjustment mechanism is triggered, and the threshold adjustment step size remains unchanged.

[0047] The specific implementation of step S06 involves establishing a multimodal fusion decision engine, which weights and fuses anomaly detection results from different modalities using an attention weight allocation mechanism. The decision engine employs a weighted voting algorithm, with each modality corresponding to a voting weight, and the total weights are normalized to 1.0. The single-modal anomaly magnitude value is calculated using the z-score normalization method, calculating the standard deviation multiple between the current observation and the historical mean. When the single-modal anomaly magnitude value is between 0.3 and 0.6, the modality weight remains unchanged, and the multimodal fusion system maintains normal operation. When the single-modal anomaly magnitude value is between 0.6 and 0.8, the modality weight is reduced to 60% of the rated weight, with linear interpolation used for weight adjustment and an adjustment time constant of 5 seconds. When the single-modal anomaly magnitude value exceeds 0.8, the modality immediately stops participating in the fusion process, its weight is set to 0, and a modality isolation mechanism is activated to prevent anomaly propagation. Weight redistribution uses a softmax normalization algorithm to ensure that the total weight of the remaining modalities is 1.0. The fusion decision output adopts a weighted average method, and the final anomaly score is the weighted average of the anomaly scores of each modality.

[0048] The specific implementation of step S07 involves establishing an anomaly root cause tracing analysis module and using a causal reasoning algorithm to perform in-depth analysis of the anomaly detection results output by the multimodal fusion decision engine. The device dependency graph is constructed based on a network topology discovery algorithm, using a simple network management protocol to obtain device connection relationships. Graph nodes represent network devices, edges represent dependency strength, and edge weights are calculated based on the traffic transmission volume and communication frequency between devices. The event time-series correlation matrix is ​​established using a sliding time window statistical method, with a time window size of 1800 seconds and a statistical interval of 60 seconds. Matrix element values ​​represent the co-occurrence frequency of different anomaly events within the time window. The causal reasoning algorithm is based on the Granger causality test principle, determining the direction of causality by analyzing the lag correlation of time-series data. The lag order is set to 5, and the significance level is set to 0.05. The anomaly propagation path calculation uses a shortest path algorithm, combining the topology and edge weight information of the device dependency graph to find the most probable propagation path from the root cause device to the affected devices. The root cause localization algorithm employed the PageRank ranking algorithm, with 100 iterations and a damping factor of 0.85. The most likely root cause device was determined based on node importance scores. The source analysis results included four parts: root cause device identification, anomaly type classification, impact scope assessment, and propagation path description.

[0049] The specific implementation of the deep operation and maintenance anomaly identification model training method involves end-to-end training using a large-scale multimodal operation and maintenance dataset. The dataset contains 1.5 million normal samples and 450,000 anomaly samples, covering 12 anomaly types including network congestion, equipment failure, and security intrusion. Data preprocessing employs standardization and normalization techniques. Time-series data is segmented using a sliding window method with a window size of 100 time steps and an overlap rate of 20%. Model initialization uses pre-trained parameters, and the multimodal encoding layer uses transformer encoder weights pre-trained on large-scale network data. The composite loss function includes a classification loss term and an attention regularization term; the classification loss uses cross-entropy loss, and the regularization coefficient is set to 0.01. The optimization algorithm uses the Adam algorithm, with an initial learning rate of 0.0001 and a β value of... The parameter is set to 0.9, β The parameter is set to 0.999. An early stopping mechanism is used during training to prevent overfitting; training stops when the validation set performance shows no improvement for 10 consecutive epochs. A cosine annealing strategy is used for learning rate scheduling, with a minimum learning rate of 1% of the initial learning rate. The gradient clipping threshold is set to 1.0 to prevent gradient explosion.

[0050] The specific implementation of the multimodal feature fusion model training is based on a training strategy combining contrastive learning and supervised learning. The training dataset contains 800,000 multimodal samples, divided into training, validation, and test sets in an 8:1:1 ratio. The contrastive learning part uses the InfoNCE loss function with a temperature parameter set to 0.07 and 256 negative samples. The supervised learning part uses the cross-entropy loss function with L2 regularization, with a regularization coefficient set to 0.001. Model parameters are initialized using the Xavier initialization method, with the bias term initialized to 0. The optimization algorithm uses the AdamW algorithm with a weight decay coefficient set to 0.01, and a linear warm-up strategy with 1000 steps. The training batch size is set to 32, and the total training epochs are 50. Data augmentation techniques include time warp, frequency domain masking, and Gaussian noise injection, with augmentation probabilities set to 0.3, 0.2, and 0.4, respectively.

[0051] The specific implementation of the adaptive threshold dynamic adjustment Bayesian algorithm is based on a parameter update mechanism established by Bayesian statistical inference theory. The algorithm input parameters include network latency distribution parameters, traffic fluctuation variance parameters, equipment temperature statistical parameters, and historical anomaly frequency parameters. The Bayesian update equation simplifies calculation using a conjugate prior distribution; the conjugate prior of the normal distribution parameters is a normal-gamma distribution, and the initial hyperparameter values ​​are set based on empirical estimates from historical data. The confidence assessment equation is calculated based on the confidence interval of the posterior distribution, with a confidence level set at 95%. Observations falling outside the confidence interval are considered anomalies. The threshold optimization equation combines business importance weights and system resource constraints, using the Lagrange multiplier method to solve for the optimal threshold parameters. Constraints include a false positive rate cap of 10% and a false negative rate cap of 5%. The algorithm output parameters include dynamically adjusted anomaly detection thresholds and anomaly confidence parameters, updated every 30 seconds.

[0052] The specific implementation of the beam search parameter adjustment function is based on the inference parameters of the network state dynamic optimization model. The function inputs are the network latency parameter and the anomaly confidence parameter, and the beam search adjustment coefficient is calculated through a weighted combination. The weight of the network latency parameter is set to 0.6, and the weight of the anomaly confidence parameter is set to 0.4. The weight coefficients are determined based on historical performance data statistics. When the adjustment coefficient is in the range of 0.2 to 0.5, the beam search parameter is adjusted to 0.6 times the original parameter, using a linear decreasing adjustment method with an adjustment step size of 0.02. When the adjustment coefficient is in the range of 0.5 to 0.8, the beam search parameter remains unchanged. When the adjustment coefficient is in the range of 0.8 to 1.0, the beam search parameter is increased to 1.4 times the original parameter, using an exponential growth adjustment method with a base of 1.05 and an adjustment frequency of once every 5 seconds.

[0053] The specific implementation of the attention weight allocation parameter adjustment function is based on dynamically optimizing the attention mechanism parameters using multi-dimensional network state information. The function input includes three dimensions: network latency parameter, anomaly confidence parameter, and modal data quality parameter. The weight allocation uses the analytic hierarchy process (AHP) to determine the importance of each parameter: network latency parameter has a weight of 0.4, anomaly confidence parameter has a weight of 0.35, and modal data quality parameter has a weight of 0.25. The attention adjustment coefficient is calculated using a weighted summation method, and standardization ensures the coefficient value is between 0 and 1. When the adjustment coefficient is in the range of 0.3 to 0.7, the attention weight parameter is adjusted to 0.8 times the original parameter using linear interpolation with 10 steps. When the adjustment coefficient is in the range of 0.7 to 0.9, the attention weight parameter remains unchanged. When the adjustment coefficient is in the range of 0.9 to 1.0, the attention weight parameter is increased to 1.2 times the original parameter using exponential decay with a decay factor of 0.98.

[0054] It should be noted that the key technical ideas of this invention are analyzed as follows. The first technical idea is a multimodal data fusion and collaborative processing mechanism. Traditional network operation and maintenance systems mainly rely on a single data source for anomaly detection, which is easily affected by data noise and single points of failure, resulting in poor detection accuracy and robustness. This invention integrates data from four different modalities—network traffic, device logs, audio signals, and thermal imaging images—to establish a multi-dimensional data representation system. By utilizing the complementarity and redundancy of different modal data, the accuracy and reliability of anomaly detection are significantly improved. The multimodal fusion mechanism can effectively overcome the limitations of single-modal data. When one modal of data is abnormal or missing, other modal data can still maintain the normal operation of the system, greatly improving the system's fault tolerance and stability.

[0055] The second technical approach is an adaptive threshold dynamic adjustment algorithm based on Bayesian inference. Traditional anomaly detection methods typically use fixed thresholds for judgment, which cannot adapt to dynamic changes in the network environment, resulting in high false positive and false negative rates. This invention uses Bayesian statistical inference theory to establish a parameter update mechanism. By maintaining the prior distribution of network state parameters and continuously updating the posterior probability distribution based on new observation data, it achieves intelligent adaptive adjustment of the anomaly detection threshold. This algorithm can accurately capture periodic changes in network operating modes and sudden anomaly events, using the statistical regularities of historical data to guide current decisions while maintaining sensitivity to new anomaly patterns, fundamentally solving the adaptability problem of fixed threshold methods.

[0056] The third technical approach is a layered anomaly detection architecture with edge-cloud collaboration. Traditional centralized anomaly detection methods suffer from high network latency, large bandwidth consumption, and poor real-time performance, making it difficult to meet the real-time monitoring needs of large-scale network environments. This invention innovatively proposes a layered processing architecture with edge-cloud collaboration. A lightweight model is deployed at the edge for real-time preliminary judgment, while a deep model is deployed in the cloud for precise analysis. The processing frequency allocation between the edge and cloud is dynamically adjusted based on network latency. This architecture fully leverages the low latency advantage of edge computing and the powerful computing advantage of cloud computing, ensuring both real-time anomaly detection and detection accuracy, significantly improving the overall system performance and user experience.

[0057] The fourth technical approach is a multimodal feature fusion and dynamic weight adjustment strategy based on an attention mechanism. Traditional multi-data source fusion methods typically employ simple weighted averaging or voting mechanisms, which cannot dynamically adjust based on real-time data quality and importance, resulting in poor fusion performance. This invention introduces an attention mechanism from deep learning, automatically adjusting the contribution weight of each modality in the fusion decision by calculating the correlation scores between different modal features and abnormal patterns, thus achieving intelligent and adaptive fusion weights. This strategy can dynamically optimize fusion parameters based on multi-dimensional information such as network state, data quality, and anomaly confidence, maximizing the information value of high-quality data while suppressing the interference of low-quality data.

[0058] The synergistic effect of these four key technological approaches forms a complete intelligent network operation and maintenance solution. Multimodal data fusion provides the system with rich information sources, the adaptive threshold adjustment algorithm ensures dynamic optimization of detection accuracy, the edge-cloud collaborative architecture guarantees the system's real-time performance and scalability, and the attention mechanism realizes intelligent information fusion. This synergy has significant advantages over traditional methods, not only greatly improving the accuracy and real-time performance of anomaly detection but also significantly reducing false alarm rates and system resource consumption. It provides efficient, reliable, and intelligent operation and maintenance support for large-scale network environments, representing the development direction of network operation and maintenance technology.

[0059] It should be noted that traditional network operation and maintenance systems often face the dilemma of balancing real-time performance and accuracy when dealing with large-scale network environments. They either employ simple and fast detection algorithms to ensure real-time response but lack accuracy, or use complex and precise analysis models but suffer from significant response latency. This invention establishes a hierarchical anomaly detection architecture, deploying a lightweight anomaly detection model at the edge for real-time preliminary judgment, and a deep operation and maintenance anomaly identification model in the cloud for precise analysis. Furthermore, it dynamically adjusts the processing frequency of the edge and cloud sides based on network latency parameters, achieving intelligent allocation of computing resources and adaptive balancing of processing load. Simultaneously, the beam search parameter adjustment function designed in this invention can dynamically adjust the inference parameters of the deep model according to network status, optimizing inference speed while ensuring detection accuracy, effectively solving the problem of balancing real-time performance and accuracy. Modern network operation and maintenance environments involve various types of monitoring devices and data sources, including network traffic data, device status logs, audio alarm signals, and infrared thermal imaging. These heterogeneous data differ significantly in format, sampling frequency, data volume, and processing requirements, making it difficult for traditional operation and maintenance systems to uniformly process and effectively utilize this multi-source heterogeneous information. This invention designs a unified multimodal data acquisition framework, employs edge computing nodes for preliminary preprocessing to standardize heterogeneous data into a unified format, and then uses a multimodal feature extraction module to process different types of data and extract corresponding feature representations, ultimately forming a unified multidimensional feature vector. This architecture not only simplifies the processing flow of heterogeneous data but also fully utilizes the complementary information from various data sources, significantly improving the system's ability to perceive and analyze complex network environments.

[0060] Specifically, the principle of this invention is as follows: The fundamental reason why this invention can solve the technical problem of insufficient accuracy in multimodal operation and maintenance data fusion processing lies in its innovative multi-level fusion processing mechanism and intelligent decision-making algorithm design. First, this invention employs a multimodal feature extraction module that simultaneously processes heterogeneous data sources such as network traffic, device logs, audio signals, and thermal imaging through a parallel processing architecture. Each modality of data obtains its optimal feature representation through the feature extraction algorithm, avoiding the problem of incomplete information in traditional single-modal systems. Second, the multimodal feature fusion model established by this invention uses a multi-level attention fusion network. Through cross-modal attention calculation layers, it automatically learns the correlation weights between different modalities, achieving intelligent alignment at the data level and effective fusion at the feature level. This mechanism can dynamically adjust the contribution weights of each modality according to the actual network state, ensuring the accuracy and reliability of the fusion results. Third, the adaptive threshold dynamic adjustment Bayesian algorithm designed in this invention maintains the prior distribution of network state parameters and continuously updates the posterior probability based on observed data, achieving intelligent adaptive adjustment of the anomaly detection threshold, effectively solving the limitation of traditional fixed thresholds that cannot adapt to dynamic changes in the network environment. Finally, the multimodal fusion decision engine constructed in this invention performs weighted fusion of detection results of different modalities through an attention weight allocation mechanism. When the data quality of a certain modality declines, its weight can be automatically reduced or even stopped from participating in the fusion, thus ensuring the stability and accuracy of the overall decision. This multi-protection mechanism ensures the high precision and strong robustness of multimodal data fusion processing.

[0061] The following provides a specific embodiment 1 of the present invention, and the specific implementation of each step in this embodiment 1 is described in detail below.

[0062] In this embodiment, the specific implementation of step S01 is the same as described above, and will not be repeated in detail here.

[0063] The specific implementation of step S02 involves establishing a multimodal feature extraction module, which extracts feature information from different types of data through a parallel processing architecture. The unified multidimensional feature vector representation is as follows:

[0064] ;

[0065] In the formula, For a unified multidimensional feature vector; This represents the temporal feature vector of network traffic. This is a text semantic feature vector; This is the audio spectrum feature vector; This is the image visual feature vector.

[0066] The parameter acquisition method is as follows: The Long Short-Term Memory (LSTM) network algorithm is used for extraction, including step 1: dividing the network traffic data into sequences of length 50 according to the time window; step 2: obtaining a 256-dimensional feature vector through encoding with a 128-hidden-unit LSM network. The extraction process uses a pre-trained language model, including step 1: segmenting and encoding the device log text; and step 2: obtaining semantic feature vectors through the output layer of a 768-dimensional pre-trained model. The Fast Fourier Transform (FFT) algorithm is used for extraction, including step 1: performing a 2048-point FFT on the audio signal; and step 2: extracting the spectral energy distribution features to form a 512-dimensional vector. The extraction is performed using a convolutional neural network, including step 1: processing the thermal imaging image through a multi-scale convolutional layer; step 2: outputting a 1024-dimensional visual feature vector from a fully connected layer.

[0067] The specific implementation of step S03 involves establishing a hierarchical anomaly detection architecture and dynamically adjusting the processing frequency based on network latency parameters. The formula for adjusting the cloud processing frequency is:

[0068] ;

[0069] In the formula, For cloud processing frequency; The reference processing frequency is set to 10Hz. This is the frequency adjustment factor; This is a network latency parameter.

[0070] The formula for calculating the frequency adjustment factor is:

[0071] ;

[0072] The formula for adjusting the edge processing frequency is:

[0073] ;

[0074] In the formula, Processing frequency for the edge side; Set the edge-side reference processing frequency to 5Hz. This is the frequency adjustment coefficient on the edge side.

[0075] The formula for calculating the edge-side frequency adjustment factor is:

[0076] ;

[0077] The parameter acquisition method is as follows: The round-trip time (RTT) measurement algorithm is used to obtain the average latency value, which includes step 1: sending test data packets to the target node every second; and step 2: recording the RTT of the data packets and calculating the average latency value.

[0078] The specific implementation of step S04 involves using a multimodal feature fusion model for feature fusion processing. The formula for calculating the attention weight allocation parameters is as follows:

[0079] ;

[0080] In the formula, This is the attention weight matrix; For query matrix; The key matrix; It is a value matrix; Let the dimension of the key vector be 256.

[0081] The softmax function is defined as follows:

[0082] ;

[0083] In the formula, Let i be the i-th element of the input vector; For vector dimensions; It is an exponential function.

[0084] The formula for dynamically adjusting the attention weight allocation parameters based on the network state is:

[0085] ;

[0086] In the formula, This is the final weight matrix; This is the weight adjustment function; For anomaly confidence parameters; These are the quality parameters for modal data.

[0087] The formula for calculating the weight adjustment function is:

[0088] ;

[0089] In the formula, This is a function for normalizing the delay parameter; This is the normalization function for the confidence parameter; This is the normalization function for the quality parameters.

[0090] The normalization function for the delay parameter is:

[0091] ;

[0092] In the formula, The minimum delay threshold is set to 10 milliseconds. The maximum latency threshold is set to 200 milliseconds.

[0093] The confidence parameter normalization function is:

[0094] ;

[0095] In the formula, The minimum confidence level is set to 0. The maximum confidence level is set to 1.

[0096] The normalization function for the quality parameters is:

[0097] ;

[0098] In the formula, Set the minimum quality score to 0; The maximum quality score is set to 1.

[0099] The parameter acquisition method is as follows: The results were obtained using statistical analysis methods, including step 1: calculating the deviation between the current test result and the historical normal value; and step 2: obtaining the confidence score through z-score standardization. Data integrity assessment was used, including step 1: detecting data missing rate and noise level; step 2: calculating the overall data quality score, ranging from 0 to 1.

[0100] The specific implementation of step S05 involves using an adaptive threshold dynamic adjustment Bayesian algorithm for anomaly detection. The Bayesian update equation is:

[0101] ;

[0102] In the formula, It is a posterior distribution; It is the likelihood function; It is the prior distribution; This represents the marginal probability. These are network state parameters; This is a sample of observational data.

[0103] The confidence assessment equation is:

[0104] ;

[0105] In the formula, For anomaly confidence parameters; To adjust the parameter, set it to 2.0; This is the current observation value; This is the mean of the posterior distribution; The standard deviation of the posterior distribution; It is a natural exponential function.

[0106] The threshold optimization equation is:

[0107] ;

[0108] In the formula, For optimized anomaly detection thresholds; The function of the independent variable is to find the function that minimizes the objective function; For threshold variables; The false alarm rate function; This is a function of the false negative rate; For detecting delay functions; These are the weighting coefficients, set to 0.4, 0.4, and 0.2 respectively.

[0109] The formula for monitoring the rate of change of abnormal confidence is:

[0110] ;

[0111] In the formula, The rate of change of abnormal confidence level; The current time; The time interval is set to 1 second.

[0112] The trigger condition for fast response adjustment is:

[0113] ;

[0114] In the formula, Duration, in seconds.

[0115] The parameter acquisition method is as follows: and The calculation uses Bayesian inference, including step 1: initializing the prior parameters of the β distribution. and Step 2: Update the posterior distribution parameters based on the new observation data. The method uses real-time monitoring and includes step 1: extracting key indicators from the fused feature vector; and step 2: standardizing the data to obtain the observed values.

[0116] The specific implementation of step S06 is to establish a multimodal fusion decision engine. The formula for calculating the single-modal anomaly amplitude value is:

[0117] ;

[0118] In the formula, Let be the abnormal amplitude value of the i-th mode; This represents the current observation value for the i-th mode; The historical mean of the i-th mode; Let be the historical standard deviation of the i-th modality.

[0119] The formula for dynamic weight adjustment is:

[0120] ;

[0121] In the formula, The new weights for the i-th mode; represents the original weights of the i-th mode.

[0122] The formula for the fusion decision output is:

[0123] ;

[0124] In the formula, For the final anomaly score; The anomaly score for the i-th mode; The total number of modalities is set to 4.

[0125] The parameter acquisition method is as follows: and The sliding window statistical calculation includes step 1: maintaining a historical data window of size 1000; and step 2: calculating the mean and standard deviation of the data within the window. The output of each modal detector is used, ranging from 0 to 1.

[0126] The specific implementation method of step S07 is the same as described above, and will not be repeated in detail here.

[0127] It needs to be explained that the specific implementation of the beam search parameter adjustment function in the deep operation and maintenance anomaly identification model is as follows:

[0128] ;

[0129] In the formula, Adjustment coefficients for beam search; This represents the normalized value of the network latency parameter. This is the normalized value of the anomaly confidence parameter.

[0130] The formula for adjusting the beam search parameters is:

[0131] ;

[0132] In the formula, The adjusted beam search parameters; This is the original beam search parameter, with a default value of 5.

[0133] The parameter acquisition method is as follows: The method uses maximum and minimum value normalization to obtain the result, including step 1: setting the delay parameter range from 10 milliseconds to 200 milliseconds; step 2: according to the formula... Calculate the normalized value. The same normalization method is used to obtain the data, with the range set to 0 to 1.

[0134] The specific implementation of the attention weight allocation parameter adjustment function in the multimodal feature fusion model is as follows:

[0135] ;

[0136] In the formula, Attention adjustment factor; This represents the normalized value of the modal data quality parameters.

[0137] The formula for adjusting the attention weight allocation parameters is:

[0138] ;

[0139] In the formula, Assign parameters to the adjusted attention weights; Assign parameters to the original attention weights; the default value is 1.0.

[0140] The parameter acquisition method is as follows: Data quality assessment was used, including step 1: calculating data integrity, accuracy and consistency indicators; step 2: obtaining a comprehensive quality score by weighted averaging and normalizing.

[0141] It's important to explain that the Bayesian update equation, based on Bayesian statistical inference theory, dynamically updates the parameter distribution by combining prior knowledge and observational data. Compared to traditional fixed-threshold methods, it can adapt to changes in the network environment, significantly reducing false alarm and false negative rates, and improving the accuracy and robustness of anomaly detection. The core advantage of this equation lies in its ability to organically combine historical experience with real-time observational data, enabling intelligent parameter updates.

[0142] The confidence assessment equation adopts a logistic function form, mapping the original anomaly metrics to a probability range of 0 to 1 through a sigmoid transformation. Compared to linear transformations, this better handles extreme value cases, avoids the saturation effect of anomaly scores, and provides smoother and more stable confidence assessment results. The equation introduces an adjustment parameter λ to achieve precise control of assessment sensitivity.

[0143] The threshold optimization equation is based on multi-objective optimization theory, and considers three key performance indicators: false alarm rate, false negative rate and detection latency. By constructing a comprehensive objective function through weighted summation, it can achieve the optimal balance between detection accuracy and real-time performance compared to single-indicator optimization, thus meeting the diverse needs of actual operation and maintenance scenarios.

[0144] The attention weight allocation parameter adjustment function is based on the weighted fusion of multi-dimensional state information. It combines three core parameters: network latency, anomaly confidence, and data quality. Compared with static weight allocation, it can dynamically optimize the contribution ratio of each modality according to the real-time state, give full play to the information value of high-quality data, effectively suppress the negative impact of low-quality data, and significantly improve the accuracy and reliability of multi-modal fusion.

[0145] The beam search parameter adjustment function adaptively optimizes the inference parameters of the model based on network conditions, dynamically balancing computational complexity and response speed while ensuring detection accuracy. Compared with fixed parameter configuration, it can intelligently adjust resource allocation according to the current network conditions, reducing computational overhead and improving response speed when the network load is high, and enhancing detection accuracy when the network conditions are good, thus achieving the optimal match between performance and efficiency.

[0146] The delay parameter normalization function, based on linear transformation theory, maps delay parameters with different dimensions and numerical ranges to a unified interval of 0 to 1. Compared to directly using the original delay values, this eliminates the influence of dimensions, ensures the weight balance of different parameters in the fusion calculation, and improves the stability and accuracy of multi-parameter fusion. The function form is:

[0147] ;

[0148] The softmax function, based on the principle of exponential normalization, transforms any real-valued vector into a probability distribution. Compared to simple linear normalization, it amplifies the weight differences of larger elements in the vector while suppressing the influence of smaller elements, achieving a non-linear allocation of attention weights and enhancing the model's ability to focus on key features. The function's form is:

[0149] ;

[0150] To better understand and implement this invention, the following is a specific application scenario example 2: A technical team applied the AI-based multimodal network information operation and maintenance system of this invention in the operation and maintenance of a large data center network. This data center contains 2800 servers, 156 switches, 89 routers, and 23 firewall devices, with a total network bandwidth capacity of 480Gbps and an average daily data traffic processing volume of approximately 185TB. The main technical challenge faced by the team was that traditional single-modal monitoring systems had low anomaly detection accuracy in complex network environments, with a false alarm rate as high as 38%, and could not effectively identify complex fault modes across devices and levels.

[0151] The technical team first deployed multimodal data acquisition units on various network devices, including network traffic monitoring sensors, device status log collectors, audio signal collectors, and thermal imaging cameras. The network traffic monitoring sensors collect network data packets at 100ms sampling intervals, the device status log collectors collect approximately 2.3MB of device operation log data per second, the audio signal collectors record device operation sounds at a 48kHz sampling rate, and the thermal imaging camera captures images of device temperature distribution at 30fps. Edge computing nodes, configured with 8-core processors and 32GB of memory, perform preliminary preprocessing on the collected raw multimodal operation and maintenance data, reducing data transmission latency from 180ms in traditional solutions to 45ms.

[0152] The multimodal feature extraction module employs a parallel processing architecture, simultaneously processing four different types of input data. The network traffic time-series feature extraction module extracts time-series features from network data packet transmission patterns, identifying an average traffic fluctuation period of 125 seconds, with peak distribution concentrated in the 8:30-9:30 and 14:00-15:00 time periods, and a periodic variation in transmission delay patterns. The text semantic feature extraction module performs semantic analysis on device status logs, establishing a feature dictionary containing 6847 keywords, achieving an accuracy rate of 89.2% in describing abnormal events. The audio spectrum feature extraction module analyzes device operating audio, identifying a normal operating frequency range of 20-8000Hz, with abnormal harmonic components mainly concentrated in the 12-15kHz frequency band. The image visual feature extraction module processes thermal imaging images, identifying a normal temperature distribution range of 35-42℃, a hotspot threshold set at 55℃, and a temperature change trend detection accuracy of 94.7%.

[0153] The technical team established a hierarchical anomaly detection architecture, deploying a lightweight anomaly detection model at the edge with 2.8M parameters and an inference time of only 12ms for real-time preliminary judgment. A deep operational anomaly recognition model was deployed in the cloud, employing a multimodal anomaly detection network based on the Transformer architecture. This model has 145M parameters, including 12 multimodal encoding layers, 8 cross-modal attention computation layers, 6 temporal modeling layers, and 3 anomaly classification output layers. When the network latency parameter value is in the 10-50ms range, the cloud processing frequency is reduced to 70% of the original frequency; when the network latency parameter value is in the 50-100ms range, the current cloud processing frequency is maintained; and when the network latency parameter value is in the 100-200ms range, the edge processing frequency is increased to 150% of the original frequency.

[0154] The training dataset for the deep operation and maintenance anomaly identification model contains 1.86 million multimodal operation and maintenance data samples, of which 78.4% are in normal operation status and 21.6% are in various abnormal statuses. (See Table 1.)

[0155] Table 1. Statistical table of abnormal state sample distribution

[0156]

[0157] The training process employed the Adam optimization algorithm, with an initial learning rate of 0.0003, a batch size of 64, and 150 training epochs. The model achieved an anomaly detection precision of 92.8%, a recall of 89.6%, and an F1 score of 91.2% on the validation set. The beam search parameters were dynamically adjusted based on network latency and anomaly confidence parameters. When the beam search adjustment coefficient was in the range of 0.2-0.5, the beam search parameters were adjusted to 0.6 times the original parameters; when the adjustment coefficient was in the range of 0.5-0.8, the current parameters remained unchanged; and when the adjustment coefficient was in the range of 0.8-1.0, the beam search parameters were increased to 1.4 times the original parameters.

[0158] The multimodal feature fusion model employs a multi-level attention fusion network structure, comprising 4 intramodal feature encoding layers, 6 cross-modal attention computation layers, and 2 fusion decision output layers. The attention weight allocation parameters are dynamically determined based on network latency, anomaly confidence, and modal data quality parameters. When the attention adjustment coefficient is in the range of 0.3-0.7, the attention weight allocation parameters are adjusted to 0.8 times the original parameters; when the adjustment coefficient is in the range of 0.7-0.9, the current parameters remain unchanged; and when the adjustment coefficient is in the range of 0.9-1.0, the weights are increased to 1.2 times the original parameters. The fused feature vector has a dimension of 512, containing comprehensive information representations from the four modalities.

[0159] An adaptive threshold dynamic adjustment Bayesian algorithm maintains the prior distribution of network state parameters and continuously updates the posterior probability distribution using observed data. Algorithm inputs include network latency distribution parameters, traffic fluctuation variance parameters, device temperature statistics parameters, and historical anomaly frequency parameters. The Bayesian update cycle is set to 30 seconds, and the sliding time window size is 300 seconds. When the anomaly confidence change rate exceeds 15% / second and the duration is greater than 3 seconds, the system immediately performs a fast response adjustment, updating the anomaly detection threshold within 2 seconds. When the anomaly confidence change rate exceeds 15% / second but the duration is less than 3 seconds, the system performs a slow response adjustment, with a threshold adjustment time of 8 seconds. Figure 2 As shown, the adaptive threshold adjustment process can effectively track changes in network state and maintain the accuracy and stability of anomaly detection.

[0160] The multimodal fusion decision engine employs an attention weight allocation mechanism to weight and fuse anomaly detection results from different modalities. When the magnitude of a single-modal anomaly is within the range of 0.3-0.6, the system maintains normal multimodal fusion operation, with all modalities participating in the decision-making process. When the magnitude of a single-modal anomaly is within the range of 0.6-0.8, the system reduces the weight of that modality to 60% of its rated weight to minimize its impact on the final decision. When the magnitude of a single-modal anomaly exceeds 0.8, the system immediately stops that modality from participating in the fusion process to prevent anomalous data from interfering with the overall decision result. Figure 3 As shown, the multimodal fusion decision-making mechanism can adaptively adjust the contribution weights of each modality to ensure the reliability of the decision results.

[0161] The technical team also established an anomaly root cause analysis module, employing causal reasoning algorithms to perform in-depth analysis of anomaly detection results. The system constructs a dependency graph encompassing all network devices, containing 3067 nodes and 8934 edges. Nodes represent network devices, and edges represent the strength of dependencies between devices. An event time-series correlation matrix records the co-occurrence frequency of various anomaly events within different time windows; the matrix has a dimension of 16×16 and covers 16 major anomaly event types. Through event correlation analysis within time windows, the system can accurately identify the root cause and propagation path of anomaly events, achieving a root cause localization accuracy of 87.3%.

[0162] Actual operation results show that the system's multimodal data acquisition capability has been significantly improved, processing 47.8GB of multimodal data per hour, including 32.1GB of network traffic data, 8.7GB of device log data, 4.2GB of audio signal data, and 2.8GB of thermal imaging data. The overall anomaly detection accuracy of the system has improved to 94.6%, the false alarm rate has decreased to 8.2%, and the average response time is 3.7 seconds. During the test, the system successfully detected and handled 268 abnormal events, including 47 network latency anomalies, 39 device overheating anomalies, 52 traffic surge anomalies, 31 hardware failure anomalies, 38 security attack anomalies, 34 configuration error anomalies, and 27 performance degradation anomalies.

[0163] This invention represents a significant technological advancement compared to traditional network operation and maintenance methods. Traditional single-modal monitoring systems can only process a single type of monitoring data, resulting in incomplete information acquisition and difficulty in accurately reflecting the true operating status of network devices. In contrast, this invention, through multimodal data acquisition and parallel feature extraction, can simultaneously utilize multiple information sources such as network traffic, device logs, audio signals, and thermal imaging to form a comprehensive perception of the network status. Traditional systems employ fixed-threshold anomaly detection methods, which cannot adapt to dynamic changes in the network environment and are prone to false alarms and missed alarms. This invention, however, uses an adaptive threshold dynamic adjustment Bayesian algorithm, which continuously optimizes detection parameters based on historical data and the current network status, significantly improving the accuracy and stability of anomaly detection. Traditional systems lack effective multi-source information fusion mechanisms, making it difficult to identify complex fault modes across devices and levels. The multimodal fusion decision engine established in this invention automatically adjusts the contribution weights of each modality through an attention weight allocation mechanism, effectively handling differences in modal data quality and changes in correlation, ensuring the reliability and robustness of the fusion decision.

[0164] It should be noted that the variables involved in this invention are explained in detail in Table 2.

[0165] Table 2 Variable Explanation Table

[0166]

[0167] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. An AI multi-modal based network information operation and maintenance system, characterized in that, A multimodal data acquisition unit is deployed on network devices to collect multimodal operation and maintenance data, including network traffic data, device status logs, audio alarm signals, and device thermal imaging images. Preliminary preprocessing is performed via edge computing nodes. The collected multimodal operation and maintenance data is then input into a multimodal feature extraction module, which extracts network traffic temporal features, text semantic features, audio spectrum features, and image visual features through a parallel processing architecture, forming a unified multidimensional feature vector. A hierarchical anomaly detection architecture is established, deploying a lightweight anomaly detection model at the edge for real-time preliminary judgment, and a deep operation and maintenance anomaly identification model in the cloud for in-depth analysis. A multimodal feature fusion model is used to fuse the unified multidimensional feature vector. Attention weight allocation parameters are determined based on network latency parameters, anomaly confidence parameters, and modality data quality parameters, outputting a fused feature vector. An adaptive threshold-dynamically adjusted Bayesian algorithm is used to detect anomalies in the fused feature vector, dynamically adjusting the anomaly detection threshold through a Bayesian inference mechanism. A multimodal fusion decision engine is established, using an attention weight allocation mechanism to weight and fuse anomaly detection results from different modalities. When the single-modal anomaly amplitude value is within the range of [0.3, 0.6], the multimodal fusion continues to operate normally. When the single-modal anomaly amplitude value is within the range of (0.6, 0.8], the single-modal weight is reduced to 60% of the rated weight. When the single-modal anomaly amplitude value exceeds 0.8, the single-modal participation in the fusion process is immediately stopped. It also includes a beam search parameter adjustment function for dynamically adjusting the beam search parameters of the deep operation and maintenance anomaly identification model according to the network status. The beam search parameters are used to balance the detection accuracy and inference speed of the deep operation and maintenance anomaly identification model. The beam search parameter adjustment function calculates the beam search adjustment coefficient based on network latency parameters and anomaly confidence parameters. When the beam search adjustment coefficient b∈[0.2, 0.5), the beam search parameters are adjusted to 0.6 times the original beam search parameters using a linear decreasing method to accelerate the inference process. When the beam search adjustment coefficient b∈[0.5, 0.8), the current beam search parameters are kept unchanged to maintain detection accuracy. When the beam search adjustment coefficient b∈[0.8, 1.0], the beam search parameters are increased to 1.4 times the original beam search parameters using an exponential increasing method to improve detection accuracy. The beam search parameter adjustment function is expressed as follows: ； In the formula, Adjustment coefficients for beam search; The normalization function for the network latency parameter is expressed as: ,in The minimum latency threshold is set to 10 milliseconds. The maximum latency threshold is set to 200 milliseconds. The normalization function for the anomaly confidence parameter is expressed as: ,in Set to 0, Set to 1; The anomaly confidence parameter is calculated using the following confidence assessment equation. ;; The anomaly confidence parameter is calculated using the confidence evaluation equation and is expressed as follows: ； In the formula, For anomaly confidence parameters; To adjust the parameter, set it to 2.0; This is the current observation value; Network state parameters The posterior distribution mean; Network state parameters The posterior distribution standard deviation; It is a natural exponential function.

2. The network information operation and maintenance system based on AI multimodality according to claim 1, characterized in that, The multimodal data acquisition unit specifically includes a network traffic monitoring sensor, a device status log collector, an audio signal acquisition device, and a thermal imaging camera.

3. The network information operation and maintenance system based on AI multimodality according to claim 2, characterized in that, The edge computing node is specifically a lightweight computing device deployed at the network edge, used for preprocessing and preliminary analysis of raw multimodal operation and maintenance data.

4. The network information operation and maintenance system based on AI multimodality according to claim 3, characterized in that, The multimodal feature extraction module adopts a parallel processing architecture, which processes different types of input data simultaneously and extracts key feature information in their respective fields.

5. The network information operation and maintenance system based on AI multimodality according to claim 4, characterized in that, The network traffic time-series characteristics refer to the time-series characteristics extracted from network data packet transmission patterns, including traffic fluctuation cycles, peak distributions, and transmission delay patterns.

6. The network information operation and maintenance system based on AI multimodality according to claim 5, characterized in that, The text semantic features refer to the semantic information extracted from the device status log text, including keyword weights, semantic relevance, and abnormal event descriptions.

7. The AI-based multimodal network information operation and maintenance system according to claim 6, characterized in that, The audio spectrum features refer to the frequency domain features extracted from the audio of the device in operation, including spectral energy distribution, harmonic components, and noise features.

8. The network information operation and maintenance system based on AI multimodality according to claim 7, characterized in that, The image visual features refer to the visual features extracted from the thermal imaging images of the device, including temperature distribution patterns, hot spots, and temperature change trends.

9. The network information operation and maintenance system based on AI multimodality according to claim 8, characterized in that, The unified multidimensional feature vector refers to a high-dimensional vector representation formed by concatenating features from different modalities after standardization.

10. The AI-based multimodal network information operation and maintenance system according to claim 9, characterized in that, The hierarchical anomaly detection architecture adopts an edge-cloud collaborative deployment mode.

Images