Urban communication abnormity monitoring and early warning system based on multi-modal data fusion

Abstract

The invention discloses an urban communication abnormity monitoring and early warning system based on multi-modal data fusion, and the system comprises a multi-modal data collection module which is used for collecting multi-modal data; the data preprocessing and aligning module is used for preprocessing the multi-modal data; the multi-scale double-tower representation modeling module is used for constructing a global embedded vector sequence and a local embedded vector sequence; the nonlinear contrast mapping module is used for constructing a projection vector for contrast learning; the abnormal score generation module is used for generating an abnormal probability score; the abnormity positioning and influence evaluation module is used for positioning a link, a node and a transmission area where communication abnormity occurs; and the graded early warning and linkage control module is used for triggering a multi-stage early warning mechanism. According to the invention, high-precision detection, positioning and early warning of urban communication abnormal events are realized, and the method is suitable for real-time monitoring and emergency response scenes of communication networks in smart cities.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of smart city communication network monitoring and early warning, and particularly relates to a city communication anomaly monitoring and early warning system based on multi-modal data fusion. BACKGROUND

[0002] In the construction process of smart city, the city communication system as an important part of information infrastructure, carries key business transmission, city management and dispatch, public service and other tasks, its stability and security have important influence on the overall operation efficiency and emergency response capability of the city. The existing communication anomaly monitoring means mainly relies on single modal data source, such as network performance index or communication equipment log, and adopts anomaly detection method based on rule threshold or simple statistical model. This kind of method in the face of high-dimensional heterogeneous, multi-source uncertainty enhanced city communication scene, often has low accuracy, response lag and poor generalization ability, etc., it is difficult to meet the high real-time and high reliability demand in complex city environment.

[0003] The existing method generally lacks the ability of structured characterization of city communication anomaly events, especially in the modeling of the interaction between time, space and multi-modal. Some researches introduce deep learning model for time series prediction or anomaly scoring, but often simply concatenate all data, ignoring the collaborative representation between modal, dynamic changes under different time scales and the influence of city communication topology structure on abnormal propagation path, resulting in the model unable to effectively capture key abnormal features, and even in the face of unknown mode or unlabeled data, the abnormality is missed.

[0004] After anomaly recognition, most of the existing systems cannot further perform anomaly positioning and business impact assessment, lack the ability to link with city communication topology graph and propagation path model, and cannot realize accurate tracing and rapid response of abnormal event chain. This makes the city lack effective early warning mechanism and dispatch basis in the face of sudden communication events, which seriously restricts the intelligent evolution of city communication system.

[0005] An integrated anomaly monitoring and early warning system is needed, which can fuse multi-modal communication data, combine multi-scale time series structure modeling, contrast learning enhancement mechanism and communication topology perception ability, to realize accurate detection, tracing and hierarchical response of city communication anomaly, so as to improve the robustness and intelligent level of smart city communication system. SUMMARY

[0006] One purpose of the present application is to propose a city communication anomaly monitoring and early warning system based on multi-modal data fusion, which fully integrates multi-source heterogeneous communication data, combines multi-scale double-tower coding modeling, multi-head contrast learning mechanism and residual discriminant network, and describes in detail the whole process from communication data preprocessing, embedding representation modeling, anomaly detection to source positioning, which has the advantages of high modeling accuracy, strong anomaly recognition robustness, good interpretability and high linkage response efficiency.

[0007] According to the city communication anomaly monitoring and early warning system based on multi-modal data fusion of the embodiment of the present application, comprising: A multi-modal data acquisition module is used to acquire multi-modal data in the city communication system. A data preprocessing and alignment module is used to preprocess the multi-modal data and extract feature vectors. A multi-scale double-tower representation modeling module is used to input the preprocessed multi-modal communication time series into a multi-scale double-tower coding structure to construct global embedding vector sequences and local embedding vector sequences respectively. A nonlinear contrast mapping module is used to adjust the nonlinear mapping of embedding vector sequences and local embedding vector sequences in the training stage to construct projection vectors for contrast learning. An anomaly score generation module is used to construct a residual discriminant subspace based on multi-scale embedding representation to generate an anomaly probability score. An anomaly positioning and impact assessment module is used to locate the link, node and transmission area where the communication anomaly occurs after the anomaly is detected. A hierarchical early warning and linkage control module is used to trigger a multi-level early warning mechanism according to the anomaly score value and risk level threshold.

[0008] Optionally, the modules are realized by the following method: S1, acquire multi-modal communication data of the city communication system; S2, preprocess the acquired multi-modal communication data to construct a unified modeling input sequence; S3, input the modeling input sequence into the multi-scale double-tower coding structure, including a global representation branch and a fine-grained representation branch, and the two branches generate multi-scale embedding representation through a Patch-Transformer network respectively; S4, introduce a multi-head projection mechanism and a multi-dimensional contrast subtask construction method, map the multi-scale embedding representation to a contrast space through a nonlinear adjustable mapping head module to construct a contrast subtask; S5, construct the residual vector between the global and fine-grained representations into an independent residual discriminant subspace and input it into a residual discriminant network to generate an anomaly probability score; S6, when the city communication anomaly monitoring and early warning system detects an abnormal event, the link, node or transmission area where the abnormal event occurs is inferred in combination with the city communication topology structure, attention response graph and propagation path model, and the affected service range is evaluated.

[0009] Optionally, the city communication anomaly monitoring and early warning system based on multi-modal data fusion has the characteristics that the multi-modal communication data includes network performance index data, communication equipment log data, video monitoring data, audio signal data and auxiliary sensor data; the network performance index includes packet loss rate, bandwidth utilization rate, signal strength and delay; and the auxiliary sensor data includes power state, temperature and humidity and electromagnetic interference strength.

[0010] Optionally, S2 includes the following specific steps: S21, performing missing value detection and noise identification on the collected multi-modal communication data, repairing the data with missing values by using mean interpolation, sliding window filling or sample interpolation, and eliminating obvious outliers and abnormal values; S22, setting a uniform time step, time-aligning the asynchronously sampled multi-modal data according to the time step, and constructing a uniform time axis; S23, extracting features according to time steps for each type of modality data after alignment, and the extracted features include time domain statistical features, frequency features, communication topology structure features and graph embedding features; S24, concatenating and combining the multi-modal feature vectors corresponding to each time step to generate a unified modeling input sequence.

[0011] Optionally, S3 includes the following specific steps: S31, inputting the modeling input sequence obtained through time synchronization and feature extraction into a multi-scale double-tower encoding structure, and the modeling input sequence is a multi-modal feature fusion vector sequence under a uniform time axis.

[0012] S32, dividing the modeling input sequence into long-term segments according to a long-term time window to obtain a long-term segment sequence, and inputting the long-term segment sequence into a global representation branch to extract a global embedding vector sequence through a Patch-Transformer encoder; S33, dividing the modeling input sequence into short-term segments according to a short-term time window to obtain a short-term segment sequence, and inputting the short-term segment into a fine-grained representation branch to extract a local embedding vector sequence through a Patch-Transformer encoder; S34, the multi-scale embedding representation includes the global embedding vector sequence output by the global representation branch and the local embedding vector sequence output by the fine-grained representation branch.

[0013] Optionally, the S4 comprises the following specific steps: S41, for the global embedding vector sequence output by the cross-fragment representation branch and the intra-fragment representation branch in the multi-scale double-tower structure respectively, and , two nonlinear mapping head modules are introduced, and BatchNorm and ReLU activation operations are integrated after each hidden layer to construct a nonlinear transformation path; S42, the nonlinear mapping head module maps the original embedding vector to a low-dimensional contrast space based on the constructed nonlinear transformation path, and outputs a projection vector. The distance measurement in the contrast space is carried out through the cosine similarity function, and is directly input as information contrast estimation loss function; S43, in the multi-head contrast learning mechanism training process based on multi-dimensional subtasks, the global and fine-grained projection vectors output by the nonlinear mapping module are used to construct positive and negative sample pairs, and the multi-head contrast learning mechanism is established in the form of information contrast estimation loss; The mapped vectors at the same time step form a positive sample pair, and the mapped vectors from different time steps, different modalities and different communication node sources form a negative sample pair; S44, a multi-head projection mechanism and a multi-dimensional contrast subtask construction method are introduced. On the basis of the nonlinear mapping head module, an independent mapping path is configured for each dimensional contrast task to construct a multi-head projection mechanism with structural decoupling characteristics, wherein each mapping head is composed of multiple layers of perception mechanism and integrates BatchNorm and ReLU activation unit; S45, the multi-dimensional contrast subtask includes a time scale contrast subtask, a modal dimension contrast subtask, and a communication topology structure contrast subtask; S46, the time scale contrast subtask, by sampling embedding vectors of different time steps at adjacent or non-adjacent time steps as negative samples, a sample pair with time delay feature difference is constructed, and the positive sample representation from different granularity branches at the same time step is combined to establish a contrast target with time perception; S47, the modal dimension contrast subtask, based on the multi-modality of the input data source, extracts the embedding representation of the corresponding modal channel, and constructs a cross-modal contrast pair, wherein the positive sample pair is composed of representations of different modalities at the same time step and the same event, and the negative sample pair is composed of embedding between non-synchronous modalities; S48, the communication topology structure contrast subtask, uses the topological structure difference of different nodes and their adjacent nodes in the city communication topology graph to construct a node contrast task based on graph structure sampling. The embedding representation of the topologically adjacent nodes at the same time step is selected as the positive sample pair, and the embedding representation of the topologically unconnected or distant nodes is selected as the negative sample pair.

[0014] S49, training is carried out by using information to estimate a loss function, corresponding multi-dimensional contrast sub-task loss items are calculated respectively, and finally each sub-loss is weighted and summed through a set weight coefficient to construct a complete joint loss function.

[0015] Optionally, the S5 comprises the following specific steps: S51, at each time step, a residual vector between a global embedding vector sequence and is calculated; a residual discriminant sub-network is introduced as a core structure of an anomaly score mechanism, and the residual discriminant sub-network is a set of training state multilayer perceptrons; S51, at each time step, a residual vector between a global embedding vector sequence and is calculated; a residual discriminant sub-network is introduced as a core structure of an anomaly score mechanism, and the residual discriminant sub-network is a set of training state multilayer perceptrons; S52, the residual vector is input into the residual discriminant sub-network, an anomaly score is output, and the anomaly score is mapped into an anomaly probability score through a Sigmoid activation function, and the anomaly probability score is used for representing an anomaly degree of a communication state at a current time step; S53, in a contrast learning training phase, a joint training strategy is adopted to guide the residual discriminant sub-network and the multi-head contrast learning module to optimize cooperatively, a training target is composed of a contrast learning loss and a residual classification loss, and a joint loss function is formed.

[0016] Optionally, the S6 comprises the following specific steps: S61, when the anomaly probability score output by the anomaly score generation module exceeds a preset threshold value, the system determines that there is a communication anomaly event at a current time step, and enters an anomaly tracing and influence evaluation phase; S62, a multi-modal communication data segment corresponding to an anomaly time step and an embedding vector thereof are extracted, and a city communication topology graph maintained by the system is combined to initialize a topology perception state graph; S63, the anomaly embedding representation is input into a response graph generation module constructed based on an attention mechanism as a query vector, and attention heat maps at a node level and a link level are obtained, which are used for locating a potential anomaly propagation path; S64, in combination with a propagation path model established by the system, a key path affected by an anomaly in the city communication topology graph is analyzed inversely, an anomaly high-occurrence area is identified, and a most possible source node and an influence link set thereof are reasoned; S65, an influence range evaluation function is constructed based on a node weight distribution, an anomaly intensity score and a path dependence degree; S66, according to a comparison result of the anomaly probability score and the preset threshold value, a business area affected and an influence level are determined, and the identification result is input into a hierarchical early warning and linkage control module to trigger a linkage response mechanism at a corresponding level.

[0017] The present application has the following advantages: The application constructs an urban communication anomaly monitoring and early warning system based on multi-modal data fusion, significantly improves the anomaly recognition ability and response efficiency of the system in the context of complex communication environment and multi-source heterogeneous data.

[0018] The application fully integrates network performance indicators, log data, video monitoring, audio signals and auxiliary sensors and other multi-modal information in the communication system, enhances the expression ability of input data through unified time axis alignment and multi-dimensional feature extraction, and provides high-quality basic data support for subsequent modeling. Secondly, the proposed multi-scale double-tower Patch encoding structure combines the global and fine-grained embedding representations constructed by long-term and short-term time windows to capture communication dynamic change characteristics at different granularity levels, effectively improving the modeling accuracy and generalization ability of the system to abnormal events.

[0019] In the training phase, the application introduces a multi-head contrast learning mechanism based on nonlinear projection path, realizes structure decoupling contrast optimization among modalities, time and topology through the construction of multi-dimensional subtasks, and strengthens the discrimination ability of the representation and the robustness of the system. In the anomaly detection link, a residual discriminant subspace based on the difference between global and local representations is constructed, and a discriminant subnetwork is designed to score the anomaly probability, replacing the traditional fixed threshold or density estimation method, and improving the detection sensitivity to weak anomalies and unknown patterns. After the anomaly is detected, the application combines the communication topology graph, attention heat map and propagation path model to perform anomaly tracing and impact assessment, accurately locates the fault nodes and links, and significantly improves the early warning response level and fault disposal efficiency of the urban communication system. The application has the advantages of strong fusion, high recognition accuracy, good structure universality, timely anomaly response, etc., and is significantly superior to the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0020] The accompanying drawings are used to provide a further understanding of the application, and constitute a part of the specification, together with the embodiments of the application, to explain the application, and do not constitute a limitation on the application. In the drawings:

[0021] Fig. 1 A method flowchart of an urban communication anomaly monitoring and early warning system based on multi-modal data fusion is proposed for the application; Fig. 2 A system flowchart of an urban communication anomaly monitoring and early warning system based on multi-modal data fusion is proposed for the application; Fig. 3 A structure diagram of the multi-head projection mechanism and multi-dimensional contrast subtask construction method proposed by the application. DETAILED DESCRIPTION

[0022] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0023] refer to Figs. 1-3 A system for monitoring and early warning of urban communication anomalies based on multimodal data fusion, comprising: A multimodal data acquisition module is used to collect multimodal data from urban communication systems. The data preprocessing and alignment module is used to preprocess multimodal data and extract feature vectors; The multi-scale dual-tower representation modeling module is used to input the preprocessed multimodal communication time series into the multi-scale dual-tower coding structure to construct the global embedding vector sequence and the local embedding vector sequence respectively. The nonlinear contrast mapping module is used to perform an adjustable nonlinear mapping between the embedded vector sequence and the local embedded vector sequence during the training phase, and to construct the projection vector for contrastive learning. An anomaly score generation module is used to construct a residual discrimination subspace based on multi-scale embedding representation and generate anomaly probability scores. The anomaly localization and impact assessment module is used to locate the link, node, and transmission area where the communication anomaly occurred after it has been detected. The graded early warning and linkage control module is used to trigger a multi-level early warning mechanism by setting thresholds based on the abnormal score and risk level.

[0024] This invention includes a multimodal data acquisition module for collecting heterogeneous data from multiple sources, such as network performance indicators, logs, video, audio, and sensors; a data preprocessing and alignment module for performing missing data repair, noise reduction, and time synchronization on the raw data, and extracting unified modeling feature vectors; a multi-scale dual-tower representation modeling module for extracting global and local embedding vectors through a dual-branch structure to capture multi-granular dynamic features; a nonlinear contrast mapping module for mapping embedding vectors to a contrast space during the training phase to construct projection vectors for multi-head contrast learning; an anomaly scoring generation module for constructing a residual discrimination subspace based on multi-scale embedding differences to generate anomaly probability scores; an anomaly localization and impact assessment module for inferring anomaly source nodes and their propagation paths by combining topology and attention mechanisms; and a hierarchical early warning and linkage control module for setting thresholds based on anomaly scores and risk levels to trigger corresponding early warning and response mechanisms, thereby achieving efficient and accurate detection and handling of communication anomalies.

[0025] In this embodiment, the modules are interconnected using the following method: S1. Collect multimodal communication data from the city's communication system; S2. Preprocess the collected multimodal communication data to construct a unified modeling input sequence; S3, input the modeling input sequence into the multi-scale double-tower coding structure, including a global representation branch and a fine-grained representation branch, and the two branches respectively generate multi-scale embedding representations through a Patch-Transformer network; S4, introduce a multi-head projection mechanism and a multi-dimensional contrastive subtask construction method, map the multi-scale embedding representations to a contrastive space through a nonlinear adjustable mapping head module, and construct a contrastive subtask; S5, construct a residual vector between the global and fine-grained representations into an independent residual discrimination subspace, and input it into a residual discrimination network to generate an anomaly probability score; S6, when the system detects an abnormal event, infer the link, node or transmission area where the abnormal event occurs by combining the city communication topology structure, the attention response graph and the propagation path model, and evaluate the affected business scope.

[0026] The present application proposes a city communication anomaly monitoring and early warning method, which first collects multi-modal data from a communication system, including performance indicators, device logs, video and audio information and sensor data, and pre-processes and time synchronizes them to construct a unified modeling input sequence; then input the input sequence into a multi-scale double-tower coding structure, and extract multi-scale embedding representations through the Patch-Transformer network of the global and fine-grained two branches; on this basis, introduce a multi-head projection mechanism and a multi-dimensional contrastive subtask construction method, use a nonlinear mapping module to project the embedding vectors to a contrastive space and perform contrastive learning; further, calculate the residual vector between the global and fine-grained representations, construct an independent residual discrimination subspace, and output an anomaly probability score through a residual discrimination network; finally, after the anomaly is detected, combine the communication topology structure, the attention response graph and the propagation path model to perform anomaly tracing analysis, infer the link, node or transmission area where the abnormal event occurs, and evaluate the business impact scope.

[0027] In the embodiment, the multi-modal communication data includes network performance indicator data, communication device log data, video monitoring data, audio signal data and auxiliary sensor data; the network performance indicators include packet loss rate, bandwidth utilization rate, signal strength and delay; the auxiliary sensor data includes power state, temperature and humidity, and electromagnetic interference strength.

[0028] In the embodiment, S2 includes the following specific steps: S21, perform missing value detection and noise identification operations on the collected multi-modal communication data, repair the data with missing values using mean interpolation, sliding window filling or sample interpolation, and eliminate obvious outliers and outliers; S22, set a unified time step , time align the asynchronously sampled multi-modal data according to the time step to construct a unified time axis wherein , ; S23, the aligned multi-modal data of each type is subjected to feature extraction by time step, and the extracted features include time domain statistical features, frequency features, communication topology structure features and graph embedding features, respectively denoted as ; S24, the multi-modal feature vector corresponding to each time step is spliced and combined to generate a unified modeling input sequence wherein represents the -dimensional fusion vector of the th time step, which is used for subsequent multi-scale representation modeling and anomaly detection.

[0029] In this embodiment, S3 includes the following specific steps: S31, the modeling input sequence obtained through time synchronization and feature extraction is input into a multi-scale double-tower encoding structure, and the input sequence is a multi-modal feature fusion vector sequence under a unified time axis, represents the -dimensional feature vector of the th time step; S32, the input sequence is divided into patches according to a long-term time window to obtain a long-term patch sequence , and the long-term patch sequence is input into a global representation branch to extract a global embedding vector sequence wherein represents the global representation vector of the th patch; S33, the input sequence is divided into patches according to a short-term time window to obtain a short-term patch sequence , and the short-term patch sequence is input into a fine-grained representation branch to extract a local embedding vector sequence wherein represents the local representation vector of the th patch; S34, the embedding representation output by the global representation branch and the embedding representation output by the fine-grained representation branch are taken as inputs of a subsequent multi-granularity contrast learning module and a residual discriminant module, respectively, for modeling the cross-scale consistency and residual anomaly pattern in the communication data.

[0030] This step models and processes the multi-modal feature fusion vector sequence under the unified time axis through a multi-scale double-tower coding structure. First, the modeling input sequence is input to the double-tower structure. The global representation branch receives patch sequences divided according to long-term time windows, extracts a global embedding vector sequence through a Patch-Transformer encoder, and is used to capture long-term dependency features of communication data. The fine-grained representation branch receives patch sequences divided according to short-term time windows, also extracts a local embedding vector sequence through a Patch-Transformer encoder, and is used to model short-term local change information. Finally, the global and local embedding representations are jointly used as inputs of a subsequent multi-granularity contrast learning module and a residual discriminant module, to realize deep characterization and modeling of cross-scale consistency structure and residual anomaly patterns.

[0031] In this embodiment, S4 includes the following specific steps: S41, for the global embedding vector sequence and the local embedding vector sequence respectively output by the cross-segment representation branch and the intra-segment representation branch in the multi-scale double-tower structure, two structure-independent and adjustable nonlinear mapping head modules are introduced, which are respectively defined as and Each is composed of multiple layers of perception, and integrates BatchNorm and ReLU activation operations after each hidden layer to build a nonlinear transformation path.

[0032] S42, the mapping head module is used to map the original embedding vector to a low-dimensional contrast space. The contrast space refers to a projection space specially constructed for contrast learning training. In this space, the geometric distribution between representations is more suitable for distinguishing positive and negative samples through similarity calculation. The mapping process is defined as and wherein and are the projected embedding vectors, which are distance measured in the contrast space through the cosine similarity function and are directly input to the information contrast estimation loss function, thereby realizing cross-scale representation alignment and consistency enhancement. This mapping mechanism is only enabled in the training stage to improve the discrimination ability of the model on semantic representation; in the inference stage, the mapping module is pruned to ensure that the computational efficiency of the system is not affected during deployment.

[0033] S43, in the training process, based on the projected representations and construct a positive and negative sample pair, and adopt an InfoNCE loss form to establish a multi-head contrast learning mechanism. Wherein, and The positive sample pair is composed of different time steps, different modalities or different communication node sources, and the similarity is measured by The function is measured in combination with the temperature parameter Control the divergence of the contrast distribution.

[0034] S44, introduce a multi-head projection mechanism and a multi-dimensional contrast sub-task construction method, configure an independent mapping path for each dimensional contrast task based on the nonlinear mapping head module, construct a multi-head projection mechanism with structural decoupling characteristics, and each mapping head is composed of multiple perception mechanisms and integrates BatchNorm and ReLU activation units to enhance the expression independence and semantic separability between different dimensional features. S45, the multi-dimensional contrast sub-task includes a time sequence scale contrast sub-task, a modal dimension contrast sub-task, and S46, the time sequence scale contrast sub-task, by sampling embedding vectors of different step lengths at adjacent or non-adjacent time steps as negative samples, constructs sample pairs with time delay feature differences, and combines the positive sample representations from different granularity branches at the same time step to establish a contrast target with time sequence perception. S47, the modal dimension contrast sub-task, based on the multi-modality of the input data source, extracts embedding representations of corresponding modal channels, constructs cross-modal contrast pairs, and the positive sample pair is composed of representations of the same event in different modalities at the same time step, and the negative sample pair is composed of embeddings between non-synchronous modalities. S48, the communication topology structure contrast sub-task, uses the topological structure difference of different nodes and their adjacent nodes in the city communication topology graph to construct a node contrast task based on graph structure sampling, and the positive sample pair selects the embedding representations of topologically adjacent nodes at the same time step, and the negative sample pair selects the embedding representations of topologically unconnected or distant nodes.

[0035] S49, use information contrast to estimate the loss function for training, respectively calculate the loss terms of the corresponding multi-dimensional contrast sub-tasks, and finally sum the sub-losses by setting the weight coefficients to construct a complete joint loss function wherein represents the number of dimensions of the contrast sub-task, is the loss weight of the dimension, is the information contrast estimation loss function in this dimension.

[0036] The application designs a multi-head contrast mapping and multi-dimensional sub-task joint optimization mechanism. By introducing structure-independent nonlinear mapping head modules on the embedding vectors output by the cross-fragment representation and the intra-fragment representation branch, the original high-dimensional representation is mapped to a low-dimensional contrast space, in which the positive and negative sample pairs are constructed using cosine similarity, and the information contrast estimation loss function is used as the training target to establish multi-dimensional contrast sub-tasks across time, modalities and topological dimensions. Each sub-task extracts sub-embedding representation through a dedicated projection path, and models the time asynchrony, modality difference and topological structure difference. Finally, the weighted joint loss function is used to realize comprehensive modeling of the potential structural consistency and discriminability in multi-scale communication features, effectively improving the semantic perception ability and feature discrimination ability of the model for abnormal patterns.

[0037] In the embodiment, S5 includes the following specific steps: S51, at each time step , a residual vector is constructed according to the patch-wise embedding vector and the in-patch embedding vector output by the multi-scale double-tower representation modeling module , which is defined as: , which is used to depict the representation difference in the global and fine-grained perspectives; S52, a residual discriminant subnetwork is introduced as the core structure of the anomaly scoring mechanism, which is a set of training state multilayer perceptrons, used to replace the traditional methods based on probability density modeling or fixed threshold, to realize nonlinear discrimination of abnormal patterns in the residual space; S53, the residual vector is input into the residual discriminant subnetwork, and an anomaly score is output, and the anomaly probability score is mapped through the Sigmoid activation function, which is used to represent the abnormality degree of the current time step communication state; S54, in the contrast learning training phase, a joint training strategy is adopted to guide the residual discriminant subnetwork and the multi-head contrast learning module to optimize cooperatively, and the training target is composed of the contrast learning loss and the residual classification loss , forming a joint loss function: ; wherein is a loss weight coefficient, used to balance the structural representation consistency and the discriminability of residual information.

[0038] The application proposes an abnormal scoring mechanism based on residual representation difference, constructs residual vectors depicting cross-scale semantic difference by calculating the residual between global (patch-wise) and local (in-patch) embedding vectors at each time step, and introduces a residual discrimination subnetwork composed of multiple layers of perception to discriminate the residual space in a nonlinear manner, thereby achieving accurate identification of abnormal patterns. Compared with traditional methods based on probability density estimation or fixed threshold, this module has stronger flexibility and expression ability. At the same time, it is jointly optimized with the multi-head contrast learning module during the training phase, coordinates the balance between embedding consistency modeling and residual discrimination ability, and takes the joint loss function as the target, thereby effectively improving the detection sensitivity and discrimination robustness of the system to weak abnormalities in the communication state.

[0039] In this embodiment, S6 includes the following specific steps: S61, when the abnormal probability score output by the abnormal scoring generation module exceeds the preset threshold, the system determines that there is a communication abnormal event at the current time step, and enters the abnormal tracing and impact assessment phase; S62, extract the abnormal time step corresponding multi-modal communication data segment and its embedding vector, combined with the city communication topology graph maintained by the system , wherein represents a set of communication nodes, represents a set of links, initializes the topology-aware state graph; S63, input the abnormal embedding representation as a query vector into the response graph generation module constructed based on the attention mechanism, and obtain the node-level and link-level attention heat maps , which are used to locate the potential abnormal propagation path; S64, combine the propagation path model established by the system , and perform inverse analysis on the key path affected by the abnormality to identify the abnormal high-risk area and infer the most possible source node and its set of affected links ; S65, based on the node weight distribution, abnormal intensity score and path dependence degree, construct an impact range evaluation function: ; wherein represents the link importance weight, is an abnormal propagation function, represents the propagation distance between the node and the source node ; S66, determine the affected business area and the influence level according to the calculation result, and pass the identification result as input to the hierarchical early warning and linkage control module to trigger the linkage response mechanism of the corresponding level.

[0040] The application proposes an abnormality positioning and influence evaluation method fusing attention mechanism and communication topology modeling. When the abnormality probability score exceeds the set threshold, the system automatically enters the abnormality tracing process. First, the multi-modal communication data segment and its embedded representation corresponding to the time step are extracted, and the topology-aware state atlas is constructed in combination with the pre-maintained urban communication topology structure. Then, the abnormal embedded vector is input to the response graph generation module as a query, and the response heat map of nodes and links is generated through the attention mechanism to realize the preliminary positioning of the abnormality propagation path. On this basis, the abnormal diffusion path is inversed in combination with the propagation path model, the source node and its influence link set are reasoned, and the influence range evaluation function is further constructed by using the node weight distribution, abnormality intensity score and path dependence degree to quantitatively evaluate the business area and level of the abnormal event influence, and the result is fed back to the hierarchical early warning module to trigger the corresponding linkage response strategy, realizing accurate and efficient communication abnormality tracing and influence control.

[0041] Embodiment 1: In order to verify the feasibility of the application in implementation, the application is applied to the abnormality monitoring and linkage early warning task of a large urban communication guarantee center. The center is responsible for the operation state supervision and abnormal response control of multiple communication sites, base station nodes, video monitoring points and routing transmission nodes in the jurisdiction. In actual operation, it is often affected by equipment failure, link congestion, adverse weather, construction disturbance and other factors, and the communication stability is challenged. Especially during the urban peak period or during the sudden event, the communication abnormality phenomenon presents obvious multi-modal coupling characteristics and dynamic diffusion trend. The traditional single-modal monitoring and static threshold detection method is difficult to accurately identify potential risks, and lacks fine-grained abnormality positioning and hierarchical response mechanism.

[0042] In the actual deployment process, first, the "city communication anomaly monitoring and early warning system based on multi-modal data fusion" proposed in the application is deployed in the communication support system. The front-end sensing layer accesses network performance indicators, log signals, video images, environmental sensors (voltage, current, temperature and humidity, electromagnetic interference intensity), and other multi-modal communication data sources from various communication nodes. By constructing a data preprocessing and alignment module, asynchronous data alignment and feature synchronization are achieved. Key statistical features, frequency domain features, and graph structure embedding vectors are extracted and encoded into time series inputs. The inputs are input into a multi-scale double-tower structure to extract long-term and short-term embedding representations. In the training phase, the system constructs multi-head projection paths based on nonlinear mapping mechanisms to complete multi-head comparison subtask training of time scale, modal dimension, and communication structure dimension in the comparison space. At the same time, a discriminant subnetwork is constructed based on global and local representation residual vectors to score the anomaly probability.

[0043] At 16:32 on a certain day, the system detected that the anomaly score exceeded the set threshold for two consecutive steps, and determined that a communication anomaly event occurred. Then the abnormal positioning module was automatically started, the link heat distribution was generated combined with the attention response map, and the communication delay anomaly between the main routing node R12 and the video monitoring node V5 was successfully located. The corresponding node attention score in the heat map is the highest. The propagation path model infers that the downstream paths G5-G8 and G9-G11 in this area have information backlog trend, and the influence range involves 6 business modules. According to the comprehensive score output by the influence evaluation function, this anomaly is evaluated as a level three early warning event, the system notifies the on-duty personnel, and instructs the communication control center to divert part of the business traffic to the backup node group B3-B4-B5, finally controls the abnormal influence range in the minimum unit and restores the normal communication delay level.

[0044] To further verify the effectiveness of the application, we analyze the data statistical results of the two weeks before and after deployment, select the abnormal events captured by the system in different time periods, positioning accuracy, response delay, false alarm rate and other core indicators for quantitative comparison. As shown in Table 1:

[0045] Table 1 Comparison results of abnormal monitoring system performance (comparison of two weeks before and after deployment)

[0046] As can be seen from the comparison result data of the abnormal monitoring system performance in Table 1, the system proposed in the application has significant application effect in actual deployment scenarios. The multi-modal fusion and multi-scale representation mechanism in the application can effectively improve the detection sensitivity and specificity of communication anomalies; the abnormality discrimination sub-network guided by contrast learning significantly enhances the discrimination ability of the abnormality score and reduces the false positive rate; the system realizes second-level judgment and node reconstruction in the linkage response strategy, guarantees the high-reliable operation of the urban communication system, and provides strong intelligent protection for the high-concurrency and low-tolerance communication environment.

[0047] The above merely describes preferred specific embodiments of the application, but the protection scope of the application is not limited thereto, and any person skilled in the art can make equivalent replacements or changes within the technical range disclosed by the application and according to the technical solution and inventive concept of the application, which should be covered within the protection scope of the application.

Claims

1. A city communication anomaly monitoring and early warning system based on multi-modal data fusion, characterized in that, The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. 2.The urban communication anomaly monitoring and early warning system based on multi-modal data fusion of claim 1, wherein, The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. 3.The urban communication anomaly monitoring and early warning system based on multi-modal data fusion of claim 2, characterized in that, The application relates to a city communication anomaly monitoring and early warning system.

4. The urban communication anomaly monitoring and early warning system based on multi-modal data fusion according to claim 2, characterized in that, The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system.

5. The urban communication anomaly monitoring and early warning system based on multi-modal data fusion according to claim 2, characterized in that, The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to a city communication anomaly monitoring and early warning system. The application relates to S31, input the modeling input sequence obtained through time synchronization and feature extraction into a multi-scale double-tower encoding structure, the modeling input sequence being a multi-modal feature fusion vector sequence under a unified time axis; S32, divide the modeling input sequence into long-term segments according to a long-term time window to obtain a long-term segment sequence, and input the long-term segment sequence into a global representation branch to extract a global embedding vector sequence through a Patch-Transformer encoder; S33, divide the modeling input sequence into short-term segments according to a short-term time window to obtain a short-term segment sequence, and input the short-term segments into a fine-grained representation branch to extract a local embedding vector sequence through a Patch-Transformer encoder; S34, the multi-scale embedding representation includes the global embedding vector sequence output by the global representation branch and the local embedding vector sequence output by the fine-grained representation branch. 6.The urban communication anomaly monitoring and early warning system based on multi-modal data fusion of claim 2, characterized in that, The S4 includes the following specific steps: S41, output the global embedding vector sequence of the cross-fragment representation branch and the intra-fragment representation branch in the multi-scale double-tower structure respectively, and Two nonlinear mapping head modules are introduced, and BatchNorm and ReLU activation operations are integrated after each hidden layer to construct a nonlinear transformation path. S42, the nonlinear mapping head module maps the original embedding vector to a low-dimensional contrast space based on the constructed nonlinear transformation path, and outputs a projection vector, which is distance measured in the contrast space through a cosine similarity function and directly input as information contrast estimation loss function; S43, in the multi-head contrast learning mechanism training process based on multi-dimensional subtasks, positive and negative sample pairs are constructed based on the global and fine-grained projection vectors output by the nonlinear mapping module, and a multi-head contrast learning mechanism is established in the form of information contrast estimation loss; the mapped vectors at the same time step form a positive sample pair, and the mapped vectors from different time steps, different modalities and different communication node sources form a negative sample pair; S44, a multi-head projection mechanism and a multi-dimensional contrast subtask construction method are introduced, an independent mapping path is configured for each dimensional contrast task based on the nonlinear mapping head module, and a multi-head projection mechanism with structural decoupling characteristics is constructed, wherein each mapping head is composed of multiple perception mechanisms and integrates BatchNorm and ReLU activation units; S45, the multi-dimensional contrast subtasks include a time scale contrast subtask, a modal dimension contrast subtask and a communication topology structure contrast subtask; S46, the time scale contrast subtask constructs a sample pair with time delay feature difference by sampling embedding vectors of different time steps at adjacent and non-adjacent time steps as negative samples, and establishes a contrast target with time perception by combining positive sample representations from different granularity branches at the same time step; S47, the modal dimension contrast subtask extracts embedding representations of corresponding modal channels based on the multi-modality of the input data source, and constructs a cross-modal contrast pair, wherein the positive sample pair is composed of representations of the same event at different modalities at the same time step, and the negative sample pair is composed of embeddings between non-synchronous modalities; S48, the communication topology structure contrast subtask utilizes the topology structure difference between different nodes and adjacent nodes in the city communication topology graph to construct a node contrast task based on graph structure sampling, the positive sample pair selects embedding representations of topologically adjacent nodes at the same time step, and the negative sample pair selects embedding representations of topologically unconnected and distant nodes. S49, training the loss function by information comparison, respectively calculating the corresponding multi-dimensional contrast sub-task loss items, and finally weighting and summing each sub-loss by setting the weight coefficient to construct a complete joint loss function. 7.The urban communication anomaly monitoring and early warning system based on multi-modal data fusion of claim 2, characterized in that, The S5 includes the following specific steps: S51、at each time step, calculate the residual vector between the global embedding vector sequence and introduce a residual discriminant subnetwork as the core structure of the anomaly scoring mechanism, which is a set of trained multi-layer perceptron; S52, inputting the residual vector into the residual discrimination sub-network, outputting an abnormal score, and mapping the abnormal score to an abnormal probability score through a Sigmoid activation function, the abnormal probability score being used to represent the abnormality degree of the communication state at the current time step; S53, in the contrast learning training stage, a joint training strategy is adopted to guide the residual discrimination sub-network and the multi-head contrast learning module to optimize cooperatively, the training target being composed of the contrast learning loss and the residual classification loss to form a joint loss function. 8.The urban communication anomaly monitoring and early warning system based on multi-modal data fusion of claim 2, characterized in that, The S6 includes the following specific steps: S61, when the abnormal probability score output by the abnormal score generation module exceeds a preset threshold, the city communication abnormality monitoring and early warning system determines that there is a communication abnormality event at the current time step, and enters the abnormality tracing and influence evaluation stage; S62, extracting the multi-modal communication data segment and the embedding vector corresponding to the abnormal time step, and initializing a topology-aware state graph in combination with the city communication topology graph maintained by the city communication abnormality monitoring and early warning system; S63, inputting the abnormal embedding representation as a query vector into a response graph generation module constructed based on an attention mechanism to obtain an attention heat map at the node level and the link level, which is used to locate the potential abnormal propagation path; S64, in combination with the propagation path model established by the city communication abnormality monitoring and early warning system, performing inversion analysis on the key path affected by the abnormality in the city communication topology graph, identifying the abnormal high-risk area, and reasoning the most possible source node and the influence link set; S65, constructing an influence range evaluation function based on the node weight distribution, the abnormal intensity score and the path dependence degree; S66, determining the affected business area and the influence level according to the comparison result of the abnormal probability score and the preset threshold, and passing the identification result as input to the hierarchical early warning and linkage control module to trigger the linkage response mechanism of the corresponding level.

Images