Real-time anomaly detection method based on multi-modal data fusion
By combining multimodal data fusion and feature reconstruction with graph attention network and random walk algorithm, the problems of low detection accuracy and slow root cause localization in existing technologies are solved, and efficient anomaly detection is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-11
- Publication Date
- 2026-04-10
AI Technical Summary
Existing real-time anomaly detection methods can only detect based on single data points, resulting in low detection accuracy, inability to quickly locate the root cause of anomalies, and low work efficiency.
A multimodal data fusion approach is adopted, which involves building a data crawling platform to obtain information of different data types, preprocessing it to generate modeling data, using multi-scale feature extraction and cross-modal collaborative attention mechanism for feature reconstruction and fusion, and combining graph attention mechanism network and random walk algorithm for root cause localization, so as to achieve multi-dimensional data understanding and rapid localization of anomaly root causes.
It improves the accuracy and sensitivity of anomaly detection, enables rapid identification of the root cause of anomalies, and enhances the robustness and detection efficiency of the model.
Smart Images
Figure SMS_18 
Figure FDA0005734580460000021 
Figure FDA0005734580460000022
Abstract
Description
Technical Field
[0001] This invention relates to the field of data detection technology, specifically to a real-time anomaly detection method based on multimodal data fusion. Background Technology
[0002] With the rapid development of the Internet, network traffic data has become an important information medium. With the rapid growth of the Internet, the storage, analysis and anomaly detection of massive traffic data has become an important problem for operators. Many scholars have adopted host-based intrusion detection and network-based intrusion detection to achieve network security protection. Most existing researchers focus on network intrusion detection based on a single information source, such as operating system events, alarm events, log files and network traffic.
[0003] However, existing real-time anomaly detection methods still have shortcomings, specifically: existing real-time anomaly detection methods can only detect based on single data, resulting in low detection accuracy, inability to quickly locate the root cause of anomalies during detection, and low work efficiency. Summary of the Invention
[0004] The purpose of this invention is to provide a real-time anomaly detection method based on multimodal data fusion to solve the problems mentioned in the background art.
[0005] To achieve the above objectives, the present invention provides the following technical solution:
[0006] A real-time anomaly detection method based on multimodal data fusion includes the following steps:
[0007] S1. Build a data crawling platform, and obtain data information of different data types through the data crawling platform, preprocess the data information, and then generate modeling data corresponding to different data types;
[0008] S2, perform multi-scale feature extraction on the generated multimodal modeling data to obtain 3D point cloud data and data features, dynamically calculate the relationship between 3D point cloud data and data features based on cross-modal collaborative attention mechanism, dynamically reconstruct the 3D point cloud data and data features, and perform multimodal fusion on the reconstructed 3D point cloud data and data features to obtain the final fused features;
[0009] S3 constructs an anomaly detection model framework, applies a graph attention mechanism network to the fusion graph to capture the complex temporal and spatial relationships between different modalities at the global sequence level, and then constructs a root cause localization model framework. After detecting an anomaly, a random walk algorithm is used to locate the root cause in the fusion features. By comparing the weights of different root causes, the top few anomaly root causes are determined.
[0010] As a preferred embodiment of the present invention, the specific method for obtaining data information of different data types in S1 is as follows: a data crawling platform with several data warehouses is constructed, several data storage nodes are set in the data warehouses, and the data warehouses and data storage nodes are set with different storage states. The data warehouses and data storage nodes are numbered respectively, and a first-level positioning index and a second-level positioning index are generated. Then, the data storage location sequence is obtained according to the first-level positioning index and the second-level positioning index. A data crawling queue including several types of queue node units is generated in the data crawling platform, and the data crawling queue begins to crawl and classify the data.
[0011] As a preferred embodiment of the present invention, the queue node unit includes a character node unit, an audio / video node unit, and an image node unit. If the data type being captured is text data, the corresponding data is enqueued to the character node unit. If the data type being captured is audio / video data, the corresponding data is enqueued to the audio / video node unit. If the data type being captured is image data, the corresponding image data is enqueued to the image node unit.
[0012] As a preferred embodiment of the present invention, the specific method for preprocessing data information in S1 is as follows: obtaining the data format corresponding to the captured data, converting each data format into a preset data format, statistically obtaining each modeling attribute, obtaining data information of different data types and corresponding modeling attributes, and then encapsulating and generating different modeling data.
[0013] As a preferred embodiment of the present invention, the specific steps for multi-scale feature extraction of multimodal modeling data in S2 are as follows: using a convolutional neural network model to extract multi-scale data features of multimodal modeling data and generate a series of feature maps, and then using the PointNet++ point cloud processing network to perform hierarchical sampling and feature extraction on multimodal modeling data to generate three-dimensional point cloud data at different scales.
[0014] As a preferred embodiment of the present invention, the multi-scale data features of the generated multimodal modeling data are represented as follows: and Where n is the number of feature vectors, d is the dimension of the feature, and s is the index for multi-scale processing, used to distinguish features extracted at different scales. The 3D point cloud data is represented as follows: and Where m is the number of points, d is the feature dimension, and s is the index for multi-scale processing, used to distinguish features extracted at different scales.
[0015] As a preferred embodiment of the present invention, the specific steps for multimodal fusion in S2 are as follows: introducing an adaptive weight generation module to calculate the weights of each modality at different scales for the reconstructed 3D point cloud data and data features. Where g is an MLP neural network module, which performs multimodal fusion of the reconstructed image features and the point cloud features based on adaptive fusion weights to obtain the final fused features.
[0016] As a preferred embodiment of the present invention, when performing root cause localization in S3, a feature selection algorithm is used to automatically select the most relevant features. Periodic feature detection is enhanced by seasonal decomposition and spectrum analysis methods to more accurately capture periodic patterns. Furthermore, local and global features are combined in image processing and spatial data.
[0017] Compared with the prior art, the beneficial effects of the present invention are:
[0018] 1. In this invention, by acquiring modal data of different data types and comprehensively utilizing the multiple features of the data, complementary information can be provided. By fusing this information, a more comprehensive and accurate data representation can be obtained. Data of different modalities can provide different perspectives and features, as well as display different data dimensions. The fusion of multi-dimensional modal data can enhance the understanding and analysis capabilities of the constructed model for input data, and to a certain extent, enhance the robustness of the constructed model. Then, a dynamic cross-modal attention mechanism is used to accurately capture the relationship between image and point cloud features, avoiding information loss due to inaccurate projection, enabling dynamic interaction between image and point cloud features, realizing deep modal collaboration, and ensuring that features at all scales of the two modalities can be effectively utilized. During detection, a graph attention mechanism network is introduced, which can capture the complex temporal and spatial relationships between different modalities at the global sequence level, improving the accuracy and sensitivity of anomaly detection. At the same time, a random walk algorithm is used to perform root cause localization in the fused graph, which can quickly locate the root cause of the anomaly, improving the accuracy and efficiency of root cause localization.
[0019] 2. In this invention, it is achieved through...
[0020] 3. In this invention, it is achieved through... Detailed Implementation
[0021] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0022] To facilitate understanding of the present invention, a more comprehensive description of the invention will be provided below, along with several embodiments. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to make the disclosure of the invention more thorough and complete.
[0023] In this embodiment, the present invention provides a technical solution:
[0024] A real-time anomaly detection method based on multimodal data fusion includes the following steps:
[0025] S1. Build a data crawling platform, and obtain data information of different data types through the data crawling platform, preprocess the data information, and then generate modeling data corresponding to different data types;
[0026] S2, perform multi-scale feature extraction on the generated multimodal modeling data to obtain 3D point cloud data and data features, dynamically calculate the relationship between 3D point cloud data and data features based on cross-modal collaborative attention mechanism, dynamically reconstruct the 3D point cloud data and data features, and perform multimodal fusion on the reconstructed 3D point cloud data and data features to obtain the final fused features;
[0027] S3 constructs an anomaly detection model framework, applies a graph attention mechanism network to the fusion graph to capture the complex temporal and spatial relationships between different modalities at the global sequence level, and then constructs a root cause localization model framework. After detecting an anomaly, a random walk algorithm is used to locate the root cause in the fusion features. By comparing the weights of different root causes, the top few anomaly root causes are determined.
[0028] Furthermore, the specific method for obtaining data information of different data types in S1 is as follows: a data crawling platform with several data warehouses is constructed, several data storage nodes are set in the data warehouses, and the data warehouses and data storage nodes are set with different storage states. The data warehouses and data storage nodes are numbered respectively, and a first-level positioning index and a second-level positioning index are generated. Then, the data storage location sequence is obtained according to the first-level positioning index and the second-level positioning index. A data crawling queue including several types of queue node units is generated in the data crawling platform, and the data crawling queue begins to crawl and classify the data.
[0029] Furthermore, the queue node unit includes a character node unit, an audio / video node unit, and an image node unit. If the data type being captured is text data, the corresponding data is enqueued to the character node unit. If the data type being captured is audio / video data, the corresponding data is enqueued to the audio / video node unit. If the data type being captured is image data, the corresponding image data is enqueued to the image node unit.
[0030] Furthermore, the specific method for preprocessing the data information in S1 is as follows: obtain the data format corresponding to the captured data, then convert each data format into a preset data format, and statistically obtain each modeling attribute, obtain data information of different data types and corresponding modeling attributes, and then encapsulate and generate different modeling data.
[0031] Furthermore, the specific steps for multi-scale feature extraction of multimodal modeling data in S2 are as follows: a convolutional neural network model is used to extract multi-scale data features of multimodal modeling data and generate a series of feature maps. Then, the PointNet++ point cloud processing network is used to perform hierarchical sampling and feature extraction on the multimodal modeling data to generate three-dimensional point cloud data at different scales.
[0032] Furthermore, the multi-scale data features of the generated multimodal modeling data are represented as follows: and Where n is the number of feature vectors, d is the dimension of the feature, and s is the index for multi-scale processing, used to distinguish features extracted at different scales. The 3D point cloud data is represented as follows: and Where m is the number of points, d is the feature dimension, and s is the index for multi-scale processing, used to distinguish features extracted at different scales.
[0033] Furthermore, the specific steps for multimodal fusion in S2 are as follows: an adaptive weight generation module is introduced to calculate the weights of each modality at different scales for the reconstructed 3D point cloud data and data features. Where g is an MLP neural network module, which performs multimodal fusion of the reconstructed image features and the point cloud features based on adaptive fusion weights to obtain the final fused features.
[0034] Furthermore, in the root cause localization process in S3, a feature selection algorithm is used to automatically select the most relevant features. Periodic feature detection is enhanced by seasonal decomposition and spectral analysis methods to more accurately capture periodic patterns. In image processing and spatial data, local and global features are combined.
[0035] Specific Implementation Cases
[0036] A data crawling platform is constructed with several data warehouses. Each data warehouse contains several data storage nodes, and the data warehouses and data storage nodes have different storage states. The data warehouses and data storage nodes are numbered, and primary and secondary location indexes are generated. The data storage location sequence is then obtained based on the primary and secondary location indexes. A data crawling queue is generated within the data crawling platform, which includes several types of queue node units, such as character node units, audio / video node units, and image node units. If the data type to be crawled is text data, the corresponding data is enqueued into the character node unit; if the data type is audio / video data, the corresponding data is enqueued into the audio / video node unit; and if the data type is image data, the corresponding image data is enqueued into the image node unit. The data crawling queue then begins to crawl and classify the data.
[0037] The system obtains the data format corresponding to the captured data, converts each data format into a preset data format, and statistically obtains the modeling attributes of each data type. It also obtains data information and corresponding modeling attributes of different data types and encapsulates them to generate different modeling data.
[0038] A convolutional neural network model is used to extract multi-scale data features from multimodal modeling data and generate a series of feature maps. The multi-scale data features are represented as follows: and Where n is the number of feature vectors, d is the dimension of the feature, and s is the index for multi-scale processing, used to distinguish features extracted at different scales. Then, the PointNet++ point cloud processing network is used to perform hierarchical sampling and feature extraction on the multimodal modeling data, generating 3D point cloud data from different scales. The 3D point cloud data is represented as follows: and Where m is the number of points, d is the feature dimension, and s is the index for multi-scale processing, used to distinguish features extracted at different scales;
[0039] Based on the cross-modal collaborative attention mechanism, the relationship between 3D point cloud data and data features is dynamically calculated, and the 3D point cloud data and data features are dynamically reconstructed.
[0040] An adaptive weight generation module is introduced to calculate the weights of each modality at different scales for the reconstructed 3D point cloud data and data features. Where g is an MLP neural network module, which performs multimodal fusion of the reconstructed image features and the point cloud features based on adaptive fusion weights to obtain the final fused features.
[0041] In constructing the anomaly detection model framework, a graph attention mechanism network is applied to the fusion graph to capture the complex temporal and spatial relationships between different modalities at the global sequence level. Then, a root cause localization model framework is constructed. After an anomaly is detected, a random walk algorithm is used to locate the root cause in the fusion features. When locating the root cause, a feature selection algorithm is used to automatically select the most relevant features. Periodic feature detection is enhanced by seasonal decomposition and spectral analysis methods to more accurately capture periodic patterns. In image processing and spatial data, local and global features are combined, and the top anomalous root causes are determined by comparing the weights of different root causes.
[0042] All standard parts used in this application can be purchased from the market. The specific connection methods of each part adopt conventional methods such as bolts, rivets, and welding that are mature in the prior art. The machinery, parts and equipment adopt conventional models in the prior art and are also general components, which are common knowledge in this field.
[0043] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A real-time anomaly detection method based on multimodal data fusion, characterized in that, Includes the following steps: S1. Build a data crawling platform, and obtain data information of different data types through the data crawling platform, preprocess the data information, and then generate modeling data corresponding to different data types; S2, perform multi-scale feature extraction on the generated multimodal modeling data to obtain 3D point cloud data and data features, dynamically calculate the relationship between 3D point cloud data and data features based on cross-modal collaborative attention mechanism, dynamically reconstruct the 3D point cloud data and data features, and perform multimodal fusion on the reconstructed 3D point cloud data and data features to obtain the final fused features; S3 constructs an anomaly detection model framework, applies a graph attention mechanism network to the fusion graph to capture the complex temporal and spatial relationships between different modalities at the global sequence level, and then constructs a root cause localization model framework. After detecting an anomaly, a random walk algorithm is used to locate the root cause in the fusion features. By comparing the weights of different root causes, the top few anomaly root causes are determined.
2. The real-time anomaly detection method based on multimodal data fusion according to claim 1, characterized in that: The specific method for obtaining data information of different data types in S1 is as follows: a data crawling platform with several data warehouses is constructed, several data storage nodes are set in the data warehouses, and the data warehouses and data storage nodes are set with different storage states. The data warehouses and data storage nodes are numbered respectively, and a first-level positioning index and a second-level positioning index are generated. Then, the data storage location sequence is obtained according to the first-level positioning index and the second-level positioning index. A data crawling queue including several types of queue node units is generated in the data crawling platform, and the data crawling queue begins to crawl and classify the data.
3. The real-time anomaly detection method based on multimodal data fusion according to claim 2, characterized in that: The queue node unit includes a character node unit, an audio / video node unit, and an image node unit. If the data type being captured is text data, the corresponding data is enqueued into the character node unit. If the data type being captured is audio / video data, the corresponding data is enqueued into the audio / video node unit. If the data type being captured is image data, the corresponding image data is enqueued into the image node unit.
4. The real-time anomaly detection method based on multimodal data fusion according to claim 1, characterized in that: The specific method for preprocessing data information in S1 is as follows: obtain the data format corresponding to the captured data, then convert each data format into a preset data format, and statistically obtain each modeling attribute, obtain data information of different data types and corresponding modeling attributes, and then encapsulate and generate different modeling data.
5. The real-time anomaly detection method based on multimodal data fusion according to claim 1, characterized in that: The specific steps for multi-scale feature extraction of multimodal modeling data in S2 are as follows: a convolutional neural network model is used to extract multi-scale data features of multimodal modeling data and generate a series of feature maps. Then, the PointNet++ point cloud processing network is used to perform hierarchical sampling and feature extraction on the multimodal modeling data to generate three-dimensional point cloud data at different scales.
6. The real-time anomaly detection method based on multimodal data fusion according to claim 5, characterized in that: The multi-scale data features of the generated multimodal modeling data are represented as follows: and Where n is the number of feature vectors, d is the dimension of the feature, and s is the index for multi-scale processing, used to distinguish features extracted at different scales. The 3D point cloud data is represented as follows: and Where m is the number of points, d is the feature dimension, and s is the index for multi-scale processing, used to distinguish features extracted at different scales.
7. The real-time anomaly detection method based on multimodal data fusion according to claim 1, characterized in that: The specific steps for multimodal fusion in S2 are as follows: An adaptive weight generation module is introduced to calculate the weights of each modality at different scales for the reconstructed 3D point cloud data and data features. Where g is an MLP neural network module, which performs multimodal fusion of the reconstructed image features and the point cloud features based on adaptive fusion weights to obtain the final fused features.
8. The real-time anomaly detection method based on multimodal data fusion according to claim 1, characterized in that: When performing root cause localization in S3, a feature selection algorithm is used to automatically select the most relevant features. Periodic feature detection is enhanced by seasonal decomposition and spectral analysis methods to more accurately capture periodic patterns. In image processing and spatial data, local and global features are combined.