Abnormity detection method based on depth auto-encoder and multi-modal fusion algorithm

Abstract

The invention relates to the technical field of anomaly detection, and discloses an anomaly detection method based on a depth auto-encoder and a multi-modal fusion algorithm, which comprises the following steps: deploying various sensors on a device side, obtaining information of various devices, and converting data collected by the sensors into a transmittable network protocol for transmission; protocols of different specifications and standards are adapted through a distributed protocol adaptation technology, and types and standards of messages are unified; edge computing nodes are deployed at geographic positions close to a data source and a user, necessary computing, storage and communication capabilities are configured, and an intelligent decision module on the edge nodes is constructed. According to the method, various sensors are deployed on the equipment side, information of various equipment is obtained, data collected by the sensors are converted into transmittable network protocols for transmission, protocols of different specifications and standards are adapted through a distributed protocol adaptation technology, and the types and standards of messages are unified.

Description

Technical Field

[0001] The present invention relates to the field of anomaly detection technology, and in particular to an anomaly detection method based on a deep autoencoder and a multimodal fusion algorithm. Background Art

[0002] In modern IoT systems, millions of connected devices continuously generate massive amounts of time series data, which not only contains real-time feedback on the operating status of the devices, but also may carry multi-dimensional information such as environmental changes and user behavior. However, it is the huge volume and high dynamics of this data that bring unprecedented challenges to anomaly detection. The goal of anomaly detection is to identify events that deviate from normal patterns in these continuous and complex time series data streams, which may indicate device failures, system intrusions, or other potential crises.

[0003] The existing patent discloses a method for detecting abnormal equipment status based on a variational autoencoder (publication number CN115438692A), which belongs to the field of intelligent diagnosis and monitoring technology of mechanical systems; it includes the following steps: collecting vibration signals of mechanical equipment during normal operation; preprocessing the signals to obtain multimodal data including time domain, frequency domain and autocorrelation, and using the data as a training set after data fusion; constructing a variational autoencoder VAE network; and training the network using the training set. Due to the rarity and variety of abnormal events themselves, coupled with the noise and diversity of data, the abnormality detection technology in the technology disclosed in the patent often fails to meet the requirements of high precision and low latency. Summary of the invention

[0004] The main technical problem solved by the present invention is to provide an anomaly detection method based on a deep autoencoder and a multimodal fusion algorithm, which solves the problems in the above-mentioned background technology.

[0005] To solve the above technical problems, according to one aspect of the present invention, more specifically, an anomaly detection method based on a deep autoencoder and a multimodal fusion algorithm comprises the following steps:

[0006] S1. Deploy various sensors on the device side to obtain various device information, and convert the data collected by the sensors into a transmittable network protocol for transmission;

[0007] S2. Adapt protocols with different specifications and standards through distributed protocol adaptation technology to unify message types and standards;

[0008] S3, deploy edge computing nodes in geographical locations close to data sources and users, equip them with necessary computing, storage, and communication capabilities, and build intelligent decision-making modules on edge nodes;

[0009] S4. Use the cloud center to build a dynamic resource monitoring system, collect the resource status of edge nodes in real time, and use the distributed load balancing algorithm to dynamically adjust task allocation and data routing;

[0010] S5. Build a deep autoencoder model based on the images, audios and device operation logs in the acquisition system;

[0011] S6. Use the normal sample set to train the autoencoder to ensure that the model can effectively capture the intrinsic structure and characteristics of the data and design a multimodal fusion strategy to map the features of each modality into the same space;

[0012] S7. Use the trained autoencoder to encode and reconstruct the new samples, calculate the reconstruction error or residual as the anomaly metric, apply the multimodal fusion strategy, integrate the anomaly metrics of each modality, and obtain the final anomaly detection result.

[0013] Furthermore, in step S2, the specific steps of state machine generation and distributed protocol adaptation of the parsing algorithm are:

[0014] S201. Clarify the specifications and standards of the distributed protocol to be adapted according to the device protocol type, and design a corresponding state machine model according to the protocol specification;

[0015] S202. Write a state machine generation tool to convert the state machine design model into source code, customize the code generation template according to the target programming language and the pavilion project architecture, and ensure that the generated code is integrated into the operation and maintenance management system;

[0016] S203, implement protocol adaptation, write a message parser, which can parse the received byte stream according to the protocol specification, restore it to a meaningful message structure, and implement a message encoder to convert the local object into a byte stream that can be transmitted over the network according to the protocol specification;

[0017] S204: Integrate the multi-protocol state machine to achieve correct state migration under different message events and execute corresponding actions.

[0018] Furthermore, the specifications and standards in step S201 include message type, message structure, interaction process, and state transition rules.

[0019] Furthermore, in step S4, in the cloud-edge real-time collaborative technology of edge intelligence and distributed load balancing algorithm, edge computing nodes are deployed at geographical locations close to data sources and users, equipped with necessary computing, storage and communication capabilities, and intelligent decision-making modules are built on edge nodes, using machine learning to perform real-time analysis and decision-making on local data.

[0020] Furthermore, the cloud center builds a dynamic resource monitoring system, collects the resource status of edge nodes in real time, and uses a distributed load balancing algorithm to dynamically adjust task allocation and data routing.

[0021] Furthermore, a cloud-edge service registration and discovery mechanism is established to enable edge services to be transparently accessed by the cloud center and other edge nodes, automatically adjust the number of edge service instances according to load changes, and realize dynamic expansion and contraction of service capacity.

[0022] Furthermore, in step S7, the specific steps of anomaly detection of the deep autoencoder and the multimodal fusion algorithm are:

[0023] S701, input the images, audios and text information in the equipment operation log in the acquisition system into the system and perform data cleaning, format conversion and standardization preprocessing;

[0024] S702, constructing a deep autoencoder model, and training the autoencoder using a normal sample set;

[0025] S703. Design a multimodal fusion strategy to map the features of each modality into the same space;

[0026] S704: Encode and reconstruct the new sample using the trained autoencoder, calculate the reconstruction error or residual as an anomaly metric, apply a multimodal fusion strategy, integrate the anomaly metrics of each modality, and obtain the final anomaly detection result.

[0027] Furthermore, in step S704, the trained autoencoder encodes and reconstructs the new sample and calculates the reconstruction error, which is:

[0028]

[0029] Where C represents the calculated reconstruction error, x i Represents the original data, represents the reconstructed data, and n represents the total number of data points;

[0030] The trained autoencoder encodes and reconstructs the new sample and calculates the residual:

[0031]

[0032] Where M represents the calculated residual, x i Represents the original data, represents the reconstructed data, and n represents the total number of data points.

[0033] Furthermore, in step S702, a deep autoencoder model is constructed, including two parts: an encoder and a decoder.

[0034] Beneficial effects:

[0035] 1. The present invention deploys various sensors on the device side to obtain various device information, and converts the data collected by the sensors into a transmittable network protocol for transmission. It adapts protocols with different specifications and standards through distributed protocol adaptation technology to unify the types and standards of messages.

[0036] 2. The present invention deploys edge computing nodes at geographical locations close to data sources and users, equips them with necessary computing, storage and communication capabilities, builds intelligent decision-making modules on edge nodes, and sets up a dynamic resource monitoring system in the cloud center to collect the resource status of edge nodes in real time, and uses distributed load balancing algorithms to dynamically adjust task allocation and data routing.

[0037] 3. The present invention constructs a deep autoencoder model based on the images, audios and equipment operation logs in the acquisition system, and trains the autoencoder using a normal sample set to ensure that the model can effectively capture the intrinsic structure and characteristics of the data and design a multimodal fusion strategy to splice and map the features of each modality into the same space.

[0038] 4. The present invention uses the trained autoencoder to encode and reconstruct new samples, calculates the reconstruction error or residual as anomaly measurement, applies a multimodal fusion strategy, integrates the anomaly measurements of each modality, and obtains the final anomaly detection result. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0040] In order to make the technical solution of the present invention clearer, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0041] Example 1

[0042] like Figure 1As shown, according to one aspect of the present invention, a method for anomaly detection based on a deep autoencoder and a multimodal fusion algorithm is provided. First, various sensors are deployed on the device side to obtain various device information, and the data collected by the sensors are converted into a transmittable network protocol for transmission. The protocols of different specifications and standards are adapted through distributed protocol adaptation technology to unify the types and standards of messages. Secondly, edge computing nodes are deployed at geographical locations close to data sources and users, equipped with necessary computing, storage and communication capabilities, and intelligent decision-making modules on edge nodes are constructed. A dynamic resource monitoring system is built in the cloud center to collect the resource status of edge nodes in real time, and distributed load balancing algorithms are used to dynamically adjust task allocation and data routing. Then, a deep autoencoder model is constructed based on images, audio and device operation logs in the acquisition system, and the autoencoder is trained using a normal sample set to ensure that the model can effectively capture the intrinsic structure and characteristics of the data and design a multimodal fusion strategy to splice and map the features of each modality to the same space. Finally, the trained autoencoder is used to encode and reconstruct the new sample, and the reconstruction error or residual is calculated as an anomaly metric. The multimodal fusion strategy is applied to integrate the anomaly metrics of each modality to obtain the final anomaly detection result.

[0043] By deploying various sensors on the device side, obtaining various types of device information, and converting the data collected by the sensors into a transmittable network protocol for transmission, the protocols with different specifications and standards are adapted through distributed protocol adaptation technology to unify the message types and standards.

[0044] Example 2

[0045] Distributed protocol adaptation technology of state machine generation and parsing algorithm: First, clarify the specifications and standards of the distributed protocol to be adapted according to the device protocol type, including message type, message structure, interaction process, state transfer rules, etc., and design the corresponding state machine model according to the protocol specification; secondly, write a state machine generation tool to convert the state machine design model into source code, customize the code generation template according to the target programming language and the exhibition hall project architecture, and ensure that the generated code is integrated into the operation and maintenance management system; then implement protocol adaptation, write a message parser that can parse the received byte stream according to the protocol specification and restore it to a meaningful message structure, and implement a message encoder to convert local objects into a byte stream that can be transmitted over the network according to the protocol specification; finally, integrate multi-protocol state machines to achieve correct state migration under different message events and execute corresponding actions.

[0046] Example 3

[0047] Real-time cloud-edge collaboration technology of edge intelligence and distributed load balancing algorithm: deploy edge computing nodes in geographical locations close to data sources and users, equip them with necessary computing, storage and communication capabilities, build intelligent decision-making modules on edge nodes, and use machine learning, rule engines and other technologies to achieve real-time analysis and decision-making of local data. The cloud center builds a dynamic resource monitoring system to collect the resource status of edge nodes in real time, and uses distributed load balancing algorithms to dynamically adjust task allocation and data routing; establishes a cloud-edge service registration and discovery mechanism so that edge services can be transparently accessed by the cloud center and other edge nodes, and automatically adjusts the number of edge service instances according to load changes to achieve dynamic expansion and contraction of service capacity.

[0048] Among them, the load balancing algorithms used to dynamically adjust task allocation and data routing are:

[0049]

[0050] By deploying edge computing nodes in geographical locations close to data sources and users, equipping them with necessary computing, storage, and communication capabilities, building intelligent decision-making modules on edge nodes, and setting up a dynamic resource monitoring system in the cloud center, the resource status of edge nodes is collected in real time, and distributed load balancing algorithms are used to dynamically adjust task allocation and data routing.

[0051] That is, the trained autoencoder encodes and reconstructs the new sample and calculates the reconstruction error, which is:

[0052]

[0053] Where C represents the calculated reconstruction error, x i Represents the original data, represents the reconstructed data, and n represents the total number of data points;

[0054] The trained autoencoder encodes and reconstructs the new sample and calculates the residual:

[0055]

[0056] Where M represents the calculated residual, x i Represents the original data, represents the reconstructed data, and n represents the total number of data points.

[0057] Example 4

[0058] Anomaly detection technology based on deep autoencoder and multimodal fusion algorithm: the images, audio and text information in the acquisition system and the equipment operation log are entered into the system and the data is preprocessed by cleaning, format conversion, standardization, etc. to ensure data quality; secondly, a deep autoencoder model is constructed, including an encoder and a decoder. The autoencoder is trained using a normal sample set to ensure that the model can effectively capture the intrinsic structure and characteristics of the data; then, a multimodal fusion strategy is designed to map the features of each modality to the same space; finally, the trained autoencoder is used to encode and reconstruct the new samples, the reconstruction error or residual is calculated as the anomaly metric, and the multimodal fusion strategy is applied to integrate the anomaly metrics of each modality to obtain the final anomaly detection result.

[0059] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. An anomaly detection method based on deep autoencoder and multimodal fusion algorithm, characterized in that: The following steps are involved: S1. Deploy various sensors on the device side to obtain various device information, and convert the data collected by the sensors into a transmittable network protocol for transmission; S2. Adapt protocols with different specifications and standards through distributed protocol adaptation technology to unify message types and standards; S3, deploy edge computing nodes in geographical locations close to data sources and users, equip them with necessary computing, storage, and communication capabilities, and build intelligent decision-making modules on edge nodes; S4. Use the cloud center to build a dynamic resource monitoring system, collect the resource status of edge nodes in real time, and use the distributed load balancing algorithm to dynamically adjust task allocation and data routing; S5. Build a deep autoencoder model based on the images, audios and device operation logs in the acquisition system; S6. Use the normal sample set to train the autoencoder to ensure that the model can effectively capture the intrinsic structure and characteristics of the data and design a multimodal fusion strategy to map the features of each modality into the same space; S7. Use the trained autoencoder to encode and reconstruct the new samples, calculate the reconstruction error or residual as the anomaly metric, apply the multimodal fusion strategy, integrate the anomaly metrics of each modality, and obtain the final anomaly detection result.

2. The anomaly detection method based on deep autoencoder and multimodal fusion algorithm according to claim 1, characterized in that: In step S2, the specific steps of state machine generation and distributed protocol adaptation of the parsing algorithm are: S201. Clarify the specifications and standards of the distributed protocol to be adapted according to the device protocol type, and design a corresponding state machine model according to the protocol specification; S202. Write a state machine generation tool to convert the state machine design model into source code, customize the code generation template according to the target programming language and the pavilion project architecture, and ensure that the generated code is integrated into the operation and maintenance management system; S203, implement protocol adaptation, write a message parser, which can parse the received byte stream according to the protocol specification, restore it to a meaningful message structure, and implement a message encoder to convert the local object into a byte stream that can be transmitted over the network according to the protocol specification; S204: Integrate the multi-protocol state machine to achieve correct state migration under different message events and execute corresponding actions.

3. The anomaly detection method based on deep autoencoder and multimodal fusion algorithm according to claim 2 is characterized in that: The specifications and standards in step S201 include message type, message structure, interaction process, and state transfer rules.

4. The anomaly detection method based on deep autoencoder and multimodal fusion algorithm according to claim 1, characterized in that: In step S4, in the cloud-edge real-time collaborative technology of edge intelligence and distributed load balancing algorithm, edge computing nodes are deployed at geographical locations close to data sources and users, equipped with necessary computing, storage and communication capabilities, and intelligent decision-making modules are built on edge nodes, using machine learning to perform real-time analysis and decision-making on local data.

5. The anomaly detection method based on deep autoencoder and multimodal fusion algorithm according to claim 4 is characterized in that: The cloud center builds a dynamic resource monitoring system, collects the resource status of edge nodes in real time, and uses a distributed load balancing algorithm to dynamically adjust task allocation and data routing.

6. The anomaly detection method based on deep autoencoder and multimodal fusion algorithm according to claim 5 is characterized in that: Establishing a cloud-edge service registration and discovery mechanism enables edge services to be transparently accessed by the cloud center and other edge nodes, automatically adjusts the number of edge service instances according to load changes, and realizes dynamic expansion and contraction of service capacity.

7. The anomaly detection method based on deep autoencoder and multimodal fusion algorithm according to claim 1, characterized in that: In step S7, the specific steps of anomaly detection of the deep autoencoder and the multimodal fusion algorithm are: S701, input the images, audios and text information in the equipment operation log in the acquisition system into the system and perform data cleaning, format conversion and standardization preprocessing; S702, constructing a deep autoencoder model, and training the autoencoder using a normal sample set; S703. Design a multimodal fusion strategy to map the features of each modality into the same space; S704: Encode and reconstruct the new sample using the trained autoencoder, calculate the reconstruction error or residual as an anomaly metric, apply a multimodal fusion strategy, integrate the anomaly metrics of each modality, and obtain the final anomaly detection result.

8. The anomaly detection method based on deep autoencoder and multimodal fusion algorithm according to claim 7 is characterized in that: In step S704, the trained autoencoder encodes and reconstructs the new sample and calculates the reconstruction error, which is: Where C represents the calculated reconstruction error, x i Represents the original data, represents the reconstructed data, and n represents the total number of data points; The trained autoencoder encodes and reconstructs the new sample and calculates the residual: In the formula, M represents the calculated residual, x i Represents the original data, represents the reconstructed data, and n represents the total number of data points.

9. The anomaly detection method based on deep autoencoder and multimodal fusion algorithm according to claim 7, characterized in that: In step S702, a deep autoencoder model is constructed, including an encoder and a decoder.

Images