Data anomaly detection method and device, medium and equipment

Abstract

The invention discloses a data anomaly detection method and device, a medium and equipment, and relates to the field of data mining, and the method comprises the steps: inputting to-be-detected data into an encoder module for feature compression processing, and obtaining a plurality of feature compression data of different dimensions; the target feature compressed data is input into a decoder module for feature reconstruction processing, feature reconstruction data is obtained, and the target feature compressed data is the feature compressed data with the minimum dimension; determining reconstruction error data based on the to-be-detected data and the feature reconstruction data; and inputting the plurality of feature compression data and the reconstruction error data into an anomaly detection network, and carrying out detection processing on abnormal data to obtain an anomaly detection result. According to the data anomaly detection method and device, data anomaly detection is carried out according to the plurality of feature compression data of different dimensions corresponding to the to-be-detected data and the reconstruction error data corresponding to the to-be-detected data, abnormal data in different dimension spaces can be detected, and the accuracy and efficiency of data anomaly detection are improved.

Description

Technical Field

[0001] The present application relates to the field of data mining, and specifically to methods, devices, media and equipment for detecting anomalies in data. Background Art

[0002] When analyzing real-world datasets, a common requirement is to determine which instances are different from all other instances. These instances that are different from the majority are called anomalies. Anomalies can be caused by errors in the data, but sometimes they also indicate a new, unknown potential risk.

[0003] In recent years, anomaly detection algorithms based on deep learning have become increasingly popular and have been applied to various tasks, such as anomaly detection using deep autoencoders. However, the detection effect of this method of detecting anomalies based on reconstruction errors needs to be improved. Summary of the invention

[0004] In order to detect abnormal data more efficiently, the present application provides a data anomaly detection method, device, medium and equipment. The technical solution is as follows:

[0005] In a first aspect, the present application provides a data anomaly detection method, the method comprising:

[0006] Obtain the data to be tested;

[0007] Inputting the data to be detected into an encoder module for feature compression processing to obtain a plurality of feature compressed data of different dimensions;

[0008] Inputting the target feature compressed data into a decoder module to perform feature reconstruction processing to obtain feature reconstructed data, wherein the target feature compressed data has the smallest dimension among the plurality of feature compressed data;

[0009] Determining reconstruction error data based on the data to be detected and the feature reconstruction data;

[0010] The plurality of feature compression data and the reconstruction error data are input into an anomaly detection network to perform detection processing on the anomaly data to obtain an anomaly detection result, wherein the anomaly detection result indicates whether the data to be detected is abnormal.

[0011] Optionally, the step of inputting the plurality of feature compression data and the reconstruction error data into an anomaly detection network to perform detection processing of the anomaly data includes:

[0012] splicing the plurality of feature compression data and the reconstruction error data to obtain target feature data;

[0013] Inputting the target feature data into the anomaly detection network, performing feature transformation processing, and obtaining feature anomaly data;

[0014] Based on the characteristic abnormality data, the abnormality detection result is determined.

[0015] Optionally, the anomaly detection network includes a multi-layer neural network, and the step of inputting the target feature data into the anomaly detection network and performing feature transformation processing to obtain feature anomaly data includes:

[0016] Inputting the target feature data into the multi-layer neural network, performing feature dimensionality reduction processing, and obtaining target feature dimensionality reduction data;

[0017] The target feature reduced dimension data is subjected to feature dimension increasing processing to obtain the feature abnormal data, wherein the dimension of the feature abnormal data is the same as the dimension of the target feature data.

[0018] Optionally, the anomaly detection network is a model based on mixed Gaussian distribution, and the inputting of the plurality of feature compression data and the reconstruction error data into the anomaly detection network, performing detection processing on the anomaly data, and obtaining an anomaly detection result includes:

[0019] splicing the plurality of feature compression data and the reconstruction error data to obtain target feature data;

[0020] Inputting the target feature data into the anomaly detection network, performing distribution probability prediction, and obtaining a mixed membership, wherein the mixed membership represents the probability that the target feature data belongs to each Gaussian distribution in a plurality of Gaussian distributions;

[0021] Based on the mixed membership, the anomaly detection result is determined.

[0022] Optionally, the plurality of feature compression data include three-dimensional first feature compression data, two-dimensional second feature compression data and one-dimensional target feature compression data, the encoder module includes an input layer, a first hidden layer, a second hidden layer and a third hidden layer, and the data to be detected is input into the encoder module, and feature compression processing is performed to obtain a plurality of feature compression data of different dimensions, including:

[0023] Inputting the data to be detected into the input layer, performing feature representation processing, and obtaining embedded representation data;

[0024] Inputting the embedded representation data into a first hidden layer and performing feature compression processing to obtain the first feature compressed data;

[0025] Inputting the first feature compressed data into the second hidden layer to perform feature compression processing to obtain the second feature compressed data;

[0026] The second feature compression data is input into the third hidden layer to perform feature compression processing to obtain the target feature compression data.

[0027] Optionally, the step of inputting the target feature compressed data into a decoder module and performing feature reconstruction processing to obtain feature reconstructed data includes:

[0028] Based on the decoding function of the decoder module, the target feature compressed data is reconstructed to obtain feature reconstructed data corresponding to the target feature compressed data;

[0029] The target feature compression data is one-dimensional data; and the dimension of the feature reconstruction data is the same as the dimension of the data to be detected.

[0030] Optionally, the method further comprises:

[0031] Obtain multiple training sample data;

[0032] Input each training sample data into the initial encoder module to perform feature compression processing to obtain multiple sample feature compression data corresponding to each training sample data;

[0033] Inputting the target sample feature compressed data corresponding to each training sample data into the initial decoder module, performing feature reconstruction processing, and obtaining the sample feature reconstructed data corresponding to each training sample data, wherein the target sample feature compressed data has the smallest dimension among the corresponding multiple sample feature compressed data;

[0034] Determine sample reconstruction error data corresponding to each training sample data based on each training sample data and sample feature reconstruction data corresponding to each training sample data;

[0035] splicing a plurality of sample feature compression data corresponding to each training sample data and sample reconstruction error data corresponding to each training sample data to obtain target sample feature data corresponding to each training sample data;

[0036] Input the target sample feature data corresponding to each training sample data into the initial anomaly detection network, perform anomaly data detection processing, and obtain sample feature anomaly data corresponding to each training sample data;

[0037] Determine first loss data based on each training sample data and sample feature reconstruction data corresponding to each training sample data;

[0038] Determine second loss data based on target sample feature data corresponding to each training sample data and sample feature abnormal data corresponding to each training sample data;

[0039] Based on the first loss data and the second loss data, the initial encoder module, the initial decoder module and the initial anomaly detection network are jointly trained to obtain the encoder module, the decoder module and the anomaly detection network.

[0040] In a second aspect, the present application provides a data anomaly detection device, the device comprising:

[0041] An acquisition module is used to acquire the data to be detected;

[0042] A feature compression module is used to input the data to be detected into an encoder module, perform feature compression processing, and obtain multiple feature compression data of different dimensions;

[0043] A feature reconstruction module, used for inputting the target feature compression data into the decoder module, performing feature reconstruction processing, and obtaining feature reconstruction data, wherein the target feature compression data has the smallest dimension among the plurality of feature compression data;

[0044] A reconstruction error determination module, used to determine reconstruction error data based on the data to be detected and the feature reconstruction data;

[0045] The anomaly detection module is used to input the multiple feature compression data and the reconstruction error data into an anomaly detection network, perform detection processing on the anomaly data, and obtain an anomaly detection result, wherein the anomaly detection result indicates whether the data to be detected is abnormal.

[0046] In a third aspect, the present application provides a computer-readable storage medium, in which at least one instruction or at least one program is stored, and the at least one instruction or at least one program is loaded and executed by a processor to implement a data anomaly detection method as described in the first aspect.

[0047] In a fourth aspect, the present application provides a computer device, comprising a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement a data anomaly detection method as described in the first aspect.

[0048] In a fifth aspect, the present application provides a computer program product, wherein the computer program product includes computer instructions, and when the computer instructions are executed by a processor, a data anomaly detection method as described in the first aspect is implemented.

[0049] The data anomaly detection method, device, medium and equipment provided by this application have the following technical effects:

[0050] The solution provided by the present application obtains multiple feature compression data of different dimensions corresponding to the data to be detected through the encoder module of the automatic encoder, and obtains feature reconstruction data corresponding to the data to be detected through the decoder module of the automatic encoder; then the reconstruction error data can be determined based on the data to be detected and the feature reconstruction data; by inputting multiple feature compression data and reconstruction error data of different dimensions into the anomaly detection network, the detection and processing of the abnormal data are performed, and an abnormal detection result indicating whether the data to be detected is abnormal can be obtained. In the solution provided by the present application, multiple feature compression data and reconstruction error data of different dimensions can distinguish abnormal data from normal data in different dimensional spaces, so that abnormal data can be accurately detected, and abnormal data can be detected more efficiently.

[0051] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0053] Figure 1 It is a schematic diagram of an implementation environment of a data anomaly detection method provided in an embodiment of the present application;

[0054] Figure 2 It is a flow chart of a data anomaly detection method provided in an embodiment of the present application;

[0055] Figure 3 It is a schematic diagram of a network structure provided in an embodiment of the present application;

[0056] Figure 4 This is another schematic diagram of a network structure provided in an embodiment of the present application;

[0057] Figure 5 It is a flowchart of a training process provided by an embodiment of the present application;

[0058] Figure 6 is a schematic diagram of a data anomaly detection device provided in an embodiment of the present application;

[0059] Figure 7 It is a schematic diagram of the hardware structure of a device for implementing a data anomaly detection method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making. Artificial intelligence technology is an interdisciplinary subject that covers a wide range of fields, including both hardware-level technology and software-level technology. Basic artificial intelligence technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics.

[0061] The solution provided in the embodiments of the present application involves technologies such as deep learning (DL) of artificial intelligence.

[0062] Deep learning (DL) is a major research direction in the field of machine learning (ML). It is introduced into machine learning to make it closer to its original goal - artificial intelligence. Deep learning is the inherent laws and representation levels of learning sample data. The information obtained in the learning process is of great help in the interpretation of data such as text, images and sounds. Its ultimate goal is to enable machines to have analytical learning capabilities like humans and to recognize data such as text, images and sounds. Deep learning is a complex machine learning algorithm that has achieved results in speech and image recognition that far exceed previous related technologies. Deep learning has achieved many results in search technology, data mining, machine learning, machine translation, natural language processing, multimedia learning, speech, recommendation and personalization technology, and other related fields. Deep learning enables machines to imitate human activities such as seeing, hearing and thinking, solves many complex pattern recognition problems, and makes great progress in artificial intelligence related technologies.

[0063] The solution provided in the embodiment of the present application can be deployed in the cloud, which also involves cloud technology, etc.

[0064] Cloud technology: refers to a hosting technology that unifies hardware, software, network and other resources in a wide area network or local area network to achieve data computing, storage, processing and sharing. It can also be understood as a general term for network technology, information technology, integration technology, management platform technology and application technology based on the cloud computing business model application, which can form a resource pool and be used on demand, flexible and convenient. The back-end services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, each item may have its own identification mark in the future, and all need to be transmitted to the back-end system for logical processing. Data of different levels will be processed separately. All kinds of industry data need strong system backing support, so cloud technology needs to be supported by cloud computing. Cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The network that provides resources is called "cloud". The resources in the "cloud" are infinitely expandable in the eyes of users, and can be obtained at any time, used on demand, expanded at any time, and paid for by use. As a cloud computing basic capability provider, a cloud computing resource pool platform will be established, referred to as a cloud platform, generally referred to as Infrastructure as a Service (IaaS), and various types of virtual resources will be deployed in the resource pool for external customers to choose to use. The cloud computing resource pool mainly includes: computing devices (which can be virtualized machines, including operating systems), storage devices, and network devices.

[0065] In order to detect abnormal data more efficiently, embodiments of the present application provide data anomaly detection methods, devices, media and equipment. The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of this application. Examples of the embodiments are shown in the drawings, where the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions.

[0066] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0067] In order to facilitate understanding of the technical solutions described in the embodiments of the present application and the technical effects produced therefrom, the embodiments of the present application explain the relevant professional terms involved:

[0068] AutoEncoder (AE): It is a type of artificial neural network (ANNs) used in semi-supervised learning and unsupervised learning. Its function is to perform representation learning on the input information by taking the input information as the learning target.

[0069] It is understandable that in the specific implementation of this application, text data, transaction data, medical images and other related data are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0070] See also Figure 1 , which is a schematic diagram of an implementation environment of a data anomaly detection method provided in an embodiment of the present application, such as Figure 1 As shown, the implementation environment may include at least a client 01 and a server 02 .

[0071] Specifically, the client 01 may include devices such as smart phones, desktop computers, tablet computers, laptop computers, vehicle terminals, digital assistants, smart wearable devices, and voice interaction devices, and may also include software running in the device, such as web pages provided to users by some service providers, or applications provided to users by these service providers. Specifically, the client 01 may be used to obtain data to be detected, and the data to be detected may be system operation data of the client 01, transaction data generated by applications running on the client 01, and the like.

[0072] Specifically, the server 02 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The server 02 may include a network communication unit, a processor, and a memory, etc. The terminal and the server may be directly or indirectly connected by wired or wireless communication, and this application is not limited here. Specifically, the server 02 may be used to perform feature compression processing on the data to be detected to obtain multiple feature compression data of different dimensions; perform feature reconstruction processing on the target feature compression data with the smallest dimension in the multiple feature compression data to obtain feature reconstruction data corresponding to the data to be detected; determine reconstruction error data based on the data to be detected and the feature reconstruction data; input multiple feature compression data and reconstruction error data of different dimensions into the anomaly detection network together, perform detection processing on the abnormal data, and obtain an abnormal detection result indicating whether the data to be detected is abnormal. The server 02 may also provide a risk warning when the data to be detected is abnormal data.

[0073] The embodiments of the present application can also be implemented in combination with cloud technology. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software and network in a wide area network or a local area network to realize the calculation, storage, processing and sharing of data. It can also be understood as a general term for network technology, information technology, integration technology, management platform technology and application technology based on the cloud computing business model application. Cloud technology needs to be supported by cloud computing. Cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The network that provides resources is called a "cloud". Specifically, the server 02 and the database are located in the cloud, and the server 02 can be a physical machine or a virtualized machine.

[0074] The following introduces a data anomaly detection method provided by this application. Figure 2 This is a flow chart of a data anomaly detection method provided in an embodiment of the present application. The present application provides method operation steps as described in the embodiment or flow chart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or server product is executed, it can be executed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings (for example, in a parallel processor or multi-threaded processing environment). Please refer to Figure 2 A data anomaly detection method provided in an embodiment of the present application may include the following steps:

[0075] S210: Acquire data to be detected.

[0076] Anomaly detection based on machine learning is widely used in finance, medical care, transportation, education and other fields. For example, in the financial field, anomaly detection can be used in tasks such as customer credit assessment and investment risk assessment; in the medical field, anomaly detection can be used in tasks such as medical image diagnosis and disease prediction; in the transportation field, anomaly detection can be used in tasks such as vehicle intelligent scheduling and traffic flow prediction.

[0077] In an embodiment of the present application, the data to be detected can be divided into sequential data (such as audio data, text data, music, time series data, protein sequence data), or non-sequential data (such as image data and other data).

[0078] S220: Input the data to be detected into the encoder module and perform feature compression processing to obtain multiple feature compression data of different dimensions.

[0079] In an embodiment of the present application, the encoder module and the decoder module constitute an auto encoder (Auto Encoder, AE) based on deep learning. The choice of the auto encoder architecture may depend on the nature of the data to be detected. When the data to be detected is image data, a convolutional network may be selected. When the data to be detected is sequence data, a model based on a long short-term memory network (Long Short-Term Memory, LSTM) may be selected; the convolutional layer and the LSTM layer may also be combined, wherein the encoder module is a convolutional neural network (Convolutional Neural Networks, CNN), and the decoder module is a multi-layer LSTM network to better reconstruct the input image data.

[0080] In the related art, an autoencoder based on deep learning is used to map the input raw data to a low-dimensional feature space, and then the low-dimensional feature space is remapped back to the original space, and the original data is attempted to be reconstructed with low-dimensional features, and whether the original data is abnormal is determined based on the size of the reconstruction error. In an embodiment of the present application, in addition to the reconstruction error, the middle hidden layer output of the autoencoder, that is, the low-dimensional features output by the encoder module, is also used to detect whether the input data to be detected is abnormal. The experimental results show that the middle hidden layer output of the autoencoder and the final reconstruction error are both useful for identifying anomalies.

[0081] In one embodiment of the present application, the encoder module includes an input layer and multiple hidden layers, the multiple hidden layers include a first hidden layer, a second hidden layer and a third hidden layer, and step S220 may include:

[0082] S221: Input the data to be detected into the input layer, perform feature representation processing, and obtain embedded representation data.

[0083] That is to embed the features of the data to be detected, which can convert text characters and voice fragments into digital vectors, or reduce the dimension of sparse data to be detected to a fixed length for subsequent processing and calculation.

[0084] S222: Input the embedded representation data into the first hidden layer, perform feature compression processing, and obtain first feature compressed data.

[0085] The first characteristic compressed data is three-dimensional data.

[0086] S223: Input the first feature compression data into the second hidden layer, perform feature compression processing, and obtain second feature compression data.

[0087] The second characteristic compressed data is two-dimensional data.

[0088] S224: Input the second feature compression data into the third hidden layer to perform feature compression processing to obtain target feature compression data.

[0089] Among them, the target feature compression data is one-dimensional data.

[0090] The output of the first hidden layer is connected to the input of the second hidden layer, and the output of the second hidden layer is connected to the input of the third hidden layer; the feature compression processing of each hidden layer can also be called feature encoding or feature dimension reduction. Through the feature extraction and nonlinear transformation of each hidden layer, the embedded representation data is encoded and reduced layer by layer to obtain three-dimensional first feature compression data, two-dimensional second feature compression data and one-dimensional target feature compression data in turn. The above multiple feature compression data of different dimensions are low-dimensional feature representations of the data to be detected, including key features related to anomalies in the data to be detected.

[0091] In one embodiment of the present application, other hidden layers may be provided before the first hidden layer to output feature compression data higher than three dimensions, and the number of hidden layers may depend on the quality requirements for the potential representation of the autoencoder.

[0092] In the above embodiment, considering that the more feature compression data is input into the anomaly detection network, the greater the amount of computation required by the anomaly detection network, and it is easy to cause misjudgment, affecting the accuracy of data anomaly detection, the multiple feature compression data are limited to three-dimensional first feature compression data, two-dimensional second feature compression data and one-dimensional target feature compression data in the embodiment of the present application. On the one hand, the subsequent computational amount is not excessively increased, and on the other hand, the anomaly can be detected more accurately by compressing the data to be detected into three-dimensional space, two-dimensional space, and one-dimensional space.

[0093] S230: Input the target feature compression data into the decoder module, perform feature reconstruction processing, and obtain feature reconstruction data, wherein the target feature compression data has the smallest dimension among the multiple feature compression data.

[0094] In an embodiment of the present application, the lowest-dimensional target feature compressed data is remapped back to the space corresponding to the data to be detected, and an attempt is made to reconstruct the data to be detected using the lowest-dimensional feature representation to determine whether the data to be detected is abnormal based on the size of the reconstruction error between the feature reconstructed data and the data to be detected.

[0095] In the embodiment of the present application, the decoder module may include one or more hidden layers and an output layer, and the input of the first hidden layer is connected to the output of the last hidden layer of the encoder module. The target feature compression data is reconstructed layer by layer through one or more hidden layers, which can also be called feature decoding or feature dimension upgrading. Specifically, the reconstruction process is also achieved through nonlinear transformation and feature extraction to obtain feature reconstructed data.

[0096] In one embodiment of the present application, the target feature compressed data is reconstructed based on a decoding function of a decoder module to obtain feature reconstructed data corresponding to the target feature compressed data.

[0097] Among them, the target feature compression data is one-dimensional data; the dimension of the feature reconstruction data is the same as the dimension of the data to be detected.

[0098] The decoding function may correspond to the nonlinear transformation and feature extraction process performed by one or more hidden layers in the decoder module.

[0099] It can be understood that the processing of the encoder module and the decoder module is equivalent to the inverse operation of each other, but the reconstructed feature reconstructed data can only be approximated to the data to be detected. If the feature reconstructed data is completely identical to the data to be detected, there will be no reconstruction error.

[0100] In another embodiment of the present application, the dimension of the feature reconstruction data is lower than the dimension of the data to be detected, and in this case, the type of the autoencoder is an undercomplete autoencoder.

[0101] In the above embodiment, the decoder module of the automatic encoder reconstructs the target feature compressed data of the lowest dimension to effectively detect abnormal data through the reconstruction error between the reconstructed feature reconstructed data and the data to be detected.

[0102] S240: Determine reconstruction error data based on the data to be detected and the feature reconstruction data.

[0103] In the embodiment of the present application, the processing processes of the encoder module and the decoder module are equivalent to inverse operations, but the reconstructed feature reconstructed data can only be approximated to the data to be detected, so there is a reconstruction error. Relevant experiments have shown that the reconstruction error is useful for anomaly detection.

[0104] In one embodiment of the present application, the reconstruction error data can be obtained by calculating the Euclidean distance between the data to be detected and the feature reconstruction data; or, the reconstruction error data can be obtained by calculating the cosine similarity between the data to be detected and the feature reconstruction data.

[0105] In one embodiment of the present application, the process of the autoencoder can be summarized by the following formula:

[0106] z c1 =h1(x;θ e1 ) (1.1)

[0107] z c2 =h2(x;θ e2 ) (1.2)

[0108] z c3 =h3(x;θ e3 ) (1.3)

[0109] x′=g(z c1 θ d ) (2)

[0110] z r =f(x;x′) (3)

[0111] z=[z c1 , z c2 , z c3 , z r ] (4)

[0112] Among them, x is the data to be detected, z c1 is the target feature compression data learned by the deep autoencoder, h1(·) represents the encoding function of the entire encoder module, θ e1 is the parameter of h1(·), z c2is the second feature compression data learned by the deep autoencoder, h2(·) represents the encoding function from the input layer to the penultimate hidden layer of the encoder module, θ e2 is the parameter of h2(·), z c3 is the first feature compression data learned by the deep autoencoder, h3(·) represents the encoding function corresponding to the input layer to the third hidden layer of the encoder module, θ e3 is the parameter of h3(·), x′ is the feature reconstruction data of x, g(·) represents the decoding function, θ d is the decoding function parameter, z r is the reconstruction error data, and f(·) represents the function for calculating the reconstruction error. Specifically, z r It can be multi-dimensional or one-dimensional, and can represent multiple distance indicators, such as absolute Euclidean distance, relative Euclidean distance, cosine similarity, etc. The automatic encoder converts z c1 , z c2 , z c3 , z r The concatenated z is provided to the subsequent anomaly detection network.

[0113] S250: Input the plurality of feature compression data and the reconstruction error data into an anomaly detection network, perform detection processing on the anomaly data, and obtain an anomaly detection result, which indicates whether the data to be detected is abnormal.

[0114] In an embodiment of the present application, after multiple feature compression data and reconstructed error data are spliced, they are input into an anomaly detection network. The anomaly detection network is used to map the input multiple feature compression data and reconstructed error data to a numerical space indicating whether the feature is abnormal, and then determine the anomaly detection result to achieve anomaly detection of the data to be detected.

[0115] In one embodiment of the present application, the anomaly detection network is a multi-layer neural network, and step S250 may include:

[0116] S251: splicing multiple feature compression data and reconstruction error data to obtain target feature data.

[0117] It is feasible to perform tensor concatenation of multiple feature compression data and reconstruction error data of different dimensions to obtain target feature data.

[0118] S252: Input the target feature data into the anomaly detection network, perform feature transformation processing, and obtain feature anomaly data.

[0119] Specifically, the anomaly detection network is a symmetrical multi-layer neural network, which is equivalent to an encoder and a decoder, and realizes the conversion of target feature data to feature anomaly data through encoding and decoding. Specifically, it may include:

[0120] S2521: Input the target feature data into a multi-layer neural network to perform dimensionality reduction processing on the features to obtain target feature dimensionality reduction data.

[0121] S2522: Perform dimension-up processing on the target feature reduced-dimensional data to obtain feature abnormal data, where the dimension of the feature abnormal data is the same as the dimension of the target feature data.

[0122] The dimensionality reduction and dimensionality increase processing of the features can refer to the aforementioned embodiments and will not be elaborated here.

[0123] The anomaly detection network is a symmetrical multi-layer neural network, such as Figure 3 As shown, the dimension of the output characteristic abnormal data z' is the same as the dimension of the input target characteristic data z, and the component data of each dimension of the characteristic abnormal data can represent the abnormal score value.

[0124] A symmetric multi-layer neural network is used to map the target feature data including reconstruction error data and multiple feature compression data of different dimensions to a numerical space indicating whether the feature is abnormal. Compared with the anomaly detection method based on density estimation, the network architecture is simple, the amount of calculation is small, and the efficiency of anomaly detection can be effectively improved.

[0125] S253: Determine anomaly detection results based on the characteristic anomaly data.

[0126] Specifically, based on the last fully connected layer of the anomaly detection network, a weighted sum calculation can be performed on the component data of each dimension in the characteristic anomaly data to obtain a numerical anomaly detection result, which can indicate whether the data to be detected is abnormal. For example, if the data to be detected is the data of a transaction record, when the anomaly detection result corresponding to the data to be detected is lower than the decision threshold set by the domain expert, the transaction record is normal.

[0127] In another embodiment of the present application, the dimension of the characteristic anomaly data output by the anomaly detection network is set to be the same as the dimension of the data to be detected, and the component data in the characteristic anomaly data can indicate the abnormal possibility of the component data in the data to be detected of the same dimension.

[0128] In the above embodiment, multiple feature compression data and reconstruction error data of different dimensions are used as the input of the anomaly detection network, which can identify data that is abnormal or outliers after being compressed into a low-dimensional space such as one-dimensional, two-dimensional or three-dimensional, thereby improving the accuracy of data anomaly detection; at the same time, the feature transformation calculation amount performed by the multi-layer neural network is small, which can effectively improve the efficiency of anomaly detection.

[0129] In one embodiment of the present application, the anomaly detection network may be a model based on a mixed Gaussian distribution, and a plurality of feature compression data and reconstruction error data are input into the anomaly detection network to perform detection processing on the anomaly data to obtain an anomaly detection result, including:

[0130] S254: splicing the plurality of feature compression data and the reconstructed error data to obtain target feature data.

[0131] S255: Input the target feature data into the anomaly detection network to perform distribution probability prediction to obtain a mixed membership, where the mixed membership represents the probability that the target feature data belongs to each of the multiple Gaussian distributions.

[0132] S256: Determine anomaly detection results based on the mixed membership.

[0133] It can be understood that the anomaly detection network constructed based on the mixed Gaussian distribution uses the training sample data for data modeling during the training phase to simulate the probability density distribution of the training sample data, which is a mixed Gaussian distribution type. In the application phase, the mixed membership is obtained by inputting the target feature data into the anomaly detection network. The mixed membership represents the probability that the target feature data belongs to each Gaussian distribution in multiple Gaussian distributions, and then the likelihood estimate of the data to be detected can be calculated based on the mixed membership. If feasible, the negative log-likelihood estimate of the data to be detected can be used as an anomaly detection result indicating whether the data to be detected is abnormal.

[0134] like Figure 4 As shown in the figure, the anomaly detection network on the right is a model built based on mixed Gaussian distribution, and the output is the mixed membership π.

[0135] In the above embodiments, anomaly detection is based on density estimation, especially using mixed Gaussian distribution. The amount of calculation in the training stage and the application stage will be relatively large, but the advantage is that complex nonlinear data can be modeled and anomaly detected, and unsupervised learning can be achieved.

[0136] like Figure 5 As shown, the embodiment of the present application also provides an end-to-end training method to obtain an encoder module, a decoder module and an anomaly detection network. Specifically, the method may include:

[0137] S310: Acquire multiple training sample data.

[0138] S320: Input each training sample data into the initial encoder module to perform feature compression processing to obtain a plurality of sample feature compression data corresponding to each training sample data.

[0139] S330: Input the target sample feature compression data corresponding to each training sample data into the initial decoder module, perform feature reconstruction processing, and obtain the sample feature reconstruction data corresponding to each training sample data, wherein the target sample feature compression data has the smallest dimension among the corresponding multiple sample feature compression data.

[0140] S340: Determine sample reconstruction error data corresponding to each training sample data based on each training sample data and the sample feature reconstruction data corresponding to each training sample data.

[0141] S350: splicing the multiple sample feature compression data corresponding to each training sample data and the sample reconstruction error data corresponding to each training sample data to obtain the target sample feature data corresponding to each training sample data.

[0142] S360: Input the target sample feature data corresponding to each training sample data into the initial anomaly detection network, perform detection processing on the anomaly data, and obtain the sample feature anomaly data corresponding to each training sample data.

[0143] Steps S310 to S360 may refer to the aforementioned embodiment and will not be described in detail here.

[0144] S370: Determine first loss data based on each training sample data and sample feature reconstruction data corresponding to each training sample data.

[0145] S380: Determine second loss data based on the target sample feature data corresponding to each training sample data and the sample feature abnormality data corresponding to each training sample data.

[0146] S390: Based on the first loss data and the second loss data, jointly train the initial encoder module, the initial decoder module, and the initial anomaly detection network to obtain an encoder module, a decoder module, and an anomaly detection network.

[0147] In the embodiment of the present application, a joint loss function is also provided, as shown in formula (5), wherein N is the number of training sample data, λ1 is the loss weight of the first loss data, λ2 is the loss weight of the second loss data, and the first loss data Medium i is the i-th training sample data, x' i Reconstruct data for the sample features of the i-th training sample data, the first loss data Middle i is the target sample feature data corresponding to the i-th training sample data, z' i is the sample feature abnormal data of the i-th training sample data.

[0148]

[0149] In the above embodiments, no matter the anomaly detection network is a simple multi-layer neural network or a complex model based on mixed Gaussian distribution, unlabeled training sample data can be used to implement unsupervised or self-supervised training, thereby reducing the difficulty of training caused by the difficulty in obtaining anomaly labels.

[0150] It can be seen from the above embodiments that a data anomaly detection method provided by the present application obtains multiple feature compression data of different dimensions corresponding to the data to be detected through the encoder module of the automatic encoder, and obtains feature reconstruction data corresponding to the data to be detected through the decoder module of the automatic encoder; and then the reconstruction error data can be determined based on the data to be detected and the feature reconstruction data; by inputting multiple feature compression data and reconstruction error data of different dimensions into the anomaly detection network, the detection and processing of the abnormal data are performed, and an abnormal detection result indicating whether the data to be detected is abnormal can be obtained. In the scheme provided by the present application, multiple feature compression data and reconstruction error data of different dimensions can distinguish abnormal data from normal data in different dimensional spaces, so that abnormal data can be accurately detected, and abnormal data can be detected more efficiently.

[0151] The present application embodiment also provides a data anomaly detection device 600, such as Figure 6 As shown, the device 600 may include:

[0152] An acquisition module 610 is used to acquire data to be detected;

[0153] The feature compression module 620 is used to input the data to be detected into the encoder module, perform feature compression processing, and obtain multiple feature compression data of different dimensions;

[0154] A feature reconstruction module 630 is used to input the target feature compression data into a decoder module, perform feature reconstruction processing, and obtain feature reconstruction data, wherein the target feature compression data has the smallest dimension among the plurality of feature compression data;

[0155] A reconstruction error determination module 640, configured to determine reconstruction error data based on the data to be detected and the feature reconstruction data;

[0156] The anomaly detection module 650 is used to input the multiple feature compression data and the reconstruction error data into an anomaly detection network, perform detection processing on the anomaly data, and obtain an anomaly detection result, wherein the anomaly detection result indicates whether the data to be detected is abnormal.

[0157] In one embodiment of the present application, the anomaly detection module 650 may include:

[0158] A first splicing unit, used for splicing the plurality of feature compression data and the reconstructed error data to obtain target feature data;

[0159] A first anomaly detection unit, configured to input the target feature data into the anomaly detection network, perform feature transformation processing, and obtain feature anomaly data;

[0160] The first abnormality detection result determining unit is used to determine the abnormality detection result based on the characteristic abnormality data.

[0161] In one embodiment of the present application, the first abnormality detection unit may include:

[0162] A first dimension reduction subunit is used to input the target feature data into the multi-layer neural network, perform dimension reduction processing on the features, and obtain target feature dimension reduction data;

[0163] The first dimension-increasing subunit is used to perform dimension-increasing processing on the target feature dimension-reduced data to obtain the feature abnormal data, and the dimension of the feature abnormal data is the same as the dimension of the target feature data.

[0164] In one embodiment of the present application, the anomaly detection network is a model based on mixed Gaussian distribution, and the anomaly detection module may include:

[0165] A second splicing unit, used for splicing the plurality of feature compression data and the reconstructed error data to obtain target feature data;

[0166] A second anomaly detection unit is used to input the target feature data into the anomaly detection network, perform distribution probability prediction, and obtain a mixed membership, where the mixed membership represents the probability that the target feature data belongs to each Gaussian distribution in a plurality of Gaussian distributions;

[0167] The second abnormality detection result determination unit is used to determine the abnormality detection result based on the mixed membership.

[0168] In one embodiment of the present application, the encoder module includes an input layer and multiple hidden layers, the multiple hidden layers include a first hidden layer, a second hidden layer and a third hidden layer, and may include:

[0169] A feature representation unit, used for inputting the data to be detected into an input layer, performing feature representation processing, and obtaining embedded representation data;

[0170] A first compression unit, configured to input the embedded representation data into the first hidden layer, perform feature compression processing, and obtain first feature compressed data;

[0171] A second compression unit, used for inputting the first feature compressed data into the second hidden layer, performing feature compression processing, and obtaining second feature compressed data;

[0172] The third compression unit is used to input the second feature compression data into the third hidden layer to perform feature compression processing to obtain the target feature compression data.

[0173] In one embodiment of the present application, the feature reconstruction module 630 may include:

[0174] A reconstruction unit, configured to reconstruct the target feature compressed data based on a decoding function of the decoder module to obtain feature reconstructed data corresponding to the target feature compressed data;

[0175] The target feature compression data is one-dimensional data; and the dimension of the feature reconstruction data is the same as the dimension of the data to be detected.

[0176] In one embodiment of the present application, the device 600 may further include:

[0177] A training sample data acquisition unit, used to acquire a plurality of training sample data;

[0178] A sample feature compression unit is used to input each training sample data into the initial encoder module to perform feature compression processing to obtain a plurality of sample feature compression data corresponding to each training sample data;

[0179] A sample feature reconstruction unit is used to input the target sample feature compression data corresponding to each training sample data into the initial decoder module to perform feature reconstruction processing to obtain the sample feature reconstruction data corresponding to each training sample data, wherein the target sample feature compression data has the smallest dimension among the corresponding multiple sample feature compression data;

[0180] A sample reconstruction error determination unit, used to determine sample reconstruction error data corresponding to each training sample data based on each training sample data and sample feature reconstruction data corresponding to each training sample data;

[0181] A sample concatenation unit is used to concatenate a plurality of sample feature compression data corresponding to each training sample data and sample reconstruction error data corresponding to each training sample data to obtain target sample feature data corresponding to each training sample data;

[0182] The sample data anomaly detection unit is used to input the target sample feature data corresponding to each training sample data into the initial anomaly detection network, perform anomaly data detection processing, and obtain the sample feature anomaly data corresponding to each training sample data;

[0183] A first loss data determining unit, configured to determine first loss data based on each training sample data and sample feature reconstruction data corresponding to each training sample data;

[0184] A second loss data determining unit, configured to determine second loss data based on target sample feature data corresponding to each training sample data and sample feature abnormality data corresponding to each training sample data;

[0185] A training unit is used to jointly train the initial encoder module, the initial decoder module and the initial anomaly detection network based on the first loss data and the second loss data to obtain the encoder module, the decoder module and the anomaly detection network.

[0186] It should be noted that the device provided in the above embodiment, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0187] An embodiment of the present application provides a computer device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement a data anomaly detection method provided in the above method embodiment.

[0188] Figure 7 A schematic diagram of the hardware structure of a device for implementing a data anomaly detection method provided in an embodiment of the present application is shown, and the device may participate in constituting or include the apparatus or system provided in an embodiment of the present application. Figure 7 As shown, the device 10 may include one or more (1002a, 1002b, ..., 1002n are used to illustrate) processors 1002 (the processor 1002 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 1004 for storing data, and a transmission device 1006 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 7 The structure shown is for illustration only and does not limit the structure of the electronic device. Figure 7More or fewer components as shown, or with Figure 7 Different configurations are shown.

[0189] It should be noted that the one or more processors 1002 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the device 10 (or mobile device). As involved in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0190] The memory 1004 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the methods in the embodiments of the present application. The processor 1002 executes various functional applications and data processing by running the software programs and modules stored in the memory 1004, that is, to implement the above-mentioned data anomaly detection method. The memory 1004 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1004 may further include a memory remotely arranged relative to the processor 1002, and these remote memories may be connected to the device 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0191] The transmission device 1006 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the device 10. In one example, the transmission device 1006 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 1006 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0192] The display may be, for example, a touch screen liquid crystal display (LCD) that may enable a user to interact with a user interface of device 10 (or mobile device).

[0193] An embodiment of the present application also provides a computer-readable storage medium, which can be set in a server to store at least one instruction or at least one program related to implementing a data anomaly detection method in a method embodiment, and the at least one instruction or the at least one program is loaded and executed by the processor to implement a data anomaly detection method provided in the above method embodiment.

[0194] Optionally, in this embodiment, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0195] The embodiment of the present invention further provides a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes a data anomaly detection method provided in the above various optional implementations.

[0196] It should be noted that the above-mentioned sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. The above-mentioned specific embodiments of the present application are described. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0197] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0198] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0199] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A data anomaly detection method, characterized in that: The method comprises: Obtain the data to be tested; Inputting the data to be detected into an encoder module for feature compression processing to obtain a plurality of feature compressed data of different dimensions; Inputting the target feature compressed data into a decoder module to perform feature reconstruction processing to obtain feature reconstructed data, wherein the target feature compressed data has the smallest dimension among the plurality of feature compressed data; Determining reconstruction error data based on the data to be detected and the feature reconstruction data; The plurality of feature compression data and the reconstruction error data are input into an anomaly detection network to perform detection processing on the anomaly data to obtain an anomaly detection result, wherein the anomaly detection result indicates whether the data to be detected is abnormal.

2. The method according to claim 1, characterized in that The step of inputting the plurality of feature compression data and the reconstruction error data into an abnormality detection network to perform abnormal data detection processing includes: splicing the plurality of feature compression data and the reconstruction error data to obtain target feature data; Inputting the target feature data into the anomaly detection network, performing feature transformation processing, and obtaining feature anomaly data; Based on the characteristic abnormality data, the abnormality detection result is determined.

3. The method according to claim 2, characterized in that The anomaly detection network includes a multi-layer neural network. The target feature data is input into the anomaly detection network, and feature transformation processing is performed to obtain feature anomaly data, including: Inputting the target feature data into the multi-layer neural network, performing feature dimensionality reduction processing, and obtaining target feature dimensionality reduction data; The target feature reduced dimension data is subjected to feature dimension increasing processing to obtain the feature abnormal data, wherein the dimension of the feature abnormal data is the same as the dimension of the target feature data.

4. The method according to claim 1, characterized in that: The anomaly detection network is a model based on mixed Gaussian distribution. The plurality of feature compression data and the reconstruction error data are input into the anomaly detection network to perform detection processing on the anomaly data to obtain an anomaly detection result, including: splicing the plurality of feature compression data and the reconstruction error data to obtain target feature data; Inputting the target feature data into the anomaly detection network, performing distribution probability prediction, and obtaining a mixed membership, wherein the mixed membership represents the probability that the target feature data belongs to each Gaussian distribution in a plurality of Gaussian distributions; Based on the mixed membership, the anomaly detection result is determined.

5. The method according to claim 1, characterized in that The plurality of feature compression data includes three-dimensional first feature compression data, two-dimensional second feature compression data and one-dimensional target feature compression data, the encoder module includes an input layer, a first hidden layer, a second hidden layer and a third hidden layer, the data to be detected is input into the encoder module, and feature compression processing is performed to obtain a plurality of feature compression data of different dimensions, including: Inputting the data to be detected into the input layer, performing feature representation processing, and obtaining embedded representation data; Inputting the embedded representation data into a first hidden layer and performing feature compression processing to obtain the first feature compressed data; Inputting the first feature compressed data into the second hidden layer to perform feature compression processing to obtain the second feature compressed data; The second feature compression data is input into the third hidden layer to perform feature compression processing to obtain the target feature compression data.

6. The method according to claim 1, characterized in that The step of inputting the target feature compressed data into a decoder module and performing feature reconstruction processing to obtain feature reconstructed data comprises: Based on the decoding function of the decoder module, the target feature compressed data is reconstructed to obtain feature reconstructed data corresponding to the target feature compressed data; The target feature compression data is one-dimensional data; and the dimension of the feature reconstruction data is the same as the dimension of the data to be detected.

7. The method according to claim 1, characterized in that The method further comprises: Obtain multiple training sample data; Input each training sample data into the initial encoder module to perform feature compression processing to obtain multiple sample feature compression data corresponding to each training sample data; Inputting the target sample feature compressed data corresponding to each training sample data into the initial decoder module, performing feature reconstruction processing, and obtaining the sample feature reconstructed data corresponding to each training sample data, wherein the target sample feature compressed data has the smallest dimension among the corresponding multiple sample feature compressed data; Determine sample reconstruction error data corresponding to each training sample data based on each training sample data and sample feature reconstruction data corresponding to each training sample data; splicing a plurality of sample feature compression data corresponding to each training sample data and sample reconstruction error data corresponding to each training sample data to obtain target sample feature data corresponding to each training sample data; Input the target sample feature data corresponding to each training sample data into the initial anomaly detection network, perform anomaly data detection processing, and obtain sample feature anomaly data corresponding to each training sample data; Determine first loss data based on each training sample data and sample feature reconstruction data corresponding to each training sample data; Determine second loss data based on target sample feature data corresponding to each training sample data and sample feature abnormal data corresponding to each training sample data; Based on the first loss data and the second loss data, the initial encoder module, the initial decoder module and the initial anomaly detection network are jointly trained to obtain the encoder module, the decoder module and the anomaly detection network.

8. A data anomaly detection device, characterized in that: The device comprises: An acquisition module is used to acquire the data to be detected; A feature compression module is used to input the data to be detected into an encoder module, perform feature compression processing, and obtain multiple feature compression data of different dimensions; A feature reconstruction module, used for inputting the target feature compression data into the decoder module, performing feature reconstruction processing, and obtaining feature reconstruction data, wherein the target feature compression data has the smallest dimension among the plurality of feature compression data; A reconstruction error determination module, used to determine reconstruction error data based on the data to be detected and the feature reconstruction data; The anomaly detection module is used to input the multiple feature compression data and the reconstruction error data into an anomaly detection network, perform detection processing on the anomaly data, and obtain an anomaly detection result, wherein the anomaly detection result indicates whether the data to be detected is abnormal.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by a processor to implement a data anomaly detection method as described in any one of claims 1 to 7.

10. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement a data anomaly detection method as described in any one of claims 1 to 7.

Images