Tracking and auditing anomaly detection method and system based on multi-modal deep learning

Abstract

The invention relates to the field of audit data anomaly detection, and provides a tracking audit anomaly detection method and system based on multi-modal deep learning, and the method comprises the steps: carrying out the preprocessing of historical audit data, mapping the historical audit data to a unified feature space, and obtaining unified feature representation data; performing block processing on the unified feature representation data, and performing operation mode feature extraction and integration to obtain audit behavior feature processing data; extracting short-term local dependency features and long-term global dependency features, and performing weighted fusion to obtain audit comprehensive feature data; performing self-supervised training on the auditing comprehensive feature data, generating a pseudo mark for unlabeled data, and training together with an abnormal sample to obtain an abnormal detection model; and identifying the real-time audit data based on the anomaly detection model to obtain an anomaly audit score, and adopting a corresponding anomaly disposal scheme. According to the invention, the accuracy and real-time performance of auditing anomaly detection are improved, and risk early warning and handling response of auditing data are realized.

Description

Technical Field

[0001] The present invention relates to the field of audit data anomaly detection, and particularly to a method and system for tracking audit anomaly detection based on multi-modal deep learning. Background Art

[0002] As an important means to ensure the safe operation of the system, audit behavior anomaly monitoring needs to perform real-time analysis and risk assessment on multi-source heterogeneous audit data including text records, operation screenshots, time series data, and various structured data. These audit data cover information in multiple dimensions such as user operation logs, system behavior records, and business process data.

[0003] The prior art adopts an unsupervised data anomaly detection method, which selects the features most sensitive to anomalies by calculating the specificity parameters of the feature distribution, and performs anomaly detection based on the selected features. However, this method only focuses on the data features of a single modality and lacks the analysis of the data time series features, cannot process multi-modal audit data, and is difficult to capture the dynamic evolution pattern of abnormal behaviors. Summary of the Invention

[0004] In view of this, the present invention proposes a method and system for tracking audit anomaly detection based on multi-modal deep learning, which solves the problems that the prior art only focuses on the data features of a single modality, lacks the analysis of the data time series features, cannot process multi-modal audit data, and is difficult to capture the dynamic evolution pattern of abnormal behaviors.

[0005] The technical solution of the present invention is realized as follows: In the first aspect, the present invention provides a method for tracking audit anomaly detection based on multi-modal deep learning, including the following steps:

[0006] Collect historical audit data from different modalities, preprocess the historical audit data to obtain clean audit data, and map the clean audit data to a unified feature space to obtain unified feature representation data;

[0007] Perform block processing on the unified feature representation data to obtain a plurality of audit data blocks, and perform operation mode feature extraction and integration on the plurality of audit data blocks to obtain audit behavior feature processing data;

[0008] Adopt a multi-scale time series recognition algorithm to extract the short-term local dependence features and long-term global dependence features of the audit behavior feature reflection processing data, and perform weighted fusion on the short-term local dependence features and long-term global dependence features to obtain audit comprehensive feature data;

[0009] Use the contrastive learning method to perform self-supervised training on the audit comprehensive feature data, generate pseudo-labels for the unlabeled data, and co-train the audit comprehensive feature data with pseudo-labels and abnormal samples to obtain an anomaly detection model;

[0010] Obtain real-time audit data, identify the real-time audit data based on the anomaly detection model to obtain an abnormal audit score, and adopt corresponding abnormal disposal plans according to the abnormal audit score.

[0011] On the basis of the above technical solutions, preferably, collect historical audit data from different modalities, preprocess the historical audit data to obtain clean audit data, and map the clean audit data to a unified feature space to obtain unified feature representation data, including:

[0012] The historical audit data includes text, images, time series, and structured data. The preprocessing includes data cleaning, missing value filling, and format standardization processing;

[0013] Perform word segmentation and stop word removal processing on text data, perform size unification and pixel normalization processing on image data, perform sampling alignment and denoising processing on time series data, and perform type conversion and outlier processing on structured data to obtain the clean audit data;

[0014] Use a feature extraction network to perform feature encoding on clean audit data of different modalities respectively, map the encoded feature vectors to a feature space of a unified dimension through linear transformation, and perform L2 normalization processing to obtain the unified feature representation data.

[0015] On the basis of the above technical solutions, preferably, perform block processing on the unified feature representation data to obtain multiple audit data blocks, and perform operation mode feature extraction and integration on the multiple audit data blocks to obtain audit behavior feature processing data, including:

[0016] Perform block processing on the unified feature representation data according to time, modality, or data type to generate multiple audit data blocks;

[0017] Perform operation mode feature extraction on the multiple audit data blocks respectively. The extracted features include behavior pattern features, inter-modal association features, and time series features. Integrate the behavior pattern features, inter-modal association features, and time series features to obtain audit behavior feature processing data.

[0018] On the basis of the above technical solutions, preferably, use a multi-scale time series recognition algorithm to extract the short-term local dependence features and long-term global dependence features of the audit behavior feature reflection processing data, and perform weighted fusion on the short-term local dependence features and long-term global dependence features to obtain audit comprehensive feature data, including:

[0019] The multi-scale time series recognition algorithm is used to extract local short-term dependence features and global long-term dependence features from the processed data of the audit behavior features;

[0020] The time window is dynamically adjusted according to the volatility of the processed data of the audit behavior features, and the short-term dependence features and long-term dependence features are weighted and fused to generate audit comprehensive feature data.

[0021] On the basis of the above technical solutions, preferably, the local short-term dependence feature extraction uses a convolutional neural network to perform a sliding window operation on the processed data of the audit behavior features, and the global long-term dependence feature extraction uses a Transformer model based on the self-attention mechanism;

[0022] The calculation formula for the local short-term dependence feature is:

[0023] F local (t) = σ(W conv ·X[t:t + k] + b conv );

[0024] Among them, F local (t) is the local short-term dependence feature at time t, σ(·) is the sigmoid activation function, W conv is the convolutional kernel weight, b conv is the convolutional kernel bias, k is the convolutional kernel size, and X[t:t + k] is the processed data of the audit behavior features within the sliding window [t:t + k];

[0025] The calculation formula for the global long-term dependence feature is:

[0026]

[0027] Among them, F global is the global long-term dependence feature, Q, K, and V are the query matrix, key matrix, and value matrix respectively, d k is the scaling factor, Attention(·) is the attention weight function, and softmax(·) is the normalization function;

[0028] The calculation formula for dynamically adjusting the time window is:

[0029]

[0030] Among them, L is the length of the dynamic time window, Δ is the fluctuation amplitude of the audit behavior features, Δ thresh is the fluctuation amplitude threshold, L min is the minimum time window length, and L max is the maximum time window length;

[0031] The calculation formula for the comprehensive audit feature data is as follows:

[0032] F fusion = α·F local + (1 - α)·F global ;

[0033] α = softmax(W a ·[F local , F global + b a );

[0034] Among them, F fusion is the comprehensive audit feature data, α is the weighted coefficient of local short-term dependence features, F local is the local short-term dependence feature, W a is the fusion weight of short-term and long-term, and b a is the fusion bias.

[0035] Based on the above technical solutions, preferably, the self-supervised training of the comprehensive audit feature data is carried out by using the contrast learning method, pseudo-labels are generated for unlabeled data, and the comprehensive audit feature data with pseudo-labels is jointly trained with abnormal samples to obtain an anomaly detection model, including:

[0036] The contrast learning method constructs positive sample pairs and negative sample pairs, maximizes the mutual information between different views of the same sequence, and obtains a pre-trained model. The positive sample pairs are generated from different data augmentation views of the same comprehensive audit feature data, and the negative sample pairs are generated from different comprehensive audit feature data;

[0037] The similarity between samples is calculated based on cosine similarity. The clustering method uses a density-based clustering algorithm. The process of generating pseudo-labels for unlabeled data includes: assigning a unique pseudo-label to each cluster in the clustering result, marking samples that cannot be assigned to any cluster as noise samples, and combining the comprehensive audit feature data with pseudo-labels and known abnormal samples as training data to retrain the anomaly detection model.

[0038] Based on the above technical solutions, preferably, the real-time audit data is obtained, the real-time audit data is identified based on the anomaly detection model to obtain an abnormal audit score, and corresponding abnormal handling solutions are taken according to the abnormal audit score, including:

[0039] The real-time audit data is mapped to a unified feature space to generate a real-time feature representation, and the deviation degree between the real-time feature representation and the normal feature distribution is calculated based on the anomaly detection model to obtain an abnormal audit score;

[0040] Classify the types of anomalies in the real-time audit data according to the comparison result between the anomaly audit score and the preset threshold, and adopt corresponding anomaly handling solutions.

[0041] In a second aspect, the present invention also provides a tracking audit anomaly detection system based on multi-modal deep learning. The system includes:

[0042] An audit data processing module, configured to collect historical audit data from different modalities, preprocess the historical audit data to obtain clean audit data, and map the clean audit data to a unified feature space to obtain unified feature representation data;

[0043] A data block integration module, configured to perform block processing on the unified feature representation data to obtain a plurality of audit data blocks, perform operation mode feature extraction and integration on the plurality of audit data blocks, and obtain audit behavior feature processing data;

[0044] A time series evolution comprehensive module, configured to adopt a multi-scale time series recognition algorithm to extract short-term local dependence features and long-term global dependence features of the audit behavior feature reflection processing data, and perform weighted fusion on the short-term local dependence features and the long-term global dependence features to obtain audit comprehensive feature data;

[0045] A detection model construction module, configured to perform self-supervised training on the audit comprehensive feature data by using a contrast learning method, generate pseudo-labels for unlabeled data, and jointly train the audit comprehensive feature data with pseudo-labels and anomaly samples to obtain an anomaly detection model;

[0046] An audit score handling module, configured to obtain real-time audit data, identify the real-time audit data based on the anomaly detection model to obtain an anomaly audit score, and adopt corresponding anomaly handling solutions according to the anomaly audit score.

[0047] In a third aspect, the present invention also provides an electronic device, including: at least one processor, at least one memory, a communication interface, and a bus;

[0048] Wherein, the processor, the memory, and the communication interface complete communication with each other through the bus, the memory stores program instructions executable by the processor, and the processor calls the program instructions to implement the steps of a tracking audit anomaly detection method based on multi-modal deep learning.

[0049] In a fourth aspect, the present invention also provides a computer-readable storage medium. The computer-readable storage medium stores computer instructions, and the computer instructions enable a computer to implement the steps of a tracking audit anomaly detection method based on multi-modal deep learning.

[0050] The tracking audit anomaly detection method and system based on multimodal deep learning of the present invention have the following beneficial effects compared with the prior art:

[0051] (1) Through multimodal deep learning, audit data of different modalities are uniformly processed, short-term local and long-term global features are dynamically integrated with multi-scale time series recognition algorithms, and self-supervised training of unlabeled data is performed using contrastive learning methods and pseudo-label generation. This effectively improves the accuracy and real-time performance of audit anomaly detection, accurately captures the dynamic abnormal behavior patterns of audit data in complex business scenarios, and realizes risk warning and disposal response of audit data through anomaly scoring;

[0052] (2) Through the multi-scale time series recognition algorithm, short-term and long-term dependency features are extracted respectively, and the convolutional neural network is used to capture the local short-term behavior change pattern. The self-attention mechanism of the Transformer model is used to extract the global long-term dependency relationship, and the time window is dynamically adjusted based on the fluctuation amplitude. When the fluctuation is severe, the window is shortened to enhance the sensitivity to local anomalies, and when the fluctuation is stable, the window is expanded to strengthen the grasp of the global pattern. The features of different scales are adaptively integrated through the attention mechanism, thereby achieving a comprehensive representation of the temporal evolution characteristics of audit behavior;

[0053] (3) By constructing positive and negative sample pairs through contrastive learning methods, the mutual information between different views of the same sequence is maximized, and a pre-trained model is generated, thereby effectively extracting the deep features of the audit comprehensive feature data. The density-based clustering algorithm is used to generate pseudo-labels for unlabeled data, and the model is trained in combination with known abnormal samples. This improves the learning ability and detection accuracy of the anomaly detection model in the unlabeled data scenario. At the same time, through cosine similarity calculation and clustering optimization, accurate identification of abnormal behavior patterns is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0055] Figure 1 A flow chart of a tracking audit anomaly detection method based on multimodal deep learning of the present invention;

[0056] Figure 2 This is a structural diagram of a tracking audit anomaly detection system based on multimodal deep learning of the present invention. DETAILED DESCRIPTION

[0057] Next, in combination with the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0058] Please refer to Figure 1 , the present invention provides a tracking audit anomaly detection method based on multi-modal deep learning, including the following steps:

[0059] Collect historical audit data from different modalities, preprocess the historical audit data to obtain clean audit data, and map the clean audit data to a unified feature space to obtain unified feature representation data;

[0060] Perform block processing on the unified feature representation data to obtain multiple audit data blocks, and perform operation mode feature extraction and integration on the multiple audit data blocks to obtain audit behavior feature processing data;

[0061] Adopt a multi-scale time series recognition algorithm to extract the short-term local dependence features and long-term global dependence features of the audit behavior feature reflection processing data, and perform weighted fusion on the short-term local dependence features and long-term global dependence features to obtain audit comprehensive feature data;

[0062] Use the contrastive learning method to perform self-supervised training on the audit comprehensive feature data, generate pseudo-labels for unlabeled data, and jointly train the audit comprehensive feature data with pseudo-labels and abnormal samples to obtain an anomaly detection model;

[0063] Obtain real-time audit data, identify the real-time audit data based on the anomaly detection model to obtain an abnormal audit score, and adopt corresponding anomaly handling solutions according to the abnormal audit score.

[0064] Specifically, in this embodiment, different modalities of audit data are uniformly processed through multi-modal deep learning, the short-term local and long-term global features are dynamically fused by combining the multi-scale time series recognition algorithm, and self-supervised training of unlabeled data is carried out by using the contrastive learning method and pseudo-label generation, effectively improving the accuracy and real-time performance of audit anomaly detection, completing the accurate capture of the dynamic abnormal behavior patterns of audit data in complex business scenarios, and realizing the risk warning and disposal response of audit data through abnormal scoring.

[0065] The step of collecting historical audit data from different modalities, preprocessing the historical audit data to obtain clean audit data, and mapping the clean audit data to a unified feature space to obtain unified feature representation data includes:

[0066] The historical audit data includes text, images, time series, and structured data, and the preprocessing includes data cleaning, missing value filling, and format standardization processing;

[0067] Perform word segmentation and stop word removal on text data, perform size unification and pixel normalization on image data, perform sampling alignment and denoising on time series data, and perform type conversion and outlier processing on structured data to obtain the clean audit data.

[0068] In a specific embodiment, the processing of text data specifically includes: using the BERT pre-trained model for word segmentation, removing stop words based on a predefined stop word list, calculating the term frequency-inverse document frequency feature through the tf-idf algorithm, and performing unified truncation or padding processing on the text length; the processing of image data specifically includes: uniformly scaling the image to a preset size, performing pixel value normalization processing, and using histogram equalization to enhance the image contrast; the processing of time series data specifically includes: resampling based on a fixed time interval, and using moving average or wavelet transform for data smoothing and denoising; the processing of structured data specifically includes: converting character-type data to numerical type, filling missing values with the median, and using the z-score method to detect and process outliers.

[0069] Use a feature extraction network to perform feature encoding on the clean audit data of different modalities respectively, map the encoded feature vectors to a feature space of a unified dimension through linear transformation, and perform L2 normalization processing to obtain the unified feature representation data.

[0070] In a specific embodiment, use a Transformer encoder to extract text semantic features for text data, use a convolutional neural network to extract visual features for image data, use a long short-term memory network to extract temporal features for time series data, and use a multi-layer perceptron to extract numerical features for structured data; the linear transformation is implemented using a fully connected layer to map feature vectors of different modalities to the same-dimensional feature space; the L2 normalization processing is achieved by calculating the Euclidean norm of the feature vectors and performing standardization to ensure that different modality features have the same dimension and scale.

[0071] Specifically, in this embodiment, comprehensive preprocessing is performed on multi-modal historical audit data, including data cleaning, missing value filling, format standardization, and specific processing for different modal data, such as word segmentation for text data, normalization for image data, and denoising for time series data, effectively improving data quality and consistency; deep learning models such as BERT, convolutional neural network, long short-term memory network, and multi-layer perceptron are used to perform feature encoding on different modal data, and through linear transformation and L2 normalization, the multi-modal features are mapped to a unified feature space, ensuring the fusion and comparability of different modal features.

[0072] Performing block processing on the unified feature representation data to obtain multiple audit data blocks, and performing operation mode feature extraction and integration on the multiple audit data blocks to obtain audit behavior feature processing data, including:

[0073] Performing block processing on the unified feature representation data according to time, modality, or data type to generate multiple audit data blocks.

[0074] In a specific embodiment, the block processing includes time slicing the unified feature representation audit data based on a time window, or modality partitioning the unified feature representation audit data based on modality;

[0075] Among them, time slicing adopts a fixed time interval or a dynamic time window strategy, and the dynamic time window is adjusted according to the fluctuation characteristics of audit behavior; modality partitioning divides the data blocks into text blocks, image blocks, time series blocks, and structured data blocks according to data sources or modality characteristics.

[0076] Performing operation mode feature extraction on the multiple audit data blocks respectively, and the extracted features include behavior mode features, inter-modal association features, and time series features, and integrating the behavior mode features, inter-modal association features, and time series features to obtain audit behavior feature processing data.

[0077] In a specific embodiment, the operation mode feature extraction includes using a convolutional neural network to extract local behavior mode features, using a self-attention mechanism to extract inter-modal association features, and using a recurrent neural network to extract short-term and long-term time series features;

[0078] The feature integration is achieved through feature splicing or weighted fusion, where the weights of weighted fusion are dynamically assigned by an attention mechanism to highlight the contribution of key features to audit behavior.

[0079] Specifically, in this embodiment, by performing block processing on the unified feature representation data, which can be flexibly divided according to time, modality, or data type, it can meet the requirements of different audit scenarios; through dynamic time window adjustment, the length of time slices is dynamically adjusted according to the fluctuation characteristics of audit behaviors, enhancing the ability to capture local behavior changes and global behavior patterns; by combining convolutional neural networks, self-attention mechanisms, and recurrent neural networks, local behavior pattern features, inter-modal association features, and short-term and long-term temporal features are respectively extracted, and key features are dynamically weighted and fused through the attention mechanism, improving the expression ability of audit behavior features and the detection accuracy of complex abnormal patterns.

[0080] The multi-scale temporal recognition algorithm is adopted to extract the short-term local dependence features and long-term global dependence features of the processing data reflected by the audit behavior features, and the short-term local dependence features and long-term global dependence features are weighted and fused to obtain audit comprehensive feature data, including:

[0081] The multi-scale temporal recognition algorithm is used to extract local short-term dependence features and global long-term dependence features from the processing data of the audit behavior features;

[0082] According to the volatility of the processing data of the audit behavior features, the time window is dynamically adjusted, and the short-term dependence features and long-term dependence features are weighted and fused to generate audit comprehensive feature data.

[0083] The local short-term dependence feature extraction uses a convolutional neural network to perform sliding window operations on the processing data of the audit behavior features, and captures the short-term behavior change patterns of the processing data of the audit behavior features through local convolutional kernels;

[0084] The global long-term dependence feature extraction uses a Transformer model based on the self-attention mechanism to capture long-term behavior dependence relationships by calculating the global attention weights of each time point in the audit behavior feature sequence;

[0085] The calculation formula for the local short-term dependence features is:

[0086] F local (t) = σ(W conv ·X[t:t + k] + b conv );

[0087] Among them, F local (t) is the local short-term dependence feature at time t, σ(·) is the sigmoid activation function, W conv is the convolutional kernel weight, b conv is the convolutional kernel bias, k is the convolutional kernel size, and X[t:t + k] is the processing data of the audit behavior features within the sliding window [t:t + k];

[0088] The calculation formula for the global long-term dependence feature is as follows:

[0089]

[0090] Among them, F global is the global long-term dependence feature, Q, K, and V are the query matrix, key matrix, and value matrix respectively, d k is the scaling factor, Attention(·) is the attention weight function, and softmax(·) is the normalization function;

[0091] The process of dynamically adjusting the time window includes: calculating the dynamic time window length based on the fluctuation amplitude of the audit behavior feature, and generating the fusion weight for the weighted fusion of the short-term dependence feature and the long-term dependence feature using the attention mechanism;

[0092] The calculation formula for dynamically adjusting the time window is as follows:

[0093]

[0094] Among them, L is the length of the dynamic time window, Δ is the fluctuation amplitude of the audit behavior feature, Δ thresh is the fluctuation amplitude threshold, L min is the minimum time window length, and L max is the maximum time window length;

[0095] The calculation formula for the comprehensive audit feature data is as follows:

[0096] F fusion = α·F local +(1 - α)·F global ;

[0097] α = softmax(W a ·[F local , F global +b a );

[0098] Among them, F fusion is the comprehensive audit feature data, α is the weighted coefficient of the local short-term dependence feature, F local is the local short-term dependence feature, W a is the fusion weight of the short-term and long-term, and b a is the fusion bias;

[0099] Among them, the dynamic time window adjusts the time span by setting the fluctuation threshold. When the fluctuation amplitude exceeds the threshold, the time window is shortened to enhance the local feature capture ability. When the fluctuation amplitude is lower than the threshold, the time window is extended to enhance the global feature capture ability.

[0100] Specifically, in this embodiment, the multi-scale time series recognition algorithm is used to extract short-term and long-term dependence features respectively. The convolutional neural network is adopted to capture the local short-term behavior change patterns, and the self-attention mechanism of the Transformer model is used to extract the global long-term dependence relationship. The time window is dynamically adjusted based on the fluctuation amplitude. When the fluctuation is severe, the window is shortened to enhance the sensitivity to local anomalies. When the fluctuation is stable, the window is enlarged to strengthen the grasp of the global pattern. The features of different scales are adaptively fused through the attention mechanism, thereby achieving a comprehensive characterization of the temporal evolution features of audit behaviors.

[0101] The self-supervised training of the audit comprehensive feature data is carried out by using the contrastive learning method to generate pseudo-labels for the unlabeled data. The audit comprehensive feature data with pseudo-labels and the abnormal samples are jointly trained to obtain an anomaly detection model, including:

[0102] The contrastive learning method constructs positive sample pairs and negative sample pairs to maximize the mutual information between different views of the same sequence, and obtains a pre-trained model. The positive sample pairs are generated by different data augmentation views of the same audit comprehensive feature data, and the negative sample pairs are generated by different audit comprehensive feature data;

[0103] In a specific embodiment, the calculation formula of the contrast loss function for mutual information calculation is:

[0104]

[0105] where L contrastive is the contrast loss function, z i and are the two feature representations of the positive sample pair respectively, z j is the feature representation of the negative sample pair, sim(,) is the similarity measure between features, τ is the temperature parameter, N is the number of samples, and exp(·) is the exponential function.

[0106] The similarity between the samples is calculated based on cosine similarity. The clustering method adopts the density-based clustering algorithm. The process of generating pseudo-labels for the unlabeled data includes: assigning a unique pseudo-label to each cluster in the clustering result, marking the samples that cannot be assigned to any cluster as noise samples, and combining the audit comprehensive feature data with pseudo-labels and the known abnormal samples as training data to retrain the anomaly detection model.

[0107] Specifically, in this embodiment, the multi-scale time series recognition algorithm is used to extract short-term and long-term dependence features respectively. The convolutional neural network is adopted to capture the local short-term behavior change patterns, and the self-attention mechanism of the Transformer model is used to extract the global long-term dependence relationship. The time window is dynamically adjusted based on the fluctuation amplitude. When the fluctuation is severe, the window is shortened to enhance the sensitivity to local anomalies. When the fluctuation is stable, the window is enlarged to strengthen the grasp of the global pattern. The features of different scales are adaptively fused through the attention mechanism, thus realizing the comprehensive characterization of the time series evolution features of audit behaviors.

[0108] The obtaining of real-time audit data, the identification of the real-time audit data based on the anomaly detection model to obtain an anomaly audit score, and the adoption of corresponding anomaly handling solutions according to the anomaly audit score include:

[0109] Mapping the real-time audit data to a unified feature space to generate a real-time feature representation, and calculating the deviation degree between the real-time feature representation and the normal feature distribution based on the anomaly detection model to obtain an anomaly audit score;

[0110] In a specific embodiment, the calculation formula of the anomaly audit score is:

[0111]

[0112] where S ano is the anomaly audit score, exp(·) is the exponential function, F real is the real-time feature representation, μ nor is the mean of the normal feature distribution, and σ nor is the standard deviation of the normal feature distribution.

[0113] Classify the anomaly types of the real-time audit data according to the comparison result between the anomaly audit score and the preset threshold, and adopt corresponding anomaly handling solutions.

[0114] In a specific embodiment, if the anomaly audit score is higher than the preset threshold, trigger a high-priority alarm, and record the anomaly data and its features;

[0115] According to the anomaly type classification result, select different handling strategies, including automatic repair, manual review, and further audit analysis;

[0116] The specific method for classifying the anomaly types is:

[0117] Compare the anomaly audit score with multiple threshold intervals, and classify it as slight anomaly, significant anomaly, and severe anomaly;

[0118] A slight anomaly triggers a low-priority alarm, a significant anomaly triggers a medium-priority alarm, a severe anomaly triggers a high-priority alarm, and enters the manual review process.

[0119] Specifically, in this embodiment, by mapping real-time audit data to a unified feature space and calculating its deviation degree from the normal feature distribution to generate an abnormal audit score, the abnormal degree of audit data can be evaluated in real time and accurately; by comparing the abnormal audit score with a preset threshold, the real-time audit data is classified into slightly abnormal, significantly abnormal, and severely abnormal, and alarms with different priorities are triggered and corresponding disposal solutions are taken, realizing hierarchical management and efficient response for anomaly detection; further, combined with various disposal solutions such as automatic repair, manual review, and further audit analysis, different types and severity levels of abnormal audit situations can be effectively dealt with, improving the real-time performance and accuracy of audit anomaly detection.

[0120] Please refer to Figure 2 , the present invention also provides a tracking audit anomaly detection system based on multi-modal deep learning, and the system includes:

[0121] An audit data processing module, configured to collect historical audit data from different modalities, preprocess the historical audit data to obtain clean audit data, and map the clean audit data to a unified feature space to obtain unified feature representation data;

[0122] A data block integration module, configured to perform block processing on the unified feature representation data to obtain multiple audit data blocks, perform operation mode feature extraction and integration on the multiple audit data blocks to obtain audit behavior feature processing data;

[0123] A time series evolution integration module, configured to adopt a multi-scale time series recognition algorithm to extract short-term local dependence features and long-term global dependence features of the audit behavior feature reflection processing data, and perform weighted fusion on the short-term local dependence features and the long-term global dependence features to obtain audit comprehensive feature data;

[0124] A detection model construction module, configured to perform self-supervised training on the audit comprehensive feature data by using a contrast learning method to generate pseudo-labels for unlabeled data, and jointly train the audit comprehensive feature data with pseudo-labels and abnormal samples to obtain an anomaly detection model;

[0125] An audit score disposal module, configured to obtain real-time audit data, identify the real-time audit data based on the anomaly detection model to obtain an abnormal audit score, and adopt corresponding anomaly disposal solutions according to the abnormal audit score.

[0126] Specifically, this embodiment organically combines the processes of collecting, preprocessing, feature extraction, time series analysis, anomaly detection, and disposal of multimodal audit data to form an efficient and automated tracking audit anomaly detection system. The audit data processing module can clean, standardize, and feature-encode multimodal data to ensure data quality and consistency; the data chunk integration module improves the ability to capture complex audit behavior patterns through chunk processing and feature integration; the time series evolution comprehensive module uses a multi-scale time series recognition algorithm to dynamically fuse short-term and long-term dependence features, enhancing the comprehensive representation of the time series features of audit behaviors; the detection model construction module realizes the efficient utilization of unlabeled data through contrastive learning and pseudo-label generation, improving the robustness of the anomaly detection model; the audit scoring and disposal module combines real-time data analysis and a hierarchical disposal scheme to achieve the real-time and accuracy of anomaly audit detection.

[0127] The present invention also discloses an electronic device, including: at least one processor, at least one memory communication interface, and a bus: wherein, the processor, the memory, and the communication interface complete mutual communication through the bus; the memory stores program instructions executable by the processor, and the processor invokes the program instructions to implement a tracking audit anomaly detection method based on multimodal deep learning.

[0128] The present invention also discloses a computer-readable storage medium, the computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to implement all or part of the steps of the tracking audit anomaly detection method described in the embodiments of the present invention. The storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory ROM, random access memory RAM, magnetic disks, or optical discs that can store program codes.

[0129] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A tracking audit anomaly detection method based on multimodal deep learning, characterized in that, Including the following steps: Collect historical audit data from different modalities, preprocess the historical audit data to obtain clean audit data, and map the clean audit data to a unified feature space to obtain unified feature representation data; Perform block processing on the unified feature representation data to obtain multiple audit data blocks, and perform operation mode feature extraction and integration on the multiple audit data blocks to obtain audit behavior feature processing data; Adopt a multi-scale time series recognition algorithm to extract the short-term local dependence features and long-term global dependence features of the audit behavior feature reflection processing data, and perform weighted fusion on the short-term local dependence features and long-term global dependence features to obtain audit comprehensive feature data; Adopt a contrastive learning method to perform self-supervised training on the audit comprehensive feature data, generate pseudo-labels for unlabeled data, and jointly train the audit comprehensive feature data with pseudo-labels and abnormal samples to obtain an anomaly detection model; Obtain real-time audit data, identify the real-time audit data based on the anomaly detection model to obtain an abnormal audit score, and adopt a corresponding anomaly handling plan according to the abnormal audit score.

2. The method for anomaly detection in tracking audit based on multi-modal deep learning according to claim 1, characterized in that, The collecting historical audit data from different modalities, preprocessing the historical audit data to obtain clean audit data, and mapping the clean audit data to a unified feature space to obtain unified feature representation data includes: The historical audit data includes text, images, time series, and structured data, and the preprocessing includes data cleaning, missing value filling, and format standardization processing; Perform word segmentation and stop word removal processing on text data, perform size unification and pixel normalization processing on image data, perform sampling alignment and denoising processing on time series data, and perform type conversion and outlier processing on structured data to obtain the clean audit data; Adopt a feature extraction network to perform feature encoding on clean audit data of different modalities respectively, map the encoded feature vectors to a feature space of a unified dimension through linear transformation, and perform L2 normalization processing to obtain the unified feature representation data.

3. The method for anomaly detection in tracking audit based on multi-modal deep learning according to claim 1, characterized in that, The performing block processing on the unified feature representation data to obtain multiple audit data blocks, and performing operation mode feature extraction and integration on the multiple audit data blocks to obtain audit behavior feature processing data includes: Perform block processing on the unified feature representation data according to time, modality, or data type to generate multiple audit data blocks; Perform operation mode feature extraction on the multiple audit data blocks respectively, and the extracted features include behavior pattern features, inter-modal association features, and time series features, and integrate the behavior pattern features, inter-modal association features, and time series features to obtain audit behavior feature processing data.

4. The anomaly detection method for tracking audit based on multimodal deep learning according to claim 1, characterized in that, The adopting a multi-scale time series recognition algorithm to extract the short-term local dependence features and long-term global dependence features of the audit behavior feature reflection processing data, and performing weighted fusion on the short-term local dependence features and long-term global dependence features to obtain audit comprehensive feature data includes: Adopt a multi-scale time series recognition algorithm to extract local short-term dependence features and global long-term dependence features from the audit behavior feature processing data; Dynamically adjust the time window according to the volatility of the data processed by the audit behavior characteristics, and perform weighted fusion on the short-term dependence characteristics and long-term dependence characteristics to generate audit comprehensive feature data.

5. The anomaly detection method for tracking audit based on multi-modal deep learning according to claim 4, characterized in that The local short-term dependence feature extraction uses a convolutional neural network to perform sliding window operations on the data processed by the audit behavior characteristics, and the global long-term dependence feature extraction uses a Transformer model based on the self-attention mechanism; The calculation formula for the local short-term dependence feature is: F local (t) = σ(W conv ·X[t:t + k] + b conv ); Among them, F local (t) is the local short-term dependence feature at time t, σ(·) is the sigmoid activation function, W conv is the convolutional kernel weight, b conv is the convolutional kernel bias, k is the convolutional kernel size, and X[t:t + k] is the processed data of audit behavior features within the sliding window [t:t + k]; The calculation formula for the global long-term dependence feature is: Among them, F global is the global long-term dependence feature, Q, K, and V are the query matrix, key matrix, and value matrix respectively, and d k is the scaling factor, Attention(·) is the attention weight function, and softmax(·) is the normalization function; The calculation formula for dynamically adjusting the time window is: Among them, L is the length of the dynamic time window, Δ is the fluctuation range of the audit behavior characteristics, and Δ thresh is the fluctuation range threshold, and L min is the minimum time window length, and L max is the maximum time window length; The calculation formula for the audit comprehensive feature data is: F fusion = α·F local + (1 - α)·F global ; α = softmax(W a ·[F local , F global + b a ); Among them, F fusion is the audit comprehensive feature data, α is the weighted coefficient of local short-term dependence features, F local is the local short-term dependence feature, W a is the fusion weight of short-term and long-term, b a is the fusion bias.

6. The method for anomaly detection in tracking audit based on multi-modal deep learning according to claim 1, characterized in that, Using the contrastive learning method to perform self-supervised training on the audit comprehensive feature data, generating pseudo-labels for unlabeled data, and jointly training the audit comprehensive feature data with pseudo-labels and abnormal samples to obtain an anomaly detection model, including: The contrastive learning method constructs positive sample pairs and negative sample pairs to maximize the mutual information between different views of the same sequence, and obtains a pre-trained model. The positive sample pairs are generated from different data augmentation views of the same audit comprehensive feature data, and the negative sample pairs are generated from different audit comprehensive feature data; Calculate the similarity between samples based on cosine similarity. The clustering method uses a density-based clustering algorithm. The process of generating pseudo-labels for unlabeled data includes: assigning a unique pseudo-label to each cluster in the clustering result, marking the samples that cannot be assigned to any cluster as noise samples, and combining the audit comprehensive feature data with pseudo-labels and known abnormal samples as training data to retrain the anomaly detection model.

7. The anomaly detection method for tracking audit based on multi-modal deep learning according to claim 1, wherein, Obtain real-time audit data, identify the real-time audit data based on the anomaly detection model to obtain an anomaly audit score, and take corresponding anomaly handling solutions according to the anomaly audit score, including: Map the real-time audit data to a unified feature space to generate a real-time feature representation, and calculate the deviation degree between the real-time feature representation and the normal feature distribution based on the anomaly detection model to obtain an anomaly audit score; According to the comparison result between the anomaly audit score and the preset threshold, classify the anomaly type of the real-time audit data and take corresponding anomaly handling solutions.

8. A tracking audit anomaly detection system based on multimodal deep learning, which is used to execute a tracking audit anomaly detection method based on multimodal deep learning according to any one of claims 1-7, characterized in that, The system includes: An audit data processing module, which is used to collect historical audit data from different modalities, preprocess the historical audit data to obtain clean audit data, and map the clean audit data to a unified feature space to obtain unified feature representation data; A data block integration module, which is used to perform block processing on the unified feature representation data to obtain multiple audit data blocks, and perform operation mode feature extraction and integration on the multiple audit data blocks to obtain data processed by audit behavior characteristics; A time series evolution comprehensive module, which is used to adopt a multi-scale time series recognition algorithm to extract the short-term local dependence characteristics and long-term global dependence characteristics of the data reflected by the audit behavior characteristics, and perform weighted fusion on the short-term local dependence characteristics and long-term global dependence characteristics to obtain audit comprehensive feature data; The detection model construction module is used to perform self-supervised training on the audit comprehensive feature data by using the contrastive learning method, generate pseudo-labels for the unlabeled data, and jointly train the audit comprehensive feature data with pseudo-labels and the abnormal samples to obtain an anomaly detection model; The audit scoring and handling module is used to obtain real-time audit data, identify the real-time audit data based on the anomaly detection model to obtain an abnormal audit score, and adopt corresponding abnormal handling solutions according to the abnormal audit score.

9. An electronic device, characterized in that, It includes: At least one processor, at least one memory, a communication interface, and a bus; Wherein, the processor, the memory, and the communication interface complete mutual communication through the bus, the memory stores program instructions executable by the processor, and the processor calls the program instructions to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions, and the computer instructions cause the computer to implement the method according to any one of claims 1 to 7.

Images