Data exception judgment method and device, computer readable storage medium and equipment

Abstract

The invention relates to the technical field of artificial intelligence, and provides a data exception judgment method, a data exception judgment device, a computer readable storage medium and electronic device.The data exception judgment method comprises the steps that feature mapping is carried out on a current data sequence to obtain mapping features, multi-head attention calculation is carried out on the mapping features, and multi-head attention calculation is carried out on the mapping features; obtaining an attention score matrix and multi-head attention features; obtaining a dependency matrix between the mapping features and historical mapping features corresponding to the historical data sequence; according to a first difference degree between the current data sequence and the multi-head attention features, a second difference degree between the multi-head attention score matrix and the dependency relationship matrix, and a third difference degree between the mapping features and historical mapping features, determining an anomaly determination score corresponding to the current data sequence; and performing anomaly judgment on the current data sequence based on the anomaly judgment score. The abnormal data can be accurately judged.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a data anomaly determination method, a data anomaly determination device, a computer-readable storage medium, and an electronic device. Background Art

[0002] In order to effectively combat the Internet's black market, e-commerce risk control platforms generally deploy a large number of risk perception models, rules or strategies, and conduct anomaly monitoring and inspections based on the relevant data hit by these models, rules or strategies, so as to perceive risks and capture anomalies in a timely manner.

[0003] In the related art, data anomaly determination is generally performed based on statistical discrimination techniques. However, the above solutions require a large amount of human resources and the determination accuracy is insufficient.

[0004] In view of this, there is an urgent need in this field to develop a new data anomaly determination method and device.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of this disclosure. Summary of the Invention

[0006] The purpose of the present disclosure is to provide a data anomaly determination method, a data anomaly determination device, a computer-readable storage medium, and an electronic device, thereby overcoming, at least to a certain extent, the technical problem of insufficient determination accuracy caused by the limitations of related technologies.

[0007] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by practice of the present disclosure.

[0008] According to a first aspect of the present disclosure, a method for determining data anomaly is provided, comprising:

[0009] Perform feature mapping on the current data sequence to obtain mapping features, and perform multi-head attention calculation on the mapping features to obtain an attention score matrix and multi-head attention features; the mapping features contain multiple sub-features, and the attention score matrix is used to represent the deep correlation relationship between the multiple sub-features;

[0010] Obtaining a dependency matrix between the mapping feature and historical mapping features corresponding to the historical data sequence;

[0011] Determining an abnormality judgment score corresponding to the current data sequence based on a first difference between the current data sequence and the multi-head attention feature, a second difference between the multi-head attention score matrix and the dependency matrix, and a third difference between the mapping feature and the historical mapping feature;

[0012] An abnormality determination is performed on the current data sequence based on the abnormality determination score.

[0013] In an exemplary embodiment of the present disclosure, performing multi-head attention calculation on the mapping feature to obtain an attention score matrix and a multi-head attention feature includes:

[0014] Performing position encoding on the mapping feature to obtain a position encoding feature;

[0015] Multi-head attention calculation is performed on the combined features of the mapping features and the position encoding features to obtain the attention score matrix and the multi-head attention features.

[0016] In an exemplary embodiment of the present disclosure, performing position encoding on the mapping feature to obtain a position encoding feature includes:

[0017] A feature group is extracted based on a sliding window; the width of the sliding window is w, and the feature group includes w-1 historical mapping features corresponding to w-1 historical data sequences and the mapping feature; w is an integer greater than 1;

[0018] Performing absolute position encoding on the mapping feature according to the position index type corresponding to the mapping feature to obtain a first encoding feature;

[0019] determining a second coding feature corresponding to the mapping feature based on a position index of the mapping feature, a position index of each historical mapping feature, a timestamp associated with the mapping feature, and a timestamp associated with each historical mapping feature;

[0020] The position coding feature is determined based on the first coding feature in combination with the second coding feature.

[0021] In an exemplary embodiment of the present disclosure, performing absolute position encoding on the mapping feature according to the position index type corresponding to the mapping feature to obtain a first encoding feature includes:

[0022] When the position index type is an even type, performing absolute position encoding on the mapping feature based on a sine function to obtain the first encoding feature;

[0023] When the position index type is an odd type, absolute position encoding is performed on the mapping feature based on a cosine function to obtain the first encoding feature.

[0024] In an exemplary embodiment of the present disclosure, determining the second coding feature corresponding to the mapping feature according to the position index of the mapping feature, the position index of each historical mapping feature, the timestamp associated with the mapping feature, and the timestamp associated with each historical mapping feature includes:

[0025] Obtaining a first difference between the position index of the mapping feature and the position index of each historical mapping feature;

[0026] determining a relative position code from the preset reference value and the first difference value according to a numerical relationship between the preset reference value and the first difference value;

[0027] Obtaining a second difference between the timestamp associated with the mapping feature and the timestamp associated with each historical mapping feature;

[0028] determining each intermediate position code according to the relative position code in combination with the second difference;

[0029] A second coding feature corresponding to the mapping feature is determined according to the accumulated values of the w-1 intermediate position codes.

[0030] In an exemplary embodiment of the present disclosure, performing multi-head attention calculation on the combined features of the mapping features and the position encoding features to obtain the attention score matrix and the multi-head attention features includes:

[0031] Extracting a plurality of segment features corresponding to a plurality of sequence lengths from the combined features;

[0032] Performing self-attention processing on each segment feature to obtain a first attention score and a first self-attention result corresponding to the sequence length;

[0033] Determining the attention score matrix according to a plurality of first attention scores corresponding one-to-one to a plurality of sequence lengths;

[0034] The multi-head attention feature is determined based on multiple first self-attention results corresponding one-to-one to multiple sequence lengths.

[0035] In an exemplary embodiment of the present disclosure, performing self-attention processing on the segment features to obtain a first attention score and a first self-attention result corresponding to the sequence length includes:

[0036] Multiplying the segment feature by the query parameter to obtain a first query feature;

[0037] Multiplying the segment feature by the key parameter to obtain a first key feature;

[0038] Multiplying the segment feature by the value parameter to obtain a first value feature;

[0039] performing association processing based on the first query feature and the first key feature to obtain a first attention score;

[0040] The first attention score is multiplied by the first value feature to obtain a first self-attention result corresponding to the sequence length.

[0041] In an exemplary embodiment of the present disclosure, performing association processing based on the first query feature and the first key feature to obtain a first attention score includes:

[0042] Performing a dot product process on the first query feature and the transpose of the first key feature to obtain a first dot product result, and performing a square root process on the number of the segment features to obtain a first square root result;

[0043] Perform maximum likelihood processing on the ratio of the first dot product result and the first square root result to obtain the first attention score.

[0044] In an exemplary embodiment of the present disclosure, determining the multi-head attention feature according to a plurality of first self-attention results corresponding one-to-one to a plurality of sequence lengths includes:

[0045] Performing splicing processing on a plurality of first self-attention results corresponding one-to-one to a plurality of sequence lengths to obtain a first splicing feature;

[0046] Perform a first full-connection process on the first splicing feature to obtain the multi-head attention feature.

[0047] In an exemplary embodiment of the present disclosure, obtaining a dependency matrix between the mapping feature and the historical mapping feature corresponding to the historical data sequence includes:

[0048] Select each baseline feature from the feature group consisting of w-1 historical mapping features and the mapping feature;

[0049] Calculating a distance index between the reference feature and other features using a Gaussian kernel function; the other features include the w-1 historical mapping features and the mapping feature;

[0050] The dependency matrix is determined according to w*w distance indicators.

[0051] In an exemplary embodiment of the present disclosure, determining the abnormality judgment score corresponding to the current data sequence based on a first difference between the current data sequence and the multi-head attention feature, a second difference between the multi-head attention score matrix and the dependency matrix, and a third difference between the mapping feature and the historical mapping feature includes:

[0052] Determining a first anomaly determination score based on a first difference between the current data sequence and the multi-head attention feature and a second difference between the multi-head attention score matrix and the dependency matrix;

[0053] determining a second abnormality determination score according to a third degree of difference between the mapping feature and the historical mapping feature;

[0054] An abnormality determination score corresponding to the current data sequence is determined according to the first abnormality determination score and the second abnormality determination score.

[0055] In an exemplary embodiment of the present disclosure, the first difference between the current data sequence and the multi-head attention feature is determined by:

[0056] Obtaining a second norm distance between the current data sequence and the multi-head attention feature;

[0057] The second norm distance is determined as the first difference.

[0058] In an exemplary embodiment of the present disclosure, the second difference between the multi-head attention score matrix and the dependency matrix is determined by:

[0059] Obtaining a first divergence between the multi-head attention score matrix and the dependency matrix, and obtaining a second divergence between the dependency matrix and the multi-head attention score matrix;

[0060] Accumulating the first divergence and the second divergence to obtain a third divergence;

[0061] The second difference is determined according to the first norm distance of the third divergence.

[0062] In an exemplary embodiment of the present disclosure, determining a first abnormality determination score based on a first difference between the current data sequence and the multi-head attention feature in combination with a second difference between the multi-head attention score matrix and the dependency matrix includes:

[0063] Obtaining a first product of the second difference and a preset weight coefficient;

[0064] calculating a third difference between the first difference and the first product;

[0065] The inverse of the third difference is obtained, and the third difference is normalized to obtain the first abnormality determination score.

[0066] In an exemplary embodiment of the present disclosure, determining the second abnormality determination score according to the third difference between the mapping feature and the historical mapping feature includes:

[0067] Extracting K designated historical mapping features adjacent to the mapping feature from the feature memory unit; K is an integer greater than 1;

[0068] calculating a norm distance between the mapping feature and each of the specified historical mapping features;

[0069] A weighted process is performed on the K norm distances to obtain the second abnormality determination score.

[0070] In an exemplary embodiment of the present disclosure, determining the abnormality determination score corresponding to the current data sequence according to the first abnormality determination score in combination with the second abnormality determination score includes:

[0071] performing normalization processing on the first abnormality determination score to obtain a first normalized value;

[0072] performing normalization processing on the second abnormality determination score to obtain a second normalized value;

[0073] The first normalized value and the second normalized value are weighted to obtain the abnormality determination score.

[0074] In an exemplary embodiment of the present disclosure, normalizing the first abnormality determination score to obtain a first normalized value includes:

[0075] Obtaining a first mean and a first standard deviation of a plurality of historical first anomaly determination scores;

[0076] A first normalized value is determined based on the first abnormality determination score, the first mean, and the first standard deviation.

[0077] In an exemplary embodiment of the present disclosure, normalizing the second abnormality determination score to obtain a second normalized value includes:

[0078] Obtain a second mean and a second standard deviation of a plurality of historical second anomaly determination scores;

[0079] A second normalized value is determined based on the second abnormality determination score, the second mean, and the second standard deviation.

[0080] In an exemplary embodiment of the present disclosure, after determining the second abnormality determination score, the method further includes:

[0081] comparing the first abnormality determination score and the second abnormality determination score;

[0082] If the second abnormality determination score is smaller than the first abnormality determination score, the mapping feature is updated into the feature memory unit.

[0083] According to a second aspect of the present disclosure, there is provided a data anomaly determination device, comprising:

[0084] A feature processing module is used to perform feature mapping on the current data sequence to obtain mapping features, and perform multi-head attention calculation on the mapping features to obtain an attention score matrix and multi-head attention features; the mapping features contain multiple sub-features, and the attention score matrix is used to represent the deep correlation relationship between the multiple sub-features;

[0085] A dependency relationship acquisition module, configured to acquire a dependency relationship matrix between the mapping feature and the historical mapping features corresponding to the historical data sequence;

[0086] an abnormality determination score determination module, configured to determine an abnormality determination score corresponding to the current data sequence based on a first difference between the current data sequence and the multi-head attention feature, a second difference between the multi-head attention score matrix and the dependency matrix, and a third difference between the mapping feature and the historical mapping feature;

[0087] An anomaly determination module is configured to perform an anomaly determination on the current data sequence based on the anomaly determination score.

[0088] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the data anomaly determination method described in the first aspect is implemented.

[0089] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the data anomaly determination method described in the first aspect above by executing the executable instructions.

[0090] As can be seen from the above technical solutions, the data anomaly determination method, data anomaly determination device, computer-readable storage medium, and electronic device in the exemplary embodiments of the present disclosure have at least the following advantages and positive effects:

[0091] In the technical solutions provided by some embodiments of the present disclosure, on the one hand, by performing feature mapping on the current data sequence to obtain mapping features, and further using multi-head attention calculation to obtain attention score matrix and multi-head attention features, this process can not only capture the location information in the data, but also effectively identify the deep correlation between different sub-features. This method is particularly suitable for data sets with complex patterns (such as seasonality and periodicity) because it can improve the accuracy of anomaly detection by learning these patterns. Furthermore, by obtaining the dependency matrix between the current mapping features and the historical mapping features corresponding to the historical data sequence, the model's ability to understand time series data can be further enhanced, allowing the system to not only consider the immediate data characteristics, but also make more accurate predictions and anomaly judgments based on historical data, thereby improving the stability and reliability of the model. Furthermore, by comprehensively considering the differences between the current data sequence and the multi-head attention features, the differences between the multi-head attention score matrix and the dependency matrix, and the differences between the mapping features and the historical mapping features, the state of the current data sequence can be comprehensively evaluated. This multi-level and multi-dimensional comparison method makes anomaly judgment more accurate and reduces the possibility of false alarms. On the other hand, anomaly judgment is performed on the current data sequence by calculating the anomaly judgment score based on the various differences mentioned above. This method provides more detailed and accurate anomaly detection results by combining multiple factors (real-time data characteristics, historical data dependencies, correlations between sub-features, etc.), which helps to discover potential problems in a timely manner, reduce system risks caused by abnormal online model rule indicators, and ensure the stable operation of the system.

[0092] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0094] Figure 1 A schematic diagram showing a flow chart of a method for determining data anomaly in an embodiment of the present disclosure;

[0095] Figure 2 A flowchart illustrating how to perform multi-head attention calculation on mapping features to obtain an attention score matrix and multi-head attention features in an embodiment of the present disclosure;

[0096] Figure 3A schematic diagram illustrating a flow chart of how to position encode a mapping feature to obtain a position encoding feature in an embodiment of the present disclosure;

[0097] Figure 4 A schematic diagram of a process for determining a second coding feature corresponding to a mapping feature according to a position index of the mapping feature, a position index of each historical mapping feature, a timestamp associated with the mapping feature, and a timestamp associated with each historical mapping feature in an embodiment of the present disclosure is shown;

[0098] Figure 5 A flowchart illustrating how to perform multi-head attention calculation on the combined features of mapping features and position encoding features to obtain an attention score matrix and multi-head attention features in an embodiment of the present disclosure;

[0099] Figure 6 A flowchart illustrating how to perform self-attention processing on each segment feature to obtain a first attention score and a first self-attention result corresponding to the sequence length in an embodiment of the present disclosure;

[0100] Figure 7 A schematic diagram illustrating a process of obtaining a dependency matrix between mapping features and historical mapping features corresponding to a historical data sequence in an embodiment of the present disclosure;

[0101] Figure 8 A schematic diagram illustrating a process for determining an anomaly judgment score corresponding to a current data sequence based on a first difference between the current data sequence and the multi-head attention feature, a second difference between the multi-head attention score matrix and the dependency matrix, and a third difference between the mapping feature and the historical mapping feature in an embodiment of the present disclosure;

[0102] Figure 9 A schematic diagram illustrating a flow chart of how to determine an abnormality determination score corresponding to a current data sequence based on a first abnormality determination score combined with a second abnormality determination score in an embodiment of the present disclosure;

[0103] Figure 10 A schematic diagram showing the distribution of the number of model calls;

[0104] Figure 11 A distribution diagram showing a plurality of abnormality determination scores obtained;

[0105] Figure 12 A schematic diagram showing the distribution of model rule hit volume versus model rule request volume;

[0106] Figure 13 A distribution diagram showing a plurality of abnormality determination scores obtained;

[0107] Figure 14 A schematic diagram illustrating the overall process of determining an abnormality determination score in an embodiment of the present disclosure;

[0108] Figure 15 A schematic structural diagram of a data anomaly determination device in an exemplary embodiment of the present disclosure is shown;

[0109] Figure 16 A schematic structural diagram of an electronic device in an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0110] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the present disclosure will be more comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or that other methods, components, devices, steps, etc. may be employed. In other cases, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0111] The terms "a", "an", "the" and "said" are used in this specification to indicate the presence of one or more elements / components / etc.; the terms "including" and "having" are used to express open-ended inclusion and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first" and "second" etc. are used only as labels and are not intended to limit the quantity of their objects.

[0112] In addition, the accompanying drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. Identical reference numerals in the drawings represent identical or similar parts, and thus repeated descriptions thereof will be omitted. Some of the blocks shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically separate entities.

[0113] Related technologies generally use statistical discrimination techniques or machine learning algorithms, such as the RRCF (Robust Random Cut Forest) algorithm, to determine anomalies. However, the above solutions have the following drawbacks:

[0114] First, statistical discrimination technology requires domain experts to summarize and design rules and empirical thresholds, which results in a large amount of human resources consumed in detecting anomalies and has a lag effect.

[0115] Second, the RRCF algorithm detects anomalies by constructing a random forest. Each tree is constructed independently and is based on a random partition of the current data point set. This does not fully consider the temporal continuity between data, making it difficult to capture the dependencies between temporal data and unable to combine contextual information to determine data anomalies.

[0116] Third, the RRCF algorithm constructs a tree by randomly selecting features and partitioning points. This method cannot fully capture the complex structure and relationships of data in multidimensional space when processing high-dimensional data. Therefore, when abnormal patterns involve interactive relationships between such multidimensional data, the RRCF algorithm may not be able to accurately identify these anomalies, that is, it cannot judge the abnormal patterns of the data based on the correlation between multidimensional data.

[0117] In view of this, the present disclosure aims to provide a data anomaly determination method to achieve at least the following objectives: by enhancing the location information in the original data stream, accurately capturing the seasonal and periodic pattern anomalies in the data, and further improving the accuracy of the data anomaly determination method by capturing the potential correlation between multi-dimensional indicators (such as hit volume and request volume), thereby reducing the system risks caused by abnormal online model rule indicators and ensuring the stable operation of the system.

[0118] In the embodiments of the present disclosure, a method for determining data anomaly is first provided, which at least to some extent overcomes the defect of insufficient determination accuracy in the related art.

[0119] Figure 1 A flow chart of a method for determining data anomaly in an embodiment of the present disclosure is shown. The execution subject of the method for determining data anomaly may be a server that performs anomaly determination.

[0120] refer to Figure 1 According to an embodiment of the present disclosure, a method for determining data anomaly includes the following steps:

[0121] Step S110: Perform feature mapping on the current data sequence to obtain mapping features, and perform multi-head attention calculation on the mapping features to obtain an attention score matrix and multi-head attention features; the mapping features contain multiple sub-features, and the attention score matrix is used to represent the deep correlation relationship between the multiple sub-features;

[0122] Step S120, obtaining a dependency matrix between the mapping features and the historical mapping features corresponding to the historical data sequence;

[0123] Step S130, determining an abnormality judgment score corresponding to the current data sequence based on a first difference between the current data sequence and the multi-head attention feature, a second difference between the multi-head attention score matrix and the dependency matrix, and a third difference between the mapping feature and the historical mapping feature;

[0124] Step S140: Perform an abnormality determination on the current data sequence based on the abnormality determination score.

[0125] exist Figure 1 In the technical solution provided by the illustrated embodiment, on the one hand, by performing feature mapping on the current data sequence to obtain mapping features, and further using multi-head attention calculation to obtain attention score matrix and multi-head attention features, this process can not only capture the location information in the data, but also effectively identify the deep correlation between different sub-features. This method is particularly suitable for data sets with complex patterns (such as seasonality and periodicity) because it can improve the accuracy of anomaly detection by learning these patterns. Furthermore, by obtaining the dependency matrix between the current mapping features and the historical mapping features corresponding to the historical data sequence, the model's ability to understand time series data can be further enhanced, so that the system can not only consider the immediate data characteristics, but also make more accurate predictions and anomaly judgments based on historical data, thereby improving the stability and reliability of the model. Furthermore, by comprehensively considering the differences between the current data sequence and the multi-head attention features, the differences between the multi-head attention score matrix and the dependency matrix, and the differences between the mapping features and the historical mapping features, the state of the current data sequence can be comprehensively evaluated. This multi-level and multi-dimensional comparison method makes anomaly judgment more accurate and reduces the possibility of false alarms. On the other hand, anomaly judgment is performed on the current data sequence by calculating the anomaly judgment score based on the various differences mentioned above. This method provides more detailed and accurate anomaly detection results by combining multiple factors (real-time data characteristics, historical data dependencies, correlations between sub-features, etc.), which helps to discover potential problems in a timely manner, reduce system risks caused by abnormal online model rule indicators, and ensure the stable operation of the system.

[0126] The following Figure 1 The specific implementation process of each step is described in detail:

[0127] Prior to step S110, it should be noted that within the data platform architecture, a risk control model can be deployed to monitor and analyze platform access data in real time. This access data is multi-dimensional, with a number of dimensions m (where m is an integer greater than 1). Specifically, these dimensions may include, but are not limited to, key indicators such as the number of requests, model call frequency, and the number of model rule matches. By comprehensively analyzing this multi-dimensional data, the security and stability of the platform can be effectively improved, thereby enabling accurate identification and rapid response to potential risks.

[0128] When monitoring platform access data, a preset time unit (e.g., every second, which can be set based on actual circumstances and is not specifically limited by this disclosure) can be used as the data collection period. Subsequently, the platform access data accumulated within each preset time period (including multiple preset time units, such as every minute, every 5 minutes, which can be set based on actual circumstances and is not specifically limited by this disclosure) is organized into an independent data sequence and input into a trained anomaly determination model. Based on this anomaly determination model, the data sequence is processed in the following steps S110-S140 to determine whether the data sequence is abnormal.

[0129] The anomaly detection model can be trained based on a training sample set consisting of multiple historical data sequences. Specifically, each training sample in the training sample set is input into the anomaly detection model to be trained to calculate a first anomaly detection score corresponding to each training sample. This first anomaly detection score serves as the model's loss function, quantifying the difference between the model's prediction and the actual label. Each time new training data is received, the model immediately updates its parameters. Specifically, the first anomaly detection score corresponding to each training sample can be used as the loss function, and the model parameters are adjusted using a backpropagation algorithm and the Adam optimizer. The Adam optimizer combines the advantages of the gradient descent algorithm, efficiently calculating the gradient of each model parameter based on the loss function and updating the parameter value according to a preset learning rate, thereby achieving dynamic optimization of the model parameters. In this way, the model can adaptively adjust its parameters each time new data is received, ensuring that it can more accurately capture normal patterns in the data and detect potential anomalies. This approach not only improves the robustness and generalization ability of the model, but also ensures that it maintains efficient anomaly detection performance in a dynamically changing data environment.

[0130] In step S110, feature mapping is performed on the current data sequence to obtain mapping features, and multi-head attention calculation is performed on the mapping features to obtain an attention score matrix and multi-head attention features.

[0131] In this step, the current data sequence x corresponding to the current time period can be t (size: m*D, D is the total number of the above-mentioned preset time units) to perform feature mapping to obtain mapping features. For example, the encoder f e (·) Map the above current data sequence to a high-dimensional space to obtain the mapping feature e t (Size: m*D). The above mapping feature contains m sub-features, and the dimension of each sub-feature is 1*D.

[0132] Afterwards, the above mapping feature e can be tPerform multi-head attention calculations to obtain the attention score matrix and multi-head attention features. The attention score matrix can be used to represent the deep correlation between the above multiple sub-features.

[0133] refer to Figure 2 , Figure 2 A flowchart illustrating how to perform multi-head attention calculation on the mapping features to obtain the attention score matrix and multi-head attention features in an embodiment of the present disclosure includes steps S201 to S203:

[0134] In step S201 , position encoding is performed on the mapping feature to obtain a position encoding feature.

[0135] In this step, the mapping features can be position-encoded to obtain position-encoded features. Position encoding is a method that embeds the position information of the input data into the feature representation, mainly used to enhance the model's ability to perceive the relative or absolute positions between elements in the sequence data.

[0136] Specifically, refer to Figure 3 , Figure 3 A flow chart showing how to position encode a mapping feature to obtain a position-encoded feature in an embodiment of the present disclosure includes steps S301 to S304:

[0137] In step S301 , a feature group is extracted based on a sliding window.

[0138] In this step, a sliding window technique can be used to extract feature groups. Specifically, the width of the sliding window can be set to w (w is an integer greater than 1) to cover continuous data sequence segments. Each feature group includes w-1 mapping features corresponding to the historical data sequence from the previous w-1 time units within the sliding window, and the mapping feature e corresponding to the current time unit. t For example, taking w as 5 as an example, the feature group extracted from the sliding window can be [e t-4 , e t-3 , e t-2 , e t-1 , e t ].

[0139] This sliding window-based approach effectively captures trends and patterns in data over time, providing strong support for subsequent anomaly detection, prediction, and other tasks. This approach not only takes into account the dynamic nature of data but also enhances the accuracy and robustness of the model by leveraging historical information.

[0140] It should be noted that each mapping feature in the sliding window corresponds to a unique position index. The position index refers to the specific position number of each mapping feature in the sliding window. The position index type can be further subdivided into odd type and even type.

[0141] For example, the position index group corresponding to the feature group extracted in the sliding window can be expressed as [i-4, i-3, i-2, i-1, i], that is, e t The corresponding position index can be i, so the position index corresponding to the history mapping feature can be expressed as j, j∈[i-w+1,i-1].

[0142] In step S302, the mapping feature is absolutely position-encoded according to the position index type corresponding to the mapping feature to obtain a first encoding feature.

[0143] In this step, the mapping feature e t The corresponding position index type encodes the absolute position of the mapping feature to obtain the first encoding feature p a (i), specifically, the first coding feature p a (i) can contain p a (i,2k) or p a (i,2k+1).

[0144] When the position index type is an even type (i.e., i=2k), the following formula 1 can be referred to to perform absolute position encoding on the mapping feature based on the sine function to obtain the first encoding feature p a (i, 2k); When the position index type is an odd type (i.e., i = 2k + 1), refer to the following formula 2 to perform absolute position encoding on the mapping feature based on the cosine function to obtain the first encoding feature p a (i,2k+1):

[0145]

[0146] In step S303, a second coding feature corresponding to the mapping feature is determined according to the position index of the mapping feature, the position index of each historical mapping feature, the timestamp associated with the mapping feature, and the timestamp associated with each historical mapping feature.

[0147] In this step, the second coding feature corresponding to the mapping feature can be determined based on the position index of the mapping feature, the position index of each historical mapping feature, the timestamp associated with the mapping feature, and the timestamp associated with each historical mapping feature. Figure 4 , Figure 4A flow chart of determining a second coding feature corresponding to a mapping feature according to a position index of the mapping feature, a position index of each historical mapping feature, a timestamp associated with the mapping feature, and a timestamp associated with each historical mapping feature in an embodiment of the present disclosure is shown, including steps S401 to S405:

[0148] In step S401 , a first difference between the position index of the mapping feature and the position index of each historical mapping feature is obtained.

[0149] In this step, a first difference value, ie, ij, between the position index i of the mapping feature and the position index j of each historical mapping feature may be obtained.

[0150] In step S402 , a relative position code is determined from the preset reference value and the first difference value according to a numerical relationship between the preset reference value and the first difference value.

[0151] In this step, for example, the preset reference value may be 0, and thus, the relative position code p may be determined according to the larger value of the preset reference value 0 and the first difference (ij) according to the following formula 3: r (i,j):

[0152] p r (i,j)=max(0,ij), j∈[i-w+1,i-1] Formula 3

[0153] In step S403, a second difference between the timestamp associated with the mapping feature and the timestamp associated with each historical mapping feature is obtained.

[0154] In this step, the timestamp t of the current data sequence associated with the mapping feature can be obtained i The timestamp t associated with the historical data series of each historically mapped feature j The second difference between i -t j .

[0155] In step S404, each intermediate position code is determined according to the relative position code and the second difference value.

[0156] In this step, the relative position encoding p r (i, j) combined with the second difference (t i -t j ), determine each intermediate position code. For example, the second difference value can be numerically attenuated according to the attenuation rate λ1, and then the attenuation value obtained is negated and the natural logarithm of the negation is calculated to obtain exp(-λ1*(t i -t j )), and further, according to p rThe product of (i, j) and the natural logarithm determines the code p of each intermediate position r (i,j)*exp(-λ1*(t i -t j )). Wherein, λ1 can be a real number greater than or equal to 0.01 and less than or equal to 0.1. Specifically, it can be set according to actual conditions, and this disclosure does not make any special limitation on this.

[0157] In step S405 , a second coding feature corresponding to the mapping feature is determined based on the accumulated values of the w-1 intermediate position codes.

[0158] In this step, the second coding feature corresponding to the mapping feature can be determined based on the accumulated value of the w-1 intermediate position codes. Specifically, the second coding feature p8(i) can be determined based on the following formula 4:

[0159]

[0160] Next reference Figure 3 In step S304, the position coding feature is determined based on the first coding feature combined with the second coding feature.

[0161] In this step, the first coding feature p a (i) Combined with the second coding feature p8(i), determine the position coding feature Specifically, the above position coding feature can be determined based on the following formula 5

[0162]

[0163] in, Represents the above position encoding feature, p a (i) represents the first coding feature mentioned above, represents the above p8(i), i.e. the second coding feature.

[0164] Next reference Figure 2 ,In step S202, multi-head attention calculation is performed on the combined features of the mapping features and the position encoding features to obtain the attention score matrix and multi-head attention features.

[0165] In this step, multi-head attention is performed on the combined features of the mapping features and the position encoding features to obtain an attention score matrix and multi-head attention features. Multi-head attention is a key component in the Transformer model, enabling the model to simultaneously focus on different parts of the input sequence and capture rich dependencies within the sequence. This mechanism significantly enhances the model's ability to handle long-range dependencies.

[0166] For example, the mapping feature e t The corresponding position encoding feature Merge. This can be achieved by direct addition or concatenation, with the goal of adding the position information of the mapping feature at each time point in the sequence to form a combined feature.

[0167] Next, you can refer to Figure 5 , Figure 5 A flowchart illustrating how to perform multi-head attention calculation on the combined features of the mapping features and the position coding features to obtain the attention score matrix and the multi-head attention features in an embodiment of the present disclosure includes steps S501 to S504:

[0168] In step S501 , a plurality of segment features corresponding to a plurality of sequence lengths are extracted from the combined features.

[0169] In this step, the above combined features e t +p e i Multiple segment features corresponding to multiple sequence lengths are intercepted from the combined feature. For example, according to multiple predefined sequence lengths (e.g., 3, 5, 7, etc.), multiple segment features of corresponding lengths are intercepted from the combined feature. The total number of multiple segment features can be d CDEFG Each fragment represents a subsequence for subsequent self-attention processing, which helps the model pay attention to information at different scales.

[0170] In step S502, self-attention processing is performed on each segment feature to obtain a first attention score and a first self-attention result corresponding to the sequence length.

[0171] In this step, self-attention processing can be performed on each segment feature to obtain the first attention score and the first self-attention result corresponding to the sequence length. Figure 6 , Figure 6 A flowchart illustrating how to perform self-attention processing on each segment feature in an embodiment of the present disclosure to obtain a first attention score and a first self-attention result corresponding to the sequence length includes steps S601 to S605:

[0172] In step S601, the segment feature is multiplied by the query parameter to obtain a first query feature.

[0173] In this step, the fragment features can be combined with the query parameter W I A multiplication process is performed to perform a linear transformation on the segment features to obtain the first query feature q.

[0174] In step S602, the segment feature is multiplied by the key parameter to obtain a first key feature.

[0175] In this step, the fragment features can be combined with the key parameter W J A multiplication process is performed to perform a linear transformation on the segment features to obtain the first key feature k.

[0176] In step S603, the segment feature is multiplied by the value parameter to obtain a first value feature.

[0177] In this step, the fragment features and the value parameter W can be K A multiplication process is performed to perform a linear transformation on the segment features to obtain the first value feature v.

[0178] In step S604, association processing is performed based on the first query feature and the first key feature to obtain a first attention score.

[0179] In this step, the transposition k of the first query feature q and the first key feature k can be M Perform dot product processing to obtain the first dot product result, and calculate the total number of fragment features d CDEFG Perform a square root process to obtain a first square root result, and then perform maximum likelihood processing on the ratio between the first dot product result and the first square root result to obtain a first attention score. For example, the first attention score as can be determined based on the following formula 6:

[0180]

[0181] Among them, as represents the above-mentioned first attention score, Softmax represents the above-mentioned maximum likelihood processing, and Softmax is a function mainly used to convert a real number vector into a probability distribution vector.

[0182] In step S605, the first attention score is multiplied by the first value feature to obtain a first self-attention result corresponding to the sequence length.

[0183] In this step, the first attention score as can be multiplied by the first value feature v to obtain the first self-attention result corresponding to the sequence length. For example, the first self-attention result attention1 corresponding to the sequence length can be calculated based on the following formula 7:

[0184] attention1=as*V Formula 7

[0185] Among them, attention1 represents the above-mentioned first attention score, V represents the above-mentioned first value feature, and attention1 represents the first self-attention result obtained by multiplying the first attention score and the first value feature.

[0186] Next reference Figure 5 In step S503, an attention score matrix is determined based on a plurality of first attention scores corresponding one-to-one to a plurality of sequence lengths.

[0187] In this step, multiple first attention scores as corresponding to multiple sequence lengths can be jointly spliced to obtain an attention score matrix AS.

[0188] In step S504, a multi-head attention feature is determined based on multiple first self-attention results corresponding one-to-one to multiple sequence lengths.

[0189] In this step, multiple first self-attention results corresponding to multiple sequence lengths can be spliced to obtain a first splicing feature. After that, the first splicing feature can be subjected to a first full connection process, for example, multiplying it by a linear transformation weight W ] , get the multi-head attention feature

[0190] Next reference Figure 1 , in step S120, a dependency matrix between the mapping features and the historical mapping features corresponding to the historical data sequence is obtained.

[0191] In this step, the dependency matrix between the mapping features and the historical mapping features corresponding to the historical data series can be obtained. Figure 7 , Figure 7 A flowchart illustrating how to obtain a dependency matrix between a mapping feature and a historical mapping feature corresponding to a historical data sequence in an embodiment of the present disclosure includes steps S701 to S703:

[0192] In step S701 , each reference feature is selected from a feature group consisting of w−1 historical mapping features and mapping features.

[0193] In this step, each baseline feature can be selected from a feature group consisting of w-1 historical mapping features and mapping features. The feature group contains a total of w features, and each feature in the feature group can be selected as a baseline feature in turn.

[0194] In step S702 , the distance index between the reference feature and other features is calculated using a Gaussian kernel function.

[0195] In this step, the Gaussian kernel function can be used to calculate the distance index between the benchmark feature and other features. The Gaussian kernel function is used to measure the similarity between data points. The smaller the value, the higher the similarity between the two features, and vice versa.

[0196] Among them, the other features can be each feature of the total w features including the reference feature. For example, when the reference feature e now Select e t When , the other features e other for e t-4 , e t-3 , e t-2 , e t-1 and e t ; When the datum feature e now Select e t-1 When other features e other Also for e t-4 , e t-3 , e t-2 , e t-1 and e t .

[0197] Specifically, the distance indicator inst between the reference feature and each other feature can be calculated based on the following formula 8:

[0198]

[0199] Here, σ represents a learnable scale parameter, and σ is a real number greater than or equal to 0.2 and less than or equal to 0.5.

[0200] In step S703, a dependency matrix is determined based on w*w distance indices.

[0201] In this step, the dependency matrix INST can be determined based on the distribution of w*w distance indicators. Specifically, the dependency matrix INST can be determined based on the following formula 9:

[0202]

[0203] Among them, Rescale represents standardization or normalization processing, and INST represents the distribution formed by w*w distance indicators inst.

[0204] Next reference Figure 1 In step S130, the abnormality judgment score corresponding to the current data sequence is determined based on the first difference between the current data sequence and the multi-head attention feature, the second difference between the multi-head attention score matrix and the dependency matrix, and the third difference between the mapping feature and the historical mapping feature.

[0205] In this step, the anomaly judgment score corresponding to the current data sequence can be determined based on the first difference between the current data sequence and the multi-head attention feature, the second difference between the multi-head attention score matrix and the dependency matrix, and the third difference between the mapping feature and the historical mapping feature.

[0206] Specifically, you can refer to Figure 8 , Figure 8 A flow chart illustrating how, in an embodiment of the present disclosure, an abnormality judgment score corresponding to a current data sequence is determined based on a first difference between the current data sequence and the multi-head attention feature, a second difference between the multi-head attention score matrix and the dependency matrix, and a third difference between the mapping feature and the historical mapping feature, including steps S801 to S803:

[0207] In step S801, a first abnormality judgment score is determined based on a first difference between the current data sequence and the multi-head attention feature and a second difference between the multi-head attention score matrix and the dependency matrix.

[0208] In this step, the first difference between the current data sequence and the multi-head attention feature can be calculated. Specifically, the second norm distance between the current data sequence and the multi-head attention feature can be obtained. The second norm distance is determined as the first difference.

[0209] Afterwards, the second difference between the multi-head attention score matrix and the dependency matrix can be calculated. Specifically, the first divergence KL(AS||INST) between the multi-head attention score matrix and the dependency matrix can be obtained, and the second divergence KL(INST||AS) between the dependency matrix and the multi-head attention score matrix can be obtained. Then, the first divergence and the second divergence are accumulated to obtain a third divergence KL(AS||INST)+KL(INST||AS). Afterwards, the second difference ||KL(AS||INST)+KL(INST||AS)||1 can be determined based on the first norm distance of the third divergence.

[0210] Next, the first abnormality judgment score can be determined based on the above-mentioned first difference and second difference. Specifically, the first product of the second difference and the preset weight coefficient λ2 (the value range of λ2 can be 0.2 to 0.3, which can be set according to actual conditions and is not specifically limited in this disclosure) can be obtained, and then the third difference between the first difference and the first product can be calculated. After that, the inverse of the third difference can be obtained and the inverse is normalized to obtain the first abnormality judgment score L T (x t ).

[0211] For example, the first abnormality determination score L can be determined based on the following formula 9: T (x t ):

[0212]

[0213] Among them, L T (x t ) represents the first abnormality judgment score, λ2*||KL(AS||INST)+KL(INST||AS)||1 represents the first product, represents the third difference mentioned above.

[0214] In step S802 , a second abnormality determination score is determined according to a third difference between the mapping feature and the historical mapping feature.

[0215] In this step, K (K is an integer greater than 1) specified historical mapping features adjacent to the mapping feature can be extracted from the feature memory unit. For example, the specified historical mapping feature can be expressed as

[0216] Then, the mapping feature e can be calculated based on the following formula 10: t With each specified history mapping feature The norm distance between

[0217]

[0218] Next, the obtained K norm distances can be weighted based on the following formula 11 to obtain the second abnormality judgment score L M (e t ):

[0219]

[0220] Among them, L M (e t ) represents the above-mentioned second abnormality judgment score, β represents a preset weight parameter, β is a real number greater than 0 and less than or equal to 0.5, and can be set according to actual conditions. This disclosure does not make any special limitations on this.

[0221] It should be noted that after determining the second abnormality judgment score, the present disclosure can further compare the first abnormality judgment score with the second abnormality judgment score. Given that the second abnormality judgment score reflects the global abnormality degree of the data sequence, while the first abnormality judgment score reflects the local abnormality degree, therefore, when the second abnormality judgment score is smaller than the first abnormality judgment score, the mapping feature e at the current time point can be used as the first abnormality judgment score. tUpdate to the feature memory unit. In this way, local anomalies (that is, anomalies that only occur in a specific time period, such anomalies are usually not universal and do not represent the general behavior pattern of the entire data set. For example, data fluctuations caused by system maintenance within an hour, such fluctuations will not persist and will not reflect the long-term operating status of the system) are avoided from being mistaken for part of the global pattern. If local anomalies are mistakenly included in the feature memory unit, the model may regard them as part of normal behavior. This means that when similar situations occur in the future, the model may not identify them as anomalies, resulting in missed reports. Therefore, by preventing these abnormal records from entering the feature memory unit, the system can more accurately capture normal data patterns, effectively distinguish local anomalies from global anomalies, and improve the accuracy and reliability of overall anomaly detection.

[0222] In step S803, the abnormality determination score corresponding to the current data sequence is determined according to the first abnormality determination score and the second abnormality determination score.

[0223] In this step, the abnormality determination score corresponding to the current data sequence can be determined based on the first abnormality determination score combined with the second abnormality determination score. Figure 9 , Figure 9 A flow chart illustrating how to determine the abnormality determination score corresponding to the current data sequence based on the first abnormality determination score combined with the second abnormality determination score in an embodiment of the present disclosure includes steps S901 to S903:

[0224] In step S901 , the first abnormality determination score is normalized to obtain a first normalized value.

[0225] In this step, the first mean μ of multiple historical first abnormality judgment scores can be obtained. T and the first standard deviation σ T , based on the first abnormality judgment score L M (x t ), the first mean μ T and the first standard deviation, determine the first standardized value.

[0226] Exemplarily, the first normalized value may be determined based on the following formula 12:

[0227]

[0228] Where b1 represents the first normalized value, L M (x t ) represents the first abnormality judgment score, μ T represents the first mean of the above multiple historical first anomaly judgment scores, σ TRepresents the first standard deviation of the aforementioned multiple first anomaly determination scores. The first mean is the average value calculated based on multiple first anomaly determination scores, representing the central tendency or typical level of these anomaly scores. The first standard deviation is a statistic used to measure the dispersion or fluctuation of multiple first anomaly determination scores.

[0229] In step S902 , the second abnormality determination score is normalized to obtain a second normalized value.

[0230] In this step, the second mean and the second standard deviation of multiple historical second abnormality determination scores may be obtained, and then the second normalized value may be determined based on the second abnormality determination score, the second mean and the second standard deviation.

[0231] Exemplarily, the second normalized value b2 may be determined based on the following formula 13:

[0232]

[0233] Wherein, b2 represents the second normalized value, L M (e t ) represents the second abnormality judgment score, μ M represents the second mean of the above multiple historical second anomaly judgment scores, σ M Represents the second standard deviation of the aforementioned plurality of historical second abnormality determination scores.

[0234] In step S903 , weighted processing is performed on the first normalized value and the second normalized value to obtain an abnormality determination score.

[0235] In this step, the first normalized value b1 and the second normalized value b2 can be weighted to obtain an abnormality determination score. For example, the abnormality determination score Score(x t ):

[0236]

[0237] Among them, Score(x t ) represents the above abnormality judgment score, represents the first abnormality determination score b1, represents the above-mentioned second abnormality determination score b2, γ represents a preset weight parameter, γ is a real number greater than or equal to 0.3 and less than or equal to 0.5, and can be set according to actual conditions. This disclosure does not make any special restrictions on this.

[0238] Next reference Figure 1 In step S140, an abnormality determination is performed on the current data sequence based on the abnormality determination score.

[0239] In this step, an abnormality determination can be performed on the current data sequence based on the above-mentioned abnormality determination score. Exemplarily, an abnormality score threshold can be set in advance, and then the above-mentioned abnormality determination score is compared with the preset abnormality score threshold. If the above-mentioned abnormality determination score is greater than or equal to the above-mentioned abnormality score threshold, it can be determined that there is an abnormality in the current data sequence. If the above-mentioned abnormality determination score is less than the above-mentioned abnormality score threshold, it can be determined that there is no abnormality in the current data sequence.

[0240] Optionally, when it is determined that the above current data sequence is abnormal, an alarm can be issued to prompt relevant personnel to take relevant measures quickly, thereby facilitating the rapid elimination of the abnormal situation.

[0241] The following describes the application scenario of the data anomaly determination method in the embodiment of the present disclosure in conjunction with Case 1. Figure 10 ). Figure 10 The following diagram shows the distribution of the number of model calls. At a specific point in time (circled), a significant drop in call count occurs, which persists for a period of time. This may indicate a persistent anomaly in the metric. By performing feature mapping, multi-head attention calculation, and constructing a dependency matrix for each current data sequence, we can obtain an anomaly score for each sequence. Figure 11 The distribution diagram of the obtained abnormality judgment scores is shown by Figure 11 As can be seen, the anomaly score peaked at time t=1343, indicated by the circle, indicating that the data pattern at that moment deviated significantly from the normal range. At this point, the system triggers an alert, indicating that an anomaly may have occurred due to a business logic configuration error, allowing for prompt investigation and repair to ensure normal business health.

[0242] The following describes the application scenario of the data anomaly determination method in the embodiment of the present disclosure in conjunction with Case 2. Take the input data including multiple model rule hit quantity sequences and model rule request quantity sequences corresponding to multiple preset time periods in one day as an example, Figure 12 , Figure 12 The data distribution of the model rule hit count versus the model rule request count is shown. As can be seen from the circled area in the figure, after t = 306, the request count drops sharply while the hit count shows a clear upward trend. This change violates the normal positive correlation between the two. By inputting this data into the trained anomaly detection model, we can obtain anomaly detection scores corresponding to each preset time period. Figure 13 The distribution diagram of the obtained abnormality judgment scores is shown by Figure 13As can be seen, the anomaly score has been on a continuous upward trend since time t = 306, reflecting an abnormal pattern in the distribution of the original sequence data. At this point, the system will also issue an early warning, providing engineers with key indicator anomaly information to facilitate subsequent anomaly investigation, effectively addressing potential issues and ensuring stable system operation.

[0243] refer to Figure 14 , Figure 14 The overall flow diagram of how to determine the abnormality judgment score in the embodiment of the present disclosure includes steps S1401 to S1408:

[0244] In step S1401, a training sample is obtained;

[0245] In step S1402, an abnormality determination model is obtained by training using training samples;

[0246] In step S1403, feature mapping is performed on the current data sequence to obtain mapping features;

[0247] In step S1404, position coding enhancement is performed on the mapping feature to obtain a position coding feature;

[0248] In step S1405, multi-head attention calculation is performed on the combined features of the mapping features and the position encoding features to obtain an attention score matrix and multi-head attention features;

[0249] In step S1406, a first abnormality judgment score is determined based on a first difference between the current data sequence and the multi-head attention feature and a second difference between the multi-head attention score matrix and the dependency matrix;

[0250] In step S1407, a second abnormality determination score is determined based on a third difference between the mapping feature and the historical mapping feature;

[0251] In step S1408 , the abnormality determination score corresponding to the current data sequence is determined according to the first abnormality determination score and the second abnormality determination score.

[0252] Based on the above technical solutions, the present disclosure has at least the following technical effects:

[0253] First, a Transformer model with an enhanced positional encoding mechanism is proposed to more accurately capture the position information in the sequence. This enhanced positional encoding helps to identify the temporal dependencies and periodic patterns of the data.

[0254] Second, feature memory units are introduced to capture long-term dependencies, enabling the model to identify abnormal patterns in data over a long period of time, thereby improving the ability to detect periodic and persistent anomalies.

[0255] Third, by combining the first anomaly determination score and the second anomaly determination score, the health status of the data can be more comprehensively assessed;

[0256] Fourth, by comprehensively considering abnormal information from multiple dimensions and generating a comprehensive abnormality score, relevant personnel can quickly locate potential problems and take corresponding measures.

[0257] The present disclosure also provides a data anomaly determination device, Figure 15 FIG. 1 is a schematic diagram showing the structure of a data anomaly determination device in an exemplary embodiment of the present disclosure; FIG. Figure 15 As shown, the data anomaly determination device 1500 may include a feature processing module 1510, a dependency acquisition module 1520, an anomaly determination score determination module 1530, and an anomaly determination module 1540.

[0258] Feature processing module 1510 is used to perform feature mapping on the current data sequence to obtain mapping features, and perform multi-head attention calculation on the mapping features to obtain an attention score matrix and multi-head attention features; the mapping features contain multiple sub-features, and the attention score matrix is used to represent the deep correlation relationship between the multiple sub-features;

[0259] A dependency relationship acquisition module 1520 is configured to acquire a dependency relationship matrix between the mapping feature and the historical mapping features corresponding to the historical data sequence;

[0260] Anomaly determination score determination module 1530, configured to determine an anomaly determination score corresponding to the current data sequence based on a first difference between the current data sequence and the multi-head attention feature, a second difference between the multi-head attention score matrix and the dependency matrix, and a third difference between the mapping feature and the historical mapping feature;

[0261] The abnormality determination module 1540 is configured to perform abnormality determination on the current data sequence based on the abnormality determination score.

[0262] In an exemplary embodiment of the present disclosure, the feature processing module 1510 performs multi-head attention calculation on the mapped features to obtain an attention score matrix and multi-head attention features, including:

[0263] Performing position encoding on the mapping feature to obtain a position encoding feature;

[0264] Multi-head attention calculation is performed on the combined features of the mapping features and the position encoding features to obtain the attention score matrix and the multi-head attention features.

[0265] In an exemplary embodiment of the present disclosure, the feature processing module 1510 performs position encoding on the mapping feature to obtain a position encoding feature, including:

[0266] A feature group is extracted based on a sliding window; the width of the sliding window is w, and the feature group includes w-1 historical mapping features corresponding to w-1 historical data sequences and the mapping feature; w is an integer greater than 1;

[0267] Performing absolute position encoding on the mapping feature according to the position index type corresponding to the mapping feature to obtain a first encoding feature;

[0268] determining a second coding feature corresponding to the mapping feature based on a position index of the mapping feature, a position index of each historical mapping feature, a timestamp associated with the mapping feature, and a timestamp associated with each historical mapping feature;

[0269] The position coding feature is determined based on the first coding feature in combination with the second coding feature.

[0270] In an exemplary embodiment of the present disclosure, the feature processing module 1510 performs absolute position encoding on the mapping feature according to the position index type corresponding to the mapping feature to obtain a first encoding feature, including:

[0271] When the position index type is an even type, performing absolute position encoding on the mapping feature based on a sine function to obtain the first encoding feature;

[0272] When the position index type is an odd type, absolute position encoding is performed on the mapping feature based on a cosine function to obtain the first encoding feature.

[0273] In an exemplary embodiment of the present disclosure, the feature processing module 1510 determines the second coding feature corresponding to the mapping feature according to the position index of the mapping feature, the position index of each historical mapping feature, the timestamp associated with the mapping feature, and the timestamp associated with each historical mapping feature, including:

[0274] Obtaining a first difference between the position index of the mapping feature and the position index of each historical mapping feature;

[0275] determining a relative position code from the preset reference value and the first difference value according to a numerical relationship between the preset reference value and the first difference value;

[0276] Obtaining a second difference between the timestamp associated with the mapping feature and the timestamp associated with each historical mapping feature;

[0277] determining each intermediate position code according to the relative position code in combination with the second difference;

[0278] A second coding feature corresponding to the mapping feature is determined according to the accumulated values of the w-1 intermediate position codes.

[0279] In an exemplary embodiment of the present disclosure, the feature processing module 1510 performs multi-head attention calculation on the combined features of the mapping features and the position encoding features to obtain the attention score matrix and the multi-head attention features, including:

[0280] Extracting a plurality of segment features corresponding to a plurality of sequence lengths from the combined features;

[0281] Performing self-attention processing on each segment feature to obtain a first attention score and a first self-attention result corresponding to the sequence length;

[0282] Determining the attention score matrix according to a plurality of first attention scores corresponding one-to-one to a plurality of sequence lengths;

[0283] The multi-head attention feature is determined based on multiple first self-attention results corresponding one-to-one to multiple sequence lengths.

[0284] In an exemplary embodiment of the present disclosure, the feature processing module 1510 performs self-attention processing on the segment features to obtain a first attention score and a first self-attention result corresponding to the sequence length, including:

[0285] Multiplying the segment feature by the query parameter to obtain a first query feature;

[0286] Multiplying the segment feature by the key parameter to obtain a first key feature;

[0287] Multiplying the segment feature by the value parameter to obtain a first value feature;

[0288] performing association processing based on the first query feature and the first key feature to obtain a first attention score;

[0289] The first attention score is multiplied by the first value feature to obtain a first self-attention result corresponding to the sequence length.

[0290] In an exemplary embodiment of the present disclosure, the feature processing module 1510 performs association processing based on the first query feature and the first key feature to obtain a first attention score, including:

[0291] Performing a dot product process on the first query feature and the transpose of the first key feature to obtain a first dot product result, and performing a square root process on the number of the segment features to obtain a first square root result;

[0292] Perform maximum likelihood processing on the ratio of the first dot product result and the first square root result to obtain the first attention score.

[0293] In an exemplary embodiment of the present disclosure, the feature processing module 1510 determines the multi-head attention feature based on a plurality of first self-attention results corresponding one-to-one to a plurality of sequence lengths, including:

[0294] Performing splicing processing on a plurality of first self-attention results corresponding one-to-one to a plurality of sequence lengths to obtain a first splicing feature;

[0295] Perform a first full-connection process on the first splicing feature to obtain the multi-head attention feature.

[0296] In an exemplary embodiment of the present disclosure, the dependency relationship acquisition module 1520 acquires a dependency relationship matrix between the mapping feature and the historical mapping feature corresponding to the historical data sequence, including:

[0297] Select each baseline feature from the feature group consisting of w-1 historical mapping features and the mapping feature;

[0298] Calculating a distance index between the reference feature and other features using a Gaussian kernel function; the other features include the w-1 historical mapping features and the mapping feature;

[0299] The dependency matrix is determined according to w*w distance indicators.

[0300] In an exemplary embodiment of the present disclosure, the abnormality determination score determination module 1530 determines the abnormality determination score corresponding to the current data sequence based on a first difference between the current data sequence and the multi-head attention feature, a second difference between the multi-head attention score matrix and the dependency matrix, and a third difference between the mapping feature and the historical mapping feature, including:

[0301] Determining a first anomaly determination score based on a first difference between the current data sequence and the multi-head attention feature and a second difference between the multi-head attention score matrix and the dependency matrix;

[0302] determining a second abnormality determination score according to a third difference between the mapping feature and the historical mapping feature;

[0303] An abnormality determination score corresponding to the current data sequence is determined according to the first abnormality determination score and the second abnormality determination score.

[0304] In an exemplary embodiment of the present disclosure, the first difference between the current data sequence and the multi-head attention feature is determined by:

[0305] Obtaining a second norm distance between the current data sequence and the multi-head attention feature;

[0306] The second norm distance is determined as the first difference.

[0307] In an exemplary embodiment of the present disclosure, the second difference between the multi-head attention score matrix and the dependency matrix is determined by:

[0308] Obtaining a first divergence between the multi-head attention score matrix and the dependency matrix, and obtaining a second divergence between the dependency matrix and the multi-head attention score matrix;

[0309] Accumulating the first divergence and the second divergence to obtain a third divergence;

[0310] The second difference is determined according to the first norm distance of the third divergence.

[0311] In an exemplary embodiment of the present disclosure, the abnormality determination score determination module 1530 determines a first abnormality determination score based on a first difference between the current data sequence and the multi-head attention feature and a second difference between the multi-head attention score matrix and the dependency matrix, including:

[0312] Obtaining a first product of the second difference and a preset weight coefficient;

[0313] calculating a third difference between the first difference and the first product;

[0314] The inverse of the third difference is obtained, and the third difference is normalized to obtain the first abnormality determination score.

[0315] In an exemplary embodiment of the present disclosure, the abnormality determination score determination module 1530 determines the second abnormality determination score according to the third difference between the mapping feature and the historical mapping feature, including:

[0316] Extracting K designated historical mapping features adjacent to the mapping feature from the feature memory unit; K is an integer greater than 1;

[0317] calculating a norm distance between the mapping feature and each of the specified historical mapping features;

[0318] A weighted process is performed on the K norm distances to obtain the second abnormality determination score.

[0319] In an exemplary embodiment of the present disclosure, the abnormality determination score determining module 1530 determines the abnormality determination score corresponding to the current data sequence according to the first abnormality determination score combined with the second abnormality determination score, including:

[0320] performing normalization processing on the first abnormality determination score to obtain a first normalized value;

[0321] performing normalization processing on the second abnormality determination score to obtain a second normalized value;

[0322] The first normalized value and the second normalized value are weighted to obtain the abnormality determination score.

[0323] In an exemplary embodiment of the present disclosure, the abnormality determination score determining module 1530 performs normalization processing on the first abnormality determination score to obtain a first normalized value, including:

[0324] Obtaining a first mean and a first standard deviation of a plurality of historical first anomaly determination scores;

[0325] A first normalized value is determined based on the first abnormality determination score, the first mean, and the first standard deviation.

[0326] In an exemplary embodiment of the present disclosure, the abnormality determination score determining module 1530 performs normalization processing on the second abnormality determination score to obtain a second normalized value, including:

[0327] Obtain a second mean and a second standard deviation of a plurality of historical second anomaly determination scores;

[0328] A second normalized value is determined based on the second abnormality determination score, the second mean, and the second standard deviation.

[0329] In an exemplary embodiment of the present disclosure, after determining the second abnormality determination score, the abnormality determination score determination module 1530 is configured to:

[0330] comparing the first abnormality determination score and the second abnormality determination score;

[0331] If the second abnormality determination score is smaller than the first abnormality determination score, the mapping feature is updated into the feature memory unit.

[0332] The specific details of each module in the above-mentioned data anomaly determination device have been described in detail in the corresponding data anomaly determination method, and therefore will not be repeated here.

[0333] It should be noted that although several modules or units of the device for action execution are mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0334] Furthermore, although the steps of the method of the present disclosure are described in a particular order in the accompanying drawings, this does not require or imply that the steps must be performed in this particular order, or that all steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0335] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0336] The present disclosure further provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or may exist independently without being assembled into the electronic device.

[0337] Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device.

[0338] Computer-readable storage media can transmit, propagate, or transfer programs for use by or in conjunction with an instruction execution system, apparatus, or device. Program code contained on a computer-readable storage medium can be transmitted using any suitable medium, including but not limited to wireless, wireline, optical cable, RF, or any suitable combination thereof.

[0339] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiments.

[0340] In addition, an electronic device capable of implementing the above method is also provided in an embodiment of the present disclosure.

[0341] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods, or program products. Therefore, various aspects of the present disclosure may be implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits," "modules," or "systems."

[0342] Refer to the following Figure 16 16 is a diagram to describe an electronic device 1600 according to this embodiment of the present disclosure. Figure 16 The electronic device 1600 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0343] like Figure 16 As shown, electronic device 1600 is implemented as a general-purpose computing device. Components of electronic device 1600 may include, but are not limited to, at least one processor 1610, at least one memory 1620, a bus 1630 connecting various system components (including memory 1620 and processor 1610), and a display 1640.

[0344] The memory stores program codes that can be executed by the processor 1610, so that the processor 1610 performs the steps described in the "Exemplary Method" section above according to various exemplary embodiments of the present disclosure. For example, the processor 1610 may perform the following steps: Figure 1 As shown in: Step S110, feature mapping is performed on the current data sequence to obtain mapping features, and multi-head attention calculation is performed on the mapping features to obtain an attention score matrix and a multi-head attention feature; the mapping feature contains multiple sub-features, and the attention score matrix is used to characterize the deep correlation between the multiple sub-features; Step S120, a dependency matrix between the mapping feature and the historical mapping feature corresponding to the historical data sequence is obtained; Step S130, based on the first difference between the current data sequence and the multi-head attention feature, the second difference between the multi-head attention score matrix and the dependency matrix, and the third difference between the mapping feature and the historical mapping feature, the abnormality judgment score corresponding to the current data sequence is determined; Step S140, abnormality judgment is performed on the current data sequence based on the abnormality judgment score.

[0345] The memory 1620 may include a readable medium in the form of a volatile storage, such as a random access memory (RAM) 16201 and / or a cache memory 16202 , and may further include a read-only memory (ROM) 16203 .

[0346] Memory 1620 may also include a program / utility 16204 having a set (at least one) of program modules 16205, such program modules 16205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0347] The bus 1630 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures.

[0348] Electronic device 1600 can also communicate with one or more external devices 1700 (e.g., a keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with electronic device 1600, and / or any device that enables electronic device 1600 to communicate with one or more other computing devices (e.g., a router, modem, etc.). Such communication can occur via input / output (I / O) interface 1650. Furthermore, electronic device 1600 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via network adapter 1660. As shown, network adapter 1660 communicates with other modules of electronic device 1600 via bus 1630. It should be understood that, although not shown, other hardware and / or software modules can be used in conjunction with electronic device 1600, including but not limited to microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0349] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow from the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the claims.

Claims

1. A method for determining data anomaly, characterized in that: include: Perform feature mapping on the current data sequence to obtain mapping features, and perform multi-head attention calculation on the mapping features to obtain an attention score matrix and multi-head attention features; The mapping feature includes multiple sub-features, and the attention score matrix is used to represent the deep correlation relationship between the multiple sub-features; Obtaining a dependency matrix between the mapping feature and historical mapping features corresponding to the historical data sequence; Determining an abnormality judgment score corresponding to the current data sequence based on a first difference between the current data sequence and the multi-head attention feature, a second difference between the multi-head attention score matrix and the dependency matrix, and a third difference between the mapping feature and the historical mapping feature; An abnormality determination is performed on the current data sequence based on the abnormality determination score.

2. The method according to claim 1, characterized in that The performing multi-head attention calculation on the mapping feature to obtain an attention score matrix and a multi-head attention feature includes: Performing position encoding on the mapping feature to obtain a position encoding feature; Multi-head attention calculation is performed on the combined features of the mapping features and the position encoding features to obtain the attention score matrix and the multi-head attention features.

3. The method according to claim 2, characterized in that The performing position encoding on the mapping feature to obtain a position encoding feature includes: A feature group is extracted based on a sliding window; the width of the sliding window is w, and the feature group includes w-1 historical mapping features corresponding to w-1 historical data sequences and the mapping feature; w is an integer greater than 1; Performing absolute position encoding on the mapping feature according to the position index type corresponding to the mapping feature to obtain a first encoding feature; determining a second coding feature corresponding to the mapping feature based on a position index of the mapping feature, a position index of each historical mapping feature, a timestamp associated with the mapping feature, and a timestamp associated with each historical mapping feature; The position coding feature is determined based on the first coding feature in combination with the second coding feature.

4. The method according to claim 3, characterized in that The absolute position encoding of the mapping feature according to the position index type corresponding to the mapping feature to obtain a first encoding feature includes: When the position index type is an even type, performing absolute position encoding on the mapping feature based on a sine function to obtain the first encoding feature; When the position index type is an odd type, absolute position encoding is performed on the mapping feature based on a cosine function to obtain the first encoding feature.

5. The method according to claim 3, characterized in that The determining, according to the position index of the mapping feature, the position index of each historical mapping feature, the timestamp associated with the mapping feature, and the timestamp associated with each historical mapping feature, a second coding feature corresponding to the mapping feature includes: Obtaining a first difference between the position index of the mapping feature and the position index of each historical mapping feature; determining a relative position code from the preset reference value and the first difference value according to a numerical relationship between the preset reference value and the first difference value; Obtaining a second difference between the timestamp associated with the mapping feature and the timestamp associated with each historical mapping feature; determining each intermediate position code according to the relative position code in combination with the second difference; A second coding feature corresponding to the mapping feature is determined according to the accumulated values of the w-1 intermediate position codes.

6. The method according to claim 2, characterized in that The performing multi-head attention calculation on the combined features of the mapping features and the position encoding features to obtain the attention score matrix and the multi-head attention features includes: Extracting a plurality of segment features corresponding to a plurality of sequence lengths from the combined features; Performing self-attention processing on each segment feature to obtain a first attention score and a first self-attention result corresponding to the sequence length; Determining the attention score matrix according to a plurality of first attention scores corresponding one-to-one to a plurality of sequence lengths; The multi-head attention feature is determined based on multiple first self-attention results corresponding one-to-one to multiple sequence lengths.

7. The method according to claim 6, characterized in that The performing self-attention processing on the segment features to obtain a first attention score and a first self-attention result corresponding to the sequence length includes: Multiplying the segment feature by the query parameter to obtain a first query feature; Multiplying the segment feature by the key parameter to obtain a first key feature; Multiplying the segment feature by the value parameter to obtain a first value feature; performing association processing based on the first query feature and the first key feature to obtain a first attention score; The first attention score is multiplied by the first value feature to obtain a first self-attention result corresponding to the sequence length.

8. The method according to claim 7, characterized in that The performing association processing based on the first query feature and the first key feature to obtain a first attention score includes: Performing a dot product process on the first query feature and the transpose of the first key feature to obtain a first dot product result, and performing a square root process on the number of the segment features to obtain a first square root result; Perform maximum likelihood processing on the ratio of the first dot product result and the first square root result to obtain the first attention score.

9. The method according to claim 6, characterized in that The determining the multi-head attention feature according to the plurality of first self-attention results corresponding one-to-one to the plurality of sequence lengths includes: Performing splicing processing on a plurality of first self-attention results corresponding one-to-one to a plurality of sequence lengths to obtain a first splicing feature; Perform a first full-connection process on the first splicing feature to obtain the multi-head attention feature.

10. The method according to claim 1, characterized in that The obtaining of a dependency matrix between the mapping feature and the historical mapping feature corresponding to the historical data sequence includes: Select each baseline feature from the feature group consisting of w-1 historical mapping features and the mapping feature; Calculating a distance index between the reference feature and other features using a Gaussian kernel function; the other features include the w-1 historical mapping features and the mapping feature; The dependency matrix is determined according to w*w distance indicators.

11. The method according to any one of claims 1 to 10, characterized in that The determining, based on a first difference between the current data sequence and the multi-head attention feature, a second difference between the multi-head attention score matrix and the dependency matrix, and a third difference between the mapping feature and the historical mapping feature, of an abnormality judgment score corresponding to the current data sequence includes: Determining a first anomaly determination score based on a first difference between the current data sequence and the multi-head attention feature and a second difference between the multi-head attention score matrix and the dependency matrix; determining a second abnormality determination score according to a third difference between the mapping feature and the historical mapping feature; An abnormality determination score corresponding to the current data sequence is determined according to the first abnormality determination score and the second abnormality determination score.

12. The method according to claim 11, characterized in that The determining, based on the first anomaly determination score and the second anomaly determination score, of an anomaly determination score corresponding to the current data sequence includes: performing normalization processing on the first abnormality determination score to obtain a first normalized value; performing normalization processing on the second abnormality determination score to obtain a second normalized value; The first normalized value and the second normalized value are weighted to obtain the abnormality determination score.

13. A data anomaly determination device, characterized in that: include: A feature processing module is used to perform feature mapping on the current data sequence to obtain mapping features, and perform multi-head attention calculation on the mapping features to obtain an attention score matrix and multi-head attention features; The mapping feature includes multiple sub-features, and the attention score matrix is used to represent the deep correlation relationship between the multiple sub-features; A dependency relationship acquisition module, configured to acquire a dependency relationship matrix between the mapping feature and the historical mapping features corresponding to the historical data sequence; an abnormality determination score determination module, configured to determine an abnormality determination score corresponding to the current data sequence based on a first difference between the current data sequence and the multi-head attention feature, a second difference between the multi-head attention score matrix and the dependency matrix, and a third difference between the mapping feature and the historical mapping feature; An anomaly determination module is configured to perform an anomaly determination on the current data sequence based on the anomaly determination score.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data anomaly determination method according to any one of claims 1 to 12 is implemented.

15. An electronic device, characterized in that: include: processor; as well as a memory for storing executable instructions of the processor; The processor is configured to execute the data anomaly determination method according to any one of claims 1 to 12 by executing the executable instructions.

Images