Data anomaly detection method and device, computer device and storage medium

Abstract

The application relates to a data anomaly detection method and device, computer equipment and a storage medium, relates to the technical field of artificial intelligence, and can be applied to the financial field or other technical fields. The method comprises the following steps: processing target transaction data according to a target acquisition time of the target transaction data, obtaining target multidimensional data containing time characteristics, performing anomaly detection on the target multidimensional data through an anomaly detection model, and obtaining an anomaly detection result of the target transaction data. The method can improve the accuracy of data anomaly detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a data anomaly detection method and device, computer equipment and storage medium, which can be applied to the financial field or other technical fields. BACKGROUND

[0002] With the development of the financial field, the amount of transaction data related to the financial field also increases sharply. In order to detect transaction data, a data anomaly detection method appears. The labeled historical transaction data can be used as sample data to train a data anomaly detection model, and then the data anomaly detection model is used to detect transaction data.

[0003] However, since transaction data fluctuates according to external factors, using the current data anomaly detection method, only relying on transaction data itself to detect transaction data will reduce the accuracy of data anomaly detection, which needs to be improved. SUMMARY

[0004] Therefore, it is necessary to provide a data anomaly detection method and device, computer equipment and storage medium capable of improving the accuracy of data anomaly detection.

[0005] In a first aspect, the present application provides a data anomaly detection method, which comprises:

[0006] processing the target transaction data according to the target acquisition time of the target transaction data to be detected to obtain target multi-dimensional data containing time characteristics;

[0007] detecting the target multi-dimensional data by using an anomaly detection model to obtain an anomaly detection result of the target transaction data.

[0008] In one embodiment, the target transaction data is processed according to the target acquisition time of the target transaction data to be detected to obtain target multi-dimensional data containing time characteristics, which comprises:

[0009] splicing the target acquisition time of the target transaction data to be detected and the target transaction data to obtain target multi-dimensional data containing time characteristics.

[0010] In one embodiment, the anomaly detection model is constructed in the following manner:

[0011] The sample multidimensional data in the sample multidimensional data set is clustered based on the at least two initial clustering centers to obtain data clusters corresponding to the initial clustering centers; reference clustering centers corresponding to the data clusters are determined according to the mean values of the sample multidimensional data contained in the data clusters; if the distance between the initial clustering center and the reference clustering center of any data cluster is greater than a first preset distance threshold, the reference clustering centers are taken as new initial clustering centers, and the operation of clustering the sample multidimensional data in the sample multidimensional data set based on the at least two initial clustering centers to obtain data clusters corresponding to the initial clustering centers is performed again until the distance between the initial clustering center and the reference clustering center of any data cluster is less than the first preset distance threshold, and the reference clustering centers are taken as model clustering centers; and an anomaly detection model is constructed according to the model clustering centers.

[0012] In one of the embodiments, the target multidimensional data is subjected to anomaly detection by the anomaly detection model to obtain an anomaly detection result of the target transaction data, including:

[0013] A model clustering center closest to the target multidimensional data is selected from the model clustering centers of the anomaly detection model as a target clustering center; if the distance between the target clustering center and the target multidimensional data is greater than a second preset distance threshold, the target multidimensional data is subjected to anomaly verification to obtain the anomaly detection result of the target transaction data.

[0014] In one of the embodiments, the target multidimensional data is subjected to anomaly verification to obtain the anomaly detection result of the target transaction data, including:

[0015] The dimension distance values of the target clustering center and the target multidimensional data in each data dimension are determined; each data dimension is sorted according to the dimension distance values in descending order; a preset number of data dimensions sorted in the front are taken as abnormal data dimensions according to the sorting result; the target multidimensional data is subjected to anomaly verification according to the dimension distance values corresponding to the abnormal data dimensions to obtain the anomaly detection result of the target transaction data.

[0016] In one of the embodiments, the target multidimensional data is subjected to anomaly verification according to the dimension distance values corresponding to the abnormal data dimensions to obtain the anomaly detection result of the target transaction data, including:

[0017] The dimension distance values corresponding to the abnormal data dimensions are weighted according to the dimension weights of the abnormal data dimensions to obtain abnormal values of the abnormal data dimensions; the abnormal value of the target multidimensional data is determined according to the abnormal values of the abnormal data dimensions; if the abnormal value of the target multidimensional data is greater than a preset abnormal threshold, the anomaly detection result of the target transaction data is determined as data anomaly.

[0018] In a second aspect, the present application also provides a data anomaly detection device, which comprises:

[0019] a data processing module, configured to process the target transaction data according to a target acquisition time of the target transaction data to be detected, and obtain target multi-dimensional data containing time characteristics;

[0020] an anomaly detection module, configured to perform anomaly detection on the target multi-dimensional data by using an anomaly detection model, and obtain an anomaly detection result of the target transaction data.

[0021] In a third aspect, the present application also provides a computer device, which comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0022] processing the target transaction data according to a target acquisition time of the target transaction data to be detected, and obtaining target multi-dimensional data containing time characteristics;

[0023] performing anomaly detection on the target multi-dimensional data by using an anomaly detection model, and obtaining an anomaly detection result of the target transaction data.

[0024] In a fourth aspect, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program implements the following steps when executed by a processor:

[0025] processing the target transaction data according to a target acquisition time of the target transaction data to be detected, and obtaining target multi-dimensional data containing time characteristics;

[0026] performing anomaly detection on the target multi-dimensional data by using an anomaly detection model, and obtaining an anomaly detection result of the target transaction data.

[0027] In a fifth aspect, the present application also provides a computer program product, which comprises a computer program, and the computer program implements the following steps when executed by a processor:

[0028] processing the target transaction data according to a target acquisition time of the target transaction data to be detected, and obtaining target multi-dimensional data containing time characteristics;

[0029] performing anomaly detection on the target multi-dimensional data by using an anomaly detection model, and obtaining an anomaly detection result of the target transaction data.

[0030] The aforementioned data anomaly detection method, apparatus, computer equipment, and storage medium incorporate target acquisition time. By processing the target transaction data using this acquisition time, multidimensional target data containing time characteristics is obtained. Subsequently, an anomaly detection model is used to detect anomalies in the multidimensional target data, yielding the anomaly detection results. Compared to related technologies, this method can mine target transaction data from both time and numerical dimensions, avoiding the impact of time fluctuations on anomaly detection results and improving the accuracy of data anomaly detection. Attached Figure Description

[0031] Figure 1 This is a diagram illustrating the application environment of a data anomaly detection method in one embodiment.

[0032] Figure 2 This is a flowchart illustrating a data anomaly detection method in one embodiment;

[0033] Figure 3 This is a flowchart illustrating the process of building an anomaly detection model in one embodiment;

[0034] Figure 4 This is a flowchart illustrating the process of determining anomaly detection results in one embodiment;

[0035] Figure 5 This is a flowchart illustrating the anomaly verification process in one embodiment;

[0036] Figure 6 This is a flowchart illustrating a data anomaly detection method in another embodiment;

[0037] Figure 7 This is a structural block diagram of a data anomaly detection device in one embodiment;

[0038] Figure 8 This is a structural block diagram of a data anomaly detection device in another embodiment;

[0039] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0041] The data anomaly detection method provided in this application embodiment can be applied to, for example... Figure 1The application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data required by the server 104 to process. For example, target transaction data. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers. For example, the server 104 processes the target transaction data according to the target acquisition time of the target transaction data to be detected, obtains target multi-dimensional data containing time characteristics, detects the target multi-dimensional data through an anomaly detection model, and obtains an anomaly detection result of the target transaction data; Further, the server 104 sends the anomaly detection result to the terminal 102 for display to the operation and maintenance party. Among them, the terminal 102 can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and Internet of Things devices, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0042] With the development of the financial field, the data volume of transaction data related to the financial field also rises sharply. In order to detect transaction data, a data anomaly detection method appears. The labeled historical transaction data can be used as sample data to train a data anomaly detection model, and then the transaction data is detected through the data anomaly detection model.

[0043] However, due to the fluctuation of transaction data according to external factors, using the current data anomaly detection method, only relying on the transaction data itself to detect the transaction data will reduce the accuracy of data anomaly detection.

[0044] Based on this, in one embodiment, as Figure 2 shown, a data anomaly detection method is provided. Taking the server in Figure 1 as an example, the method includes the following steps:

[0045] S201, according to the target acquisition time of the target transaction data to be detected, processing the target transaction data to obtain target multi-dimensional data containing time characteristics.

[0046] Among them, the target transaction data refers to the data generated when the target business transacts, which can include but is not limited to transaction volume and transaction success rate and other types of data; the target business refers to any financial business in the institution; the target acquisition time refers to the specific time of obtaining the target transaction data, which can include but is not limited to year, month, week and time, etc.; the target multi-dimensional data refers to data containing multiple data dimensions.

[0047] Optionally, in order to analyze the target transaction data from multiple dimensions, the target transaction data to be detected and the target acquisition time of the target transaction data can be directly input into the data fusion model, and the data fusion model outputs the target multi-dimensional data containing time characteristics according to the target transaction data to be detected, the target acquisition time of the target transaction data, and the model parameters.

[0048] Alternatively, the target acquisition time of the target transaction data to be detected and the target transaction data can be spliced to obtain the target multi-dimensional data containing time characteristics.

[0049] Optionally, multiple dimensions of time data can be extracted from the target acquisition time, and then the extracted multiple dimensions of time data are directly spliced to the target transaction data to be detected to obtain the target multi-dimensional data containing time characteristics.

[0050] For example, the target transaction data A is "transaction volume-100; transaction success rate-90%", and the target acquisition time of the target transaction data A is "May 10, 2020, 10:00". Four dimensions of time data can be extracted from the target acquisition time, which are "year-2020; month-5; week-2; time-40". Among them, the extraction method of the "week" dimension is to divide every 7 days into a week from the first day, so the 10th day is the second week; the extraction method of the "time" dimension is to divide every 15 minutes into a time period from 0:00, so 10:00 is the 40th time period. Further, the four dimensions of time data extracted from the target acquisition time are spliced with the target transaction data A to obtain the target multi-dimensional data A1, which is "transaction volume-100; transaction success rate-90%; year-2020; month-5; week-2; time-40".

[0051] It can be understood that the target transaction data of the target business will change periodically according to the change of time, for example, the transaction volume in some fixed time period B will surge, at this time, if only the numerical value of the target transaction data is used to detect the abnormality of the target transaction data, the normal target transaction data in the fixed time period B will be detected as abnormal data, and further detection errors will occur. Combining the target transaction data with the time characteristics of the target transaction data can solve the above problems.

[0052] S202, performing abnormality detection on the target multi-dimensional data by using an abnormality detection model to obtain an abnormality detection result of the target transaction data.

[0053] The abnormality detection model is a trained neural network model used for abnormality detection of the target transaction data; the abnormality detection result refers to the detection result of the target transaction data, including data normal and data abnormal.

[0054] Optionally, the target multi-dimensional data can be input into the anomaly detection model, and the anomaly detection model can analyze the target multi-dimensional data from multiple dimensions according to the target multi-dimensional data and parameters of the anomaly detection model, so as to obtain the anomaly detection result of the target transaction data.

[0055] In the above data anomaly detection method, the target acquisition time is introduced, the target transaction data is processed through the target acquisition time of the target transaction data, the target multi-dimensional data containing the time feature is obtained, and then the anomaly detection model is used to perform anomaly detection on the target multi-dimensional data, so as to obtain the anomaly detection result of the target transaction data. Compared with the related art, the above method can mine the target transaction data from the time dimension and the value dimension, avoid the influence of time fluctuation on the anomaly detection result, and improve the accuracy of data anomaly detection.

[0056] In order to improve the accuracy of the anomaly detection model construction, on the basis of the above embodiment, in the present embodiment, an optional way of constructing the anomaly detection model is provided, as shown in Figure 3 The specific steps include the following steps:

[0057] S301, based on at least two initial clustering centers, clustering each sample multi-dimensional data in the sample multi-dimensional data set to obtain a data cluster corresponding to each initial clustering center.

[0058] Among them, the sample multi-dimensional data refers to the multi-dimensional data containing time features obtained by processing the historical transaction data of the target business according to the acquisition time of the historical transaction data; the sample multi-dimensional data set refers to a set containing sample multi-dimensional data; and the data cluster refers to a data set with similar features.

[0059] Optionally, the number of initial clustering centers K can be determined according to the number of elbow points in the out-of-group variance curve diagram obtained in the model historical training process; then, according to each initial clustering center, each sample multi-dimensional data is divided into the range of the nearest initial clustering center, that is, each sample multi-dimensional data in the sample multi-dimensional data set is clustered to obtain K data clusters.

[0060] S302, determining the reference clustering center corresponding to each data cluster according to the mean value of the sample multi-dimensional data contained in each data cluster.

[0061] Among them, the reference clustering center refers to the center point calculated according to the sample multi-dimensional data in the data cluster.

[0062] Optionally, since the correlation between the preset initial clustering center and the sample multidimensional data is weak, in order to improve the accuracy of the initial clustering center, for each data cluster, the mean value can be calculated according to the sample multidimensional data contained in the data cluster; then, the calculated mean value is taken as the reference clustering center corresponding to the data cluster.

[0063] S303, it is judged whether the distance between the initial clustering center corresponding to any data cluster and the reference clustering center is greater than the first preset distance threshold, if yes, S304 is executed; if no, S305 is executed.

[0064] The first preset threshold refers to the data preset for measuring the distance difference between the initial clustering center and the reference clustering center.

[0065] Optionally, the distance between the initial clustering center and the reference clustering center can be compared with the first preset distance threshold, if the distance between the initial clustering center and the reference clustering center is greater than the first preset distance threshold, S304 can be executed; if the distance between the initial clustering center and the reference clustering center is less than the first preset distance threshold, S305 can be executed.

[0066] S304, each reference clustering center is taken as a new initial clustering center, and S301 is returned to be executed.

[0067] It can be understood that when the distance between the initial clustering center and the reference clustering center is greater than the first preset distance threshold, it indicates that the error between the initial clustering center and the reference clustering center is large, and the next round of clustering center updating operation is needed at this time.

[0068] Optionally, each reference clustering center can be taken as a new initial clustering center, and then S301 is returned to be executed, so as to realize the updating of the clustering center, until the distance between the initial clustering center and the reference clustering center is less than the first preset distance threshold.

[0069] S305, each reference clustering center is taken as a model clustering center.

[0070] The model clustering center refers to the clustering center finally determined after multiple times.

[0071] It can be understood that when the distance between the initial clustering center and the reference clustering center is less than the first preset distance threshold, it indicates that the error between the initial clustering center and the reference clustering center is small, at this time, the updating operation of the clustering center can be stopped, and each reference clustering center is taken as a model clustering center.

[0072] S306, an anomaly detection model is constructed according to the model clustering center.

[0073] Optionally, since each model clustering center is determined according to the sample multidimensional data, each model clustering center has similar data features with the sample multidimensional data, and at this time, the model clustering center can be used as a model parameter to construct the anomaly detection model.

[0074] Further, when performing data anomaly detection, the model clustering center in the anomaly detection model can be compared with the target multidimensional data to determine whether there is a model clustering center that has a high similarity with the target multidimensional data, and then an anomaly detection result of the target transaction data is obtained.

[0075] In this embodiment, the mean clustering method is used, and the model clustering center can be directly determined according to the sample multidimensional data, and then the anomaly detection model is constructed, without the need to label the sample multidimensional data, thereby saving human resources and improving the efficiency of model construction.

[0076] In order to improve the accuracy of the anomaly detection result, on the basis of the above embodiment, in this embodiment, an optional way of determining the anomaly detection result is provided, as shown in Figure 4 The specific steps include the following steps:

[0077] S401, from each model clustering center of the anomaly detection model, a model clustering center closest to the target multidimensional data is selected as a target clustering center.

[0078] The target clustering center refers to the clustering center closest to the target multidimensional data among the model clustering centers.

[0079] Optionally, since the closer the distance between the model clustering center and the target multidimensional data, the greater the similarity between the model clustering center and the target multidimensional data, the distance between each model clustering center and the target multidimensional data can be calculated, and the model clustering center closest to the target multidimensional data is selected as the target clustering center.

[0080] S402, if the distance between the target clustering center and the target multidimensional data is greater than a second preset distance threshold, the target multidimensional data is verified for anomaly, and an anomaly detection result of the target transaction data is obtained.

[0081] The second preset distance threshold refers to a data preset for measuring the distance difference between the target clustering center and the target multidimensional data.

[0082] Optionally, the distance between the target clustering center and the target multidimensional data can be calculated, and if the distance between the target clustering center and the target multidimensional data is less than the second preset distance threshold, it indicates that the target multidimensional data has similar data features with the target clustering center, that is, the anomaly detection result of the target transaction data is normal data.

[0083] If the distance between the target cluster center and the target multidimensional data is greater than the second preset distance threshold, it indicates that the target multidimensional data and the target cluster center do not have similar data characteristics, and at this time, the target multidimensional data needs to be verified for abnormality, and then the abnormal detection result of the target transaction data is obtained.

[0084] In the embodiment, the second preset distance threshold is introduced, and the distance between the target cluster center and the target multidimensional data is compared with the second preset distance threshold to determine the abnormal detection result of the target transaction data, thereby improving the accuracy of the abnormal detection result determination.

[0085] It can be understood that in the case that there are too many data dimensions in the target multidimensional data, the abnormal detection result will be affected, and therefore, in order to further improve the accuracy of the abnormal detection result determination, an optional abnormality verification method is provided in the embodiment, as shown in Figure 5 The method comprises the following steps:

[0086] S501, determining a dimension distance value of the target cluster center and the target multidimensional data in each data dimension.

[0087] The dimension distance value refers to the distance value of the target multidimensional data in each data dimension, and can represent the abnormality degree of the target multidimensional data in each dimension.

[0088] Optionally, for each data dimension, the dimension distance value of the target cluster center and the target multidimensional data in the data dimension can be calculated. For example, the target multidimensional data C is "transaction volume-100; time-40", and therefore the coordinates of the target multidimensional data C are (100, 40), and the coordinates of the target cluster center are (90, 20). At this time, in the "transaction volume" dimension, the dimension distance value is 10; and in the "time" dimension, the dimension distance value is 20.

[0089] S502, sorting each data dimension according to the dimension distance value in descending order.

[0090] Optionally, in order to determine the abnormality degree of each data dimension, each data dimension can be sorted according to the dimension distance value in descending order.

[0091] S503, according to the sorting result, taking a preset number of data dimensions in the front as abnormal data dimensions.

[0092] The abnormal data dimension refers to the reason causing the abnormality of the target multidimensional data; and the preset number refers to the number of abnormal data dimensions determined according to the number of data dimensions contained in the target multidimensional data, for example, the target multidimensional data contains 8 data dimensions, and the preset number can be 3.

[0093] Optionally, since the data dimension arranged in the front is a data dimension with a greater abnormality degree, the preset number of data dimensions arranged in the front can be determined as abnormal data dimensions according to the sorting result.

[0094] S504, according to the dimension distance value corresponding to the abnormal data dimension, performing abnormal verification on the target multi-dimensional data to obtain an abnormal detection result of the target transaction data.

[0095] Optionally, the dimension distance values corresponding to the abnormal data dimensions can be directly added, and the calculation result is compared with a preset abnormal threshold value. If the calculation result is greater than the abnormal threshold value, the abnormal detection result of the target transaction data is data abnormality; if the calculation result is less than the abnormal threshold value, the abnormal detection result of the target transaction data is data normality.

[0096] It can be understood that since the number of orders is different under each abnormal data dimension, directly adding the dimension distance values corresponding to each abnormal data dimension will lead to an inaccurate abnormal verification result. Therefore, a corresponding dimension weight can be set for each abnormal data dimension before abnormal verification. Subsequently, the following steps are used to determine the abnormal detection result. The dimension weight refers to a numerical value used to represent the importance of each data dimension.

[0097] The first step is to perform weighted processing on the dimension distance value corresponding to the abnormal data dimension according to the dimension weight of the abnormal data dimension to obtain an abnormal value of the abnormal data dimension.

[0098] Optionally, for each abnormal data dimension, the dimension weight of the abnormal data dimension can be multiplied by the dimension distance value corresponding to the abnormal data dimension, and the calculation result can be used as the abnormal value of the abnormal data dimension.

[0099] The second step is to determine an abnormal value of the target multi-dimensional data according to the abnormal value of the abnormal data dimension.

[0100] Optionally, the abnormal values of each abnormal data dimension are added, and the calculation result is used as the abnormal value of the target multi-dimensional data.

[0101] The third step is to determine that the abnormal detection result of the target transaction data is data abnormality if the abnormal value of the target multi-dimensional data is greater than a preset abnormal threshold value.

[0102] The preset abnormal value refers to a numerical value preset for judging the abnormal detection result of the target multi-dimensional data.

[0103] Optionally, the abnormal value of the target multi-dimensional data is compared with a preset abnormal threshold value, if the abnormal value of the target multi-dimensional data is greater than the preset abnormal threshold value, it is determined that the abnormal detection result of the target transaction data is data abnormality; if the abnormal value of the target multi-dimensional data is less than the preset abnormal threshold value, it is determined that the abnormal detection result of the target transaction data is data normality.

[0104] In the embodiment, the abnormal data dimension is introduced, and in the case that the number of data dimensions is large, the abnormal data dimension with a large influence degree on the target multi-dimensional data is determined, so that the accuracy of the abnormal detection result of the target multi-dimensional data is ensured.

[0105] Figure 6 For another flowchart of the data abnormality detection method in the embodiment, on the basis of the above embodiment, the embodiment provides an optional example of a data abnormality detection method. In combination with Figure 6 , the specific implementation process is as follows:

[0106] S601, the target acquisition time of the target transaction data to be detected and the target transaction data are spliced to obtain target multi-dimensional data containing time characteristics.

[0107] S602, from the model clustering centers of the abnormality detection model, the model clustering center closest to the target multi-dimensional data is selected as the target clustering center.

[0108] Optionally, the abnormality detection model is constructed in the following manner: based on at least two initial clustering centers, each sample multi-dimensional data in a sample multi-dimensional data set is clustered to obtain data clusters corresponding to each initial clustering center; according to the mean value of the sample multi-dimensional data contained in each data cluster, a reference clustering center corresponding to each data cluster is determined; if the distance between the initial clustering center and the reference clustering center of any data cluster is greater than a first preset distance threshold value, each reference clustering center is taken as a new initial clustering center, and the operation of clustering each sample multi-dimensional data in the sample multi-dimensional data set based on at least two initial clustering centers to obtain data clusters corresponding to each initial clustering center is returned until the distance between the initial clustering center and the reference clustering center of any data cluster is less than the first preset distance threshold value, each reference clustering center is taken as a model clustering center; and the abnormality detection model is constructed according to the model clustering center.

[0109] S603, it is judged whether the distance between the target clustering center and the target multi-dimensional data is greater than a second preset distance threshold value, if yes, S604 is executed; if no, S611 is executed.

[0110] S604, the dimension distance value of the target clustering center and the target multi-dimensional data in each data dimension is determined.

[0111] S605, sort the data dimensions according to the dimension distance values in descending order.

[0112] S606, according to the sorting result, take the data dimensions in the front of the sorting result as the abnormal data dimensions.

[0113] S607, according to the dimension weight of the abnormal data dimension, weight the dimension distance value corresponding to the abnormal data dimension to obtain the abnormal value of the abnormal data dimension.

[0114] S608, according to the abnormal value of the abnormal data dimension, determine the abnormal value of the target multi-dimensional data.

[0115] S609, determine whether the abnormal value of the target multi-dimensional data is greater than a preset abnormal threshold, if yes, execute S610; if not, execute S611.

[0116] S610, determine that the abnormal detection result of the target transaction data is data abnormal.

[0117] S611, determine that the abnormal detection result of the target transaction data is data normal.

[0118] The specific process of S601-S611 can be referred to the description of the method embodiments, and the implementation principle and technical effects are similar, which will not be repeated here.

[0119] It should be understood that, although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise stated herein, the execution of these steps has no strict sequence limitation, and these steps can be executed in other order. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0120] Based on the same inventive concept, the embodiments of the present application also provide a data anomaly detection device for implementing the data anomaly detection method as described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more data anomaly detection device embodiments provided below can be referred to the limitations of the data anomaly detection method in the above, which will not be repeated here.

[0121] In one embodiment, as Figure 7As shown, a data anomaly detection apparatus 1 is provided, comprising a data processing module 10 and an anomaly detection module 20, wherein:

[0122] The data processing module 10 is configured to process the target transaction data according to a target acquisition time of the target transaction data to be detected, to obtain target multi-dimensional data containing time characteristics.

[0123] The anomaly detection module 20 is configured to perform anomaly detection on the target multi-dimensional data by using an anomaly detection model, to obtain an anomaly detection result of the target transaction data.

[0124] In one embodiment, the data processing module 10 is specifically configured to:

[0125] Splice the target acquisition time of the target transaction data to be detected and the target transaction data, to obtain the target multi-dimensional data containing time characteristics.

[0126] In one embodiment, the data anomaly detection apparatus further comprises a model construction module, wherein the model construction module is specifically configured to:

[0127] Cluster each sample multi-dimensional data in the sample multi-dimensional data set based on at least two initial cluster centers to obtain data clusters corresponding to the initial cluster centers; determine reference cluster centers corresponding to each data cluster according to the mean values of the sample multi-dimensional data contained in each data cluster; if the distance between the initial cluster center and the reference cluster center corresponding to any data cluster is greater than a first preset distance threshold, take each reference cluster center as a new initial cluster center, and return to perform the operation of clustering each sample multi-dimensional data in the sample multi-dimensional data set based on at least two initial cluster centers to obtain data clusters corresponding to the initial cluster centers, until the distance between the initial cluster center and the reference cluster center corresponding to any data cluster is less than the first preset distance threshold, take each reference cluster center as a model cluster center; and construct an anomaly detection model according to the model cluster center.

[0128] In one embodiment, as shown, Figure 8 The anomaly detection module 20 comprises:

[0129] The target selection unit 21 is configured to select, from each model cluster center of the anomaly detection model, a model cluster center closest to the target multi-dimensional data as a target cluster center;

[0130] The anomaly detection unit 22 is configured to perform anomaly verification on the target multi-dimensional data if the distance between the target cluster center and the target multi-dimensional data is greater than a second preset distance threshold, to obtain the anomaly detection result of the target transaction data.

[0131] In one embodiment, the anomaly detection unit 22 comprises:

[0132] The distance value determination sub-unit is configured to determine a dimension distance value of the target cluster center and the target multi-dimensional data in each data dimension;

[0133] The sorting sub-unit is configured to sort each data dimension according to the dimension distance value in descending order;

[0134] The abnormal dimension determination sub-unit is configured to determine a preset number of data dimensions in the front of the sorting result as abnormal data dimensions according to the sorting result;

[0135] The anomaly detection sub-unit is configured to perform anomaly verification on the target multi-dimensional data according to the dimension distance value corresponding to the abnormal data dimension, and obtain an anomaly detection result of the target transaction data.

[0136] In an embodiment, the anomaly detection sub-unit is specifically configured to:

[0137] weight the dimension distance value corresponding to the abnormal data dimension according to the dimension weight of the abnormal data dimension to obtain an anomaly value of the abnormal data dimension; determine an anomaly value of the target multi-dimensional data according to the anomaly value of the abnormal data dimension; and if the anomaly value of the target multi-dimensional data is greater than a preset anomaly threshold, determine that the anomaly detection result of the target transaction data is data anomaly.

[0138] Each module in the data anomaly detection apparatus described above can be realized by software, hardware, or a combination thereof, in whole or in part. Each module described above can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in a computer device in software form, so as to be called and executed by a processor to perform the operations corresponding to each module.

[0139] In an embodiment, a computer device is provided, which can be a server, and an internal structure diagram of the computer device can be as shown in Figure 9 The computer device includes a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is configured to store target multi-dimensional data. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through a network connection. The computer program is executed by the processor to implement a data anomaly detection method.

[0140] Those skilled in the art can understand that Figure 9 The skilled in the art can understand that

[0141] In one embodiment, a computer device is provided, comprising a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0142] According to the target acquisition time of the target transaction data to be detected, the target transaction data is processed to obtain target multi-dimensional data containing time characteristics;

[0143] The target multi-dimensional data is detected by the anomaly detection model to obtain an anomaly detection result of the target transaction data.

[0144] In one embodiment, when the processor executes the logic in the computer program for processing the target transaction data to be detected according to the target acquisition time of the target transaction data, the following steps are implemented:

[0145] The target acquisition time of the target transaction data to be detected and the target transaction data are spliced to obtain target multi-dimensional data containing time characteristics.

[0146] In one embodiment, when the processor executes the logic in the computer program for constructing the anomaly detection model, the following steps are implemented:

[0147] Based on the at least two initial clustering centers, each sample multi-dimensional data in the sample multi-dimensional data set is clustered to obtain data clusters corresponding to each initial clustering center; according to the mean value of the sample multi-dimensional data contained in each data cluster, a reference clustering center corresponding to each data cluster is determined; if the distance between the initial clustering center and the reference clustering center corresponding to any data cluster is greater than a first preset distance threshold, each reference clustering center is taken as a new initial clustering center, and the operation of clustering each sample multi-dimensional data in the sample multi-dimensional data set based on the at least two initial clustering centers to obtain data clusters corresponding to each initial clustering center is returned to be executed until the distance between the initial clustering center and the reference clustering center corresponding to any data cluster is less than the first preset distance threshold, each reference clustering center is taken as a model clustering center; and the anomaly detection model is constructed according to the model clustering center.

[0148] In one embodiment, when the processor executes the logic of performing, by the anomaly detection model, anomaly detection on the target multi-dimensional data in the computer program to obtain the anomaly detection result of the target transaction data, the following steps are specifically implemented:

[0149] From the model cluster centers of the anomaly detection model, the model cluster center closest to the target multi-dimensional data is selected as the target cluster center; if the distance between the target cluster center and the target multi-dimensional data is greater than the second preset distance threshold, the target multi-dimensional data is subjected to anomaly verification to obtain the anomaly detection result of the target transaction data.

[0150] In one embodiment, when the processor executes the logic of performing anomaly verification on the target multi-dimensional data in the computer program to obtain the anomaly detection result of the target transaction data, the following steps are specifically implemented:

[0151] The dimension distance values of the target cluster center and the target multi-dimensional data in each data dimension are determined; each data dimension is sorted according to the dimension distance values in descending order; a preset number of data dimensions in the front of the sorting result are taken as the abnormal data dimensions; the target multi-dimensional data is subjected to anomaly verification according to the dimension distance values corresponding to the abnormal data dimensions to obtain the anomaly detection result of the target transaction data.

[0152] In one embodiment, when the processor executes the logic of performing anomaly verification on the target multi-dimensional data according to the dimension distance values corresponding to the abnormal data dimensions in the computer program to obtain the anomaly detection result of the target transaction data, the following steps are specifically implemented:

[0153] The dimension distance values corresponding to the abnormal data dimensions are weighted processed according to the dimension weights of the abnormal data dimensions to obtain the abnormal values of the abnormal data dimensions; the abnormal value of the target multi-dimensional data is determined according to the abnormal values of the abnormal data dimensions; if the abnormal value of the target multi-dimensional data is greater than a preset abnormal threshold, it is determined that the anomaly detection result of the target transaction data is data anomaly.

[0154] In one embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor, the following steps are implemented:

[0155] The target transaction data is processed according to the target acquisition time of the target transaction data to be detected to obtain target multi-dimensional data containing time characteristics;

[0156] The target multi-dimensional data is subjected to anomaly detection by the anomaly detection model to obtain the anomaly detection result of the target transaction data.

[0157] In one embodiment, the code logic in the computer program for processing the target transaction data according to the target acquisition time of the target transaction data to be detected to obtain the target multi-dimensional data containing the time feature is implemented by the processor as follows:

[0158] The target acquisition time of the target transaction data to be detected and the target transaction data are spliced to obtain the target multi-dimensional data containing the time feature.

[0159] In one embodiment, the code logic in the computer program for constructing the anomaly detection model is implemented by the processor as follows:

[0160] Based on the at least two initial clustering centers, each sample multi-dimensional data in the sample multi-dimensional data set is clustered to obtain a data cluster corresponding to each initial clustering center; a reference clustering center corresponding to each data cluster is determined according to the mean value of the sample multi-dimensional data contained in each data cluster; if the distance between the initial clustering center and the reference clustering center corresponding to any data cluster is greater than a first preset distance threshold, each reference clustering center is taken as a new initial clustering center, and the operation of clustering each sample multi-dimensional data in the sample multi-dimensional data set based on the at least two initial clustering centers to obtain a data cluster corresponding to each initial clustering center is returned to be executed until the distance between the initial clustering center and the reference clustering center corresponding to any data cluster is less than the first preset distance threshold, each reference clustering center is taken as a model clustering center; and an anomaly detection model is constructed according to the model clustering center.

[0161] In one embodiment, the code logic in the computer program for performing anomaly detection on the target multi-dimensional data by the anomaly detection model to obtain the anomaly detection result of the target transaction data is implemented by the processor as follows:

[0162] From the model clustering centers of the anomaly detection model, a model clustering center closest to the target multi-dimensional data is selected as a target clustering center; if the distance between the target clustering center and the target multi-dimensional data is greater than a second preset distance threshold, the target multi-dimensional data is subjected to anomaly verification to obtain the anomaly detection result of the target transaction data.

[0163] In one embodiment, the code logic in the computer program for performing anomaly verification on the target multi-dimensional data to obtain the anomaly detection result of the target transaction data is implemented by the processor as follows:

[0164] Determine the dimension distance value of the target cluster center and the target multidimensional data in each data dimension; sort each data dimension according to the dimension distance value in descending order; select the first preset number of data dimensions as the abnormal data dimensions according to the sorting result; and perform abnormal verification on the target multidimensional data according to the dimension distance value corresponding to the abnormal data dimensions to obtain the abnormal detection result of the target transaction data.

[0165] In one embodiment, when the code logic in the computer program for performing abnormal verification on the target multidimensional data according to the dimension distance value corresponding to the abnormal data dimensions to obtain the abnormal detection result of the target transaction data is executed by the processor, the following steps are specifically implemented:

[0166] According to the dimension weight of the abnormal data dimension, the dimension distance value corresponding to the abnormal data dimension is weighted to obtain the abnormal value of the abnormal data dimension; the abnormal value of the target multidimensional data is determined according to the abnormal value of the abnormal data dimension; and if the abnormal value of the target multidimensional data is greater than a preset abnormal threshold, the abnormal detection result of the target transaction data is determined as data abnormality.

[0167] In one embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the following steps:

[0168] According to the target acquisition time of the target transaction data to be detected, the target transaction data is processed to obtain target multidimensional data containing time characteristics.

[0169] The target multidimensional data is detected by the abnormal detection model to obtain the abnormal detection result of the target transaction data.

[0170] In one embodiment, when the computer program is executed by the processor to process the target transaction data to obtain target multidimensional data containing time characteristics according to the target acquisition time of the target transaction data to be detected, the following steps are specifically implemented:

[0171] The target acquisition time of the target transaction data to be detected and the target transaction data are spliced to obtain target multidimensional data containing time characteristics.

[0172] In one embodiment, when the computer program is executed by the processor to construct the abnormal detection model, the following steps are specifically implemented:

[0173] The sample multidimensional data in the sample multidimensional data set is clustered based on the at least two initial clustering centers to obtain data clusters corresponding to the initial clustering centers; reference clustering centers corresponding to the data clusters are determined according to the mean values of the sample multidimensional data contained in the data clusters; if the distance between the initial clustering center and the reference clustering center corresponding to any data cluster is greater than a first preset distance threshold, the reference clustering centers are taken as new initial clustering centers, and the operation of clustering the sample multidimensional data in the sample multidimensional data set based on the at least two initial clustering centers to obtain data clusters corresponding to the initial clustering centers is performed again until the distance between the initial clustering center and the reference clustering center corresponding to any data cluster is less than the first preset distance threshold, and the reference clustering centers are taken as model clustering centers; and an anomaly detection model is constructed according to the model clustering centers.

[0174] In one embodiment, when the computer program is executed by the processor to perform the operation of performing anomaly detection on the target multidimensional data through the anomaly detection model to obtain the anomaly detection result of the target transaction data, the following steps are specifically implemented:

[0175] A model clustering center closest to the target multidimensional data is selected from the model clustering centers of the anomaly detection model as a target clustering center; if the distance between the target clustering center and the target multidimensional data is greater than a second preset distance threshold, the target multidimensional data is subjected to anomaly verification to obtain the anomaly detection result of the target transaction data.

[0176] In one embodiment, when the computer program is executed by the processor to perform the operation of performing anomaly verification on the target multidimensional data to obtain the anomaly detection result of the target transaction data, the following steps are specifically implemented:

[0177] Dimension distance values of the target clustering center and the target multidimensional data in each data dimension are determined; each data dimension is sorted according to the dimension distance values in descending order; a preset number of data dimensions sorted in the front are taken as abnormal data dimensions according to the sorting result; and the target multidimensional data is subjected to anomaly verification according to the dimension distance values corresponding to the abnormal data dimensions to obtain the anomaly detection result of the target transaction data.

[0178] In one embodiment, when the computer program is executed by the processor to perform the operation of performing anomaly verification on the target multidimensional data according to the dimension distance values corresponding to the abnormal data dimensions to obtain the anomaly detection result of the target transaction data, the following steps are specifically implemented:

[0179] The dimension distance value corresponding to the abnormal data dimension is weighted according to the dimension weight of the abnormal data dimension, to obtain an abnormal value of the abnormal data dimension; the abnormal value of the target multi-dimensional data is determined according to the abnormal value of the abnormal data dimension; if the abnormal value of the target multi-dimensional data is greater than a preset abnormal threshold, the abnormal detection result of the target transaction data is determined as data abnormality.

[0180] It should be noted that the data involved in the present application (including but not limited to target multi-dimensional data and sample multi-dimensional data) are all data authorized by users or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0181] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to a memory, database or other medium used in the embodiments provided by the present application can include at least one of a non-volatile and volatile memory. The non-volatile memory can include a read-only memory (ROM), a magnetic tape, a floppy disk, a flash memory, an optical storage, a high-density embedded non-volatile memory, a resistive memory (ReRAM), a magnetoresistive random access memory (MRAM), a ferroelectric memory (FRAM), a phase change memory (PCM), a graphene memory, etc. The volatile memory can include a random access memory (RAM) or an external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided by the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided by the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0182] Any technical features in the above embodiments can be combined, and for the sake of brevity, not all possible combinations are described above, however, as long as the combinations of technical features do not have contradictions, they shall be considered within the scope of the present disclosure.

[0183] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it shall not be understood as a limitation on the patent scope of the present application. It shall be pointed out that, for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, and these shall be within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A data anomaly detection method, characterized by, The method comprises: processing the target transaction data according to the target acquisition time of the target transaction data to be detected to obtain target multi-dimensional data containing time characteristics; performing anomaly detection on the target multi-dimensional data through an anomaly detection model to obtain an anomaly detection result of the target transaction data; wherein the anomaly detection model is constructed in the following manner: performing clustering on each sample multi-dimensional data in a sample multi-dimensional data set based on at least two initial clustering centers to obtain data clusters corresponding to each initial clustering center; determining reference clustering centers corresponding to each data cluster according to the mean values of the sample multi-dimensional data contained in each data cluster; if the distance between the initial clustering center and the reference clustering center of any data cluster is greater than a first preset distance threshold, then taking each reference clustering center as a new initial clustering center and returning to perform the operation of performing clustering on each sample multi-dimensional data in a sample multi-dimensional data set based on at least two initial clustering centers to obtain data clusters corresponding to each initial clustering center until the distance between the initial clustering center and the reference clustering center of any data cluster is less than the first preset distance threshold, and taking each reference clustering center as a model clustering center; constructing the anomaly detection model according to the model clustering center.

2. The method of claim 1, wherein, The processing of the target transaction data according to the target acquisition time of the target transaction data to be detected to obtain target multi-dimensional data containing time characteristics comprises: splicing the target acquisition time of the target transaction data to be detected and the target transaction data to obtain target multi-dimensional data containing time characteristics.

3. The method of claim 1, wherein, The performing of anomaly detection on the target multi-dimensional data through an anomaly detection model to obtain an anomaly detection result of the target transaction data comprises: selecting a model clustering center closest to the target multi-dimensional data from each model clustering center of the anomaly detection model as a target clustering center; if the distance between the target clustering center and the target multi-dimensional data is greater than a second preset distance threshold, then performing anomaly verification on the target multi-dimensional data to obtain an anomaly detection result of the target transaction data.

4. The method of claim 3, wherein, The performing of anomaly verification on the target multi-dimensional data to obtain an anomaly detection result of the target transaction data comprises: determining the dimension distance values of the target clustering center and the target multi-dimensional data in each data dimension; sorting each data dimension according to the dimension distance values in descending order; taking a preset number of data dimensions sorted in the front as abnormal data dimensions according to the sorting result; performing anomaly verification on the target multi-dimensional data according to the dimension distance values corresponding to the abnormal data dimensions to obtain an anomaly detection result of the target transaction data.

5. The method of claim 4, wherein, The performing of anomaly verification on the target multi-dimensional data according to the dimension distance values corresponding to the abnormal data dimensions to obtain an anomaly detection result of the target transaction data comprises: performing weighted processing on the dimension distance values corresponding to the abnormal data dimensions according to the dimension weights of the abnormal data dimensions to obtain abnormal values of the abnormal data dimensions. According to the abnormal value of the abnormal data dimension, an abnormal value of the target multi-dimensional data is determined; If the abnormal value of the target multi-dimensional data is greater than a preset abnormal threshold, it is determined that the abnormal detection result of the target transaction data is data abnormality.

6. A data anomaly detection apparatus characterized by comprising: The apparatus comprises: A data processing module is configured to process the target transaction data according to a target acquisition time of the target transaction data to be detected, and obtain target multi-dimensional data containing time characteristics; An abnormal detection module is configured to perform abnormal detection on the target multi-dimensional data by using an abnormal detection model, and obtain an abnormal detection result of the target transaction data; A model construction module is configured to: cluster each sample multi-dimensional data in a sample multi-dimensional data set based on at least two initial clustering centers to obtain data clusters corresponding to the initial clustering centers; determine reference clustering centers corresponding to each data cluster according to the mean values of the sample multi-dimensional data contained in each data cluster; if the distance between the initial clustering center and the reference clustering center of any data cluster is greater than a first preset distance threshold, use each reference clustering center as a new initial clustering center, and return to perform the operation of clustering each sample multi-dimensional data in the sample multi-dimensional data set based on at least two initial clustering centers to obtain data clusters corresponding to the initial clustering centers until the distance between the initial clustering center and the reference clustering center of any data cluster is less than the first preset distance threshold, and use each reference clustering center as a model clustering center; and construct the abnormal detection model according to the model clustering center. 7.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-6 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the method of any one of claims 1 to 5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 5.

9. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 5. The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 5.

Images