ERP (Enterprise Resource Planning) data anomaly analysis method and system based on big data

Abstract

The invention relates to the technical field of ERP (Enterprise Resource Planning) data processing, in particular to an ERP data exception analysis method and system based on big data. Extracting time sequence features and frequency domain features of the ERP potential abnormal data, inputting the time sequence features and the frequency domain features into a fluctuation detection model, carrying out fluctuation recognition on the ERP potential abnormal data to obtain ERP fluctuation data, and carrying out multi-dimensional linkage analysis on the ERP fluctuation data through a covariance analysis method to obtain linkage strength among different types of ERP fluctuation data; according to the linkage strength and the time sequence characteristics and the frequency domain characteristics of the ERP potential abnormal data, a service category target threshold value is obtained through a dynamic threshold value adjustment model, abnormal judgment is conducted on the ERP fluctuation data according to the service category target threshold value, when the ERP fluctuation data exceed the target threshold value range, the ERP fluctuation data are marked as first target abnormal data, and a data abnormal early warning mechanism is triggered. And more accurate data anomaly analysis is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of ERP data processing, and in particular to a method and system for analyzing ERP data anomaly based on big data. Background Art

[0002] With the expansion of enterprise scale and the diversification of management needs, ERP system has become an important tool for modern enterprise management, which can help enterprises realize efficient resource allocation, real-time monitoring and automation of business processes. However, for some enterprises, the business data involved in ERP system is usually highly dynamic and complex. Especially in sales enterprises, data will be affected by external factors such as seasonal changes, holiday promotions, market demand fluctuations, etc., and will fluctuate greatly in a short period of time. Traditional anomaly analysis methods usually rely on fixed rules or thresholds to analyze data anomalies. When facing such enterprise data, traditional methods often have difficulty in effectively distinguishing normal business fluctuations from real anomalies, resulting in normal business data fluctuations being misjudged as abnormal data, which in turn brings unnecessary interference and extra attention to staff. Therefore, how to identify the fluctuation patterns of business data and analyze data anomalies in combination with the fluctuation patterns, avoid the interference of normal data fluctuations, and achieve more accurate data anomaly analysis, has become an urgent problem to be solved. Summary of the invention

[0003] The present invention provides an ERP data anomaly analysis method and system based on big data, which solves the problem that the volatility of ERP business data interferes with data anomaly determination and makes data anomaly analysis inaccurate.

[0004] To achieve the above object, on the one hand, the present invention provides an ERP data anomaly analysis method based on big data, the method comprising: Obtain raw business data of preset categories from the ERP system, clean and denoise the raw business data to obtain standardized business data, compare the standardized business data with the preset initial abnormality judgment threshold, and mark the standardized business data outside the initial abnormality judgment threshold as potential abnormal data for ERP.

[0005] The time series characteristics and frequency domain characteristics of ERP potential abnormal data are extracted and input into the fluctuation detection model to perform fluctuation identification on the ERP potential abnormal data to obtain ERP fluctuation data. The ERP fluctuation data is subjected to multi-dimensional linkage analysis through covariance analysis to obtain the linkage strength between different categories of ERP fluctuation data.

[0006] According to the linkage strength, the time series characteristics and frequency domain characteristics of ERP potential abnormal data, the business category target threshold is obtained through the dynamic threshold adjustment model. The ERP fluctuation data is judged as abnormal according to the business category target threshold. When the ERP fluctuation data exceeds the target threshold range, it is marked as the first target abnormal data and the data abnormality early warning mechanism is triggered.

[0007] Furthermore, the method of extracting the time series characteristics and frequency domain characteristics of the ERP potential abnormal data and inputting them into the fluctuation detection model to perform fluctuation identification on the ERP potential abnormal data to obtain ERP fluctuation data, and performing multi-dimensional linkage analysis on the ERP fluctuation data through the covariance analysis method to obtain the linkage strength between different categories of ERP fluctuation data includes: The time series features of ERP potential abnormal data are extracted, including the coefficient of variation of business data, the trend of business data, and the volatility index of business data. The ERP potential abnormal data is converted into frequency domain data through Fourier transform and the frequency domain features of ERP potential abnormal data are extracted, including the main frequency intensity of business data, the frequency concentration of business data, and the harmonic ratio of business data.

[0008] The business data variation coefficient, business data trend, business data fluctuation index, business data main frequency intensity, business data frequency concentration, and business data harmonic ratio of the data are summarized to obtain a potential abnormal data target feature vector, and the potential abnormal data target feature vector is input into a fluctuation detection model to identify the fluctuation of ERP potential abnormal data to obtain a fluctuation category, wherein the fluctuation category includes periodic fluctuation and irregular fluctuation, and the ERP potential abnormal data whose fluctuation category is irregular fluctuation is marked as the second target abnormal data and triggers a data abnormality early warning mechanism, and the ERP potential abnormal data whose fluctuation category is periodic fluctuation is marked as ERP fluctuation data; the fluctuation detection model is a mapping relationship between the potential abnormal data target feature vector and the fluctuation category established by a support vector machine algorithm based on the historical data of ERP potential abnormal data.

[0009] The ERP fluctuation data were subjected to multidimensional linkage analysis through covariance analysis to obtain the linkage strength between different categories of ERP fluctuation data.

[0010] Furthermore, the method for extracting the time series characteristics of the ERP potential abnormal data including the business data variation coefficient, the business data trend degree, and the business data fluctuation index includes: According to the preset time window length, the sliding window method is used to obtain the data of the previous preset time window length with the corresponding data at the current time as the end point in the ERP potential abnormal data to obtain the first target data sequence, and the business data variation coefficient is calculated based on the data in the first target data sequence. for: .

[0011] in is the first target data sequence data points, is the mean of all data in the first target data sequence, is the number of data in the first target data sequence, is the coefficient of variation of business data.

[0012] The business data trend degree is calculated based on the data in the first target data sequence for: .

[0013] in is the first target data sequence data points, It is the trend of business data.

[0014] The business data fluctuation index is calculated based on the data in the first target data sequence for: .

[0015] in is the first target data sequence The maximum value of the data point, is the first target data sequence The minimum value of the data point, It is the business data volatility index.

[0016] Furthermore, the method of converting the ERP potential abnormal data into frequency domain data through Fourier transform and extracting the frequency domain features of the ERP potential abnormal data including the main frequency intensity of the business data, the frequency concentration of the business data, and the harmonic ratio of the business data includes: The first target data sequence is expanded according to the preset time window length by the zero-filling extension method to a minimum integer power of 2 of the number of data in the first target data sequence to obtain the second target data sequence.

[0017] Perform Fourier transform on the second target data sequence to obtain a spectrum sequence, and calculate the spectrum energy of each spectrum component in the spectrum sequence according to the spectrum sequence for: .

[0018] in is the frequency number and from start, is the first The spectral components corresponding to the frequency numbers are: is the first The real part of the spectral component corresponding to the frequency number, is the first The imaginary part of the spectral component corresponding to the frequency number, It is The spectral energy of the spectral component corresponding to the frequency sequence number.

[0019] Summarize the spectrum energy of each spectrum component to obtain the spectrum energy sequence, and calculate the main frequency intensity of the service data based on the spectrum energy sequence for: .

[0020] in is the spectrum energy sequence, is the maximum value in the spectrum energy sequence, is the number of data in the spectral energy sequence, It is the main frequency strength of business data.

[0021] Calculate the frequency concentration of service data based on the spectrum energy sequence for: .

[0022] in It is The frequency value of the spectral component corresponding to the frequency sequence number, It is the frequency concentration of business data.

[0023] Calculate the harmonic ratio of service data based on the spectrum energy sequence for: .

[0024] in is the business data harmonic ratio.

[0025] Furthermore, the method of performing multi-dimensional linkage analysis on the ERP fluctuation data by covariance analysis to obtain linkage strength between different categories of ERP fluctuation data includes: The business data variation coefficient, business data trend, business data fluctuation index, business data main frequency intensity, business data frequency concentration, and business data harmonic ratio of different categories of ERP fluctuation data are summarized to obtain the initial feature matrix of business linkage.

[0026] Calculate the business linkage covariance based on the business linkage initial feature matrix for: .

[0027] in is the first row of data of the initial feature matrix, is the second row of data of the initial feature matrix, is the mean of the first row of data in the initial feature matrix, is the mean of the second row of data in the initial feature matrix, is the number of columns of the initial feature matrix, is the business linkage covariance.

[0028] Get linkage strength based on business linkage covariance for: .

[0029] in is the standard deviation of the first row of data samples in the initial feature matrix, is the standard deviation of the data sample in the second row of the initial feature matrix, Linkage strength.

[0030] Furthermore, the method of obtaining the target threshold of the business category through a dynamic threshold adjustment model according to the linkage strength, the time series characteristics and the frequency domain characteristics of the ERP potential abnormal data includes: The comprehensive linkage strength is obtained based on the linkage strength between ERP fluctuation data and all business categories analyzed in linkage with it. for: .

[0031] in It is the ERP fluctuation data and The linkage strength of the business category with which the linkage analysis is conducted, is the number of business categories to be analyzed for linkage, It is the comprehensive linkage strength.

[0032] The business data variation coefficient, business data fluctuation index and business data main frequency intensity in the time series characteristics of ERP potential abnormal data are obtained. According to the business data variation coefficient, business data fluctuation index, business data main frequency intensity and comprehensive linkage intensity, the initial abnormality judgment threshold is corrected to obtain the business category target threshold: ; .

[0033] in is the upper limit of the initial abnormality judgment threshold, is the lower limit of the initial abnormality judgment threshold, is the coefficient of variation of business data, is the business data volatility index, is the main frequency strength of business data, is the natural logarithm function, is the upper threshold value of the business category target, is the lower limit of the target threshold of the business category.

[0034] Furthermore, before obtaining the comprehensive linkage strength according to the linkage strength between the ERP fluctuation data and all the business categories analyzed in linkage therewith, the method further includes: The linkage strength of ERP fluctuation data is compared with the preset linkage strength threshold list between categories. When the linkage strength between ERP fluctuation data and the business category to be analyzed with linkage is greater than or equal to the preset linkage strength threshold between categories, the ERP fluctuation data is marked as threshold data to be adjusted; When the linkage strength between ERP fluctuation data and the business category with which it is linked for analysis is less than the preset inter-category linkage strength threshold, the ERP fluctuation data is marked as abnormal fluctuation and the data abnormality warning mechanism is triggered.

[0035] Furthermore, the training method of the fluctuation detection model includes: Historical ERP potential abnormal data is obtained and its time series features and frequency domain features are extracted to obtain sample feature vectors. The fluctuation categories of the samples are manually labeled to obtain sample labels, including periodic fluctuations and irregular fluctuations. The sample feature vectors and sample labels are used to construct a fluctuation detection sample data set. 80% of the sample data set is used as training data to obtain a fluctuation detection training data set, and 20% of the sample data set is used as verification data to obtain a fluctuation detection verification data set.

[0036] The radial basis kernel function is selected as the kernel function, the fluctuation detection training data set is input, the penalty factor and kernel function parameters are fitted by the grid search method, and the classification accuracy of periodic fluctuations and irregular fluctuations under different parameter combinations is evaluated by k-fold cross validation. The parameter combination with the highest classification accuracy is selected as the optimal parameter.

[0037] The model is verified through the verification data set. When the classification accuracy of periodic fluctuations and irregular fluctuations is greater than the preset threshold, the fluctuation detection model is established. Otherwise, the model is retrained by returning to parameter optimization until the preset performance requirements are met.

[0038] Based on the same inventive concept, the present invention provides an ERP data anomaly analysis system based on big data, the system comprising: a data processing module, an anomaly analysis module, and a threshold adjustment module, and the modules are connected in sequence.

[0039] The data processing module is used to obtain original business data of preset categories from the ERP system, perform data cleaning and denoising on the original business data to obtain standardized business data, compare the standardized business data with a preset initial abnormality determination threshold, and mark the standardized business data outside the initial abnormality determination threshold as potential abnormal data of ERP; The abnormality analysis module is used to extract the time series characteristics and frequency domain characteristics of the ERP potential abnormal data and input them into the fluctuation detection model to perform fluctuation identification on the ERP potential abnormal data to obtain ERP fluctuation data, and perform multi-dimensional linkage analysis on the ERP fluctuation data through the covariance analysis method to obtain the linkage strength between different categories of ERP fluctuation data; The threshold adjustment module is used to obtain the business category target threshold through a dynamic threshold adjustment model according to the linkage strength, the time series characteristics and the frequency domain characteristics of the ERP potential abnormal data, and to make an abnormal judgment on the ERP fluctuation data according to the business category target threshold. When the ERP fluctuation data exceeds the target threshold range, it is marked as the first target abnormal data and triggers the data abnormality early warning mechanism.

[0040] Compared with the prior art, the present invention has the following beneficial effects: extracting data time series and frequency domain features through sliding window analysis and realizing data periodic fluctuation analysis in combination with support vector machine, realizing linkage strength analysis between different categories of fluctuation data through covariance analysis, and further realizing dynamic threshold adjustment by combining fluctuation analysis and linkage strength analysis results, thereby avoiding interference from normal data fluctuations as a whole and realizing more accurate data anomaly analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 This is a flowchart of an ERP data anomaly analysis method based on big data according to Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the module composition of the big data-based ERP data anomaly analysis system according to Example 2 of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] Example 1: Figure 1 As shown, this embodiment provides an ERP data anomaly analysis method based on big data, and the method includes: S1. Obtain original business data of preset categories from the ERP system, perform data cleaning and denoising on the original business data to obtain standardized business data, compare the standardized business data with a preset initial abnormality determination threshold, and mark the standardized business data outside the initial abnormality determination threshold as potential abnormal data of ERP; the preset categories include sales data, inventory data, procurement data and financial data; the data cleaning and denoising adopt conventional data preprocessing methods in the industry, such as removing duplicate data, filling missing values ​​and correcting abnormal values, and the denoising, such as denoising the data by a moving average method, so as to standardize the data for subsequent analysis.

[0045] S2. Extract the time series characteristics and frequency domain characteristics of ERP potential abnormal data and input them into the fluctuation detection model to perform fluctuation identification on the ERP potential abnormal data to obtain ERP fluctuation data. Perform multi-dimensional linkage analysis on the ERP fluctuation data through covariance analysis to obtain the linkage strength between different categories of ERP fluctuation data.

[0046] S3. According to the linkage strength, the time series characteristics and frequency domain characteristics of the ERP potential abnormal data, the business category target threshold is obtained through the dynamic threshold adjustment model, and the ERP fluctuation data is judged as abnormal according to the business category target threshold. When the ERP fluctuation data exceeds the target threshold range, it is marked as the first target abnormal data and the data abnormality early warning mechanism is triggered. The early warning mechanism includes but is not limited to pop-up window alarms, system notifications, abnormal logs, etc., so as to help managers deal with abnormal situations in a timely manner and avoid unnecessary business losses.

[0047] It should be noted that the method of extracting the time series characteristics and frequency domain characteristics of the ERP potential abnormal data and inputting them into the fluctuation detection model to perform fluctuation identification on the ERP potential abnormal data to obtain the ERP fluctuation data, and performing multi-dimensional linkage analysis on the ERP fluctuation data by the covariance analysis method to obtain the linkage strength between different categories of ERP fluctuation data includes: S21. Extract the time series characteristics of ERP potential abnormal data including business data variation coefficient, business data trend, and business data volatility index, convert the ERP potential abnormal data into frequency domain data through Fourier transform, and extract the frequency domain characteristics of ERP potential abnormal data including business data main frequency intensity, business data frequency concentration, and business data harmonic ratio.

[0048] S22. Summarize the business data variation coefficient, business data trend, business data fluctuation index, business data main frequency intensity, business data frequency concentration, and business data harmonic ratio to obtain a potential abnormal data target feature vector, input the potential abnormal data target feature vector into the fluctuation detection model to identify the fluctuation of the ERP potential abnormal data to obtain a fluctuation category, wherein the fluctuation category includes periodic fluctuations and irregular fluctuations, mark the ERP potential abnormal data whose fluctuation category is irregular fluctuations as the second target abnormal data and trigger the data abnormality early warning mechanism, and mark the ERP potential abnormal data whose fluctuation category is periodic fluctuations as ERP fluctuation data; the fluctuation detection model is a mapping relationship between the potential abnormal data target feature vector and the fluctuation category established by the support vector machine algorithm based on the historical data of the ERP potential abnormal data.

[0049] S23. Perform multidimensional linkage analysis on the ERP fluctuation data through covariance analysis to obtain the linkage strength between different categories of ERP fluctuation data.

[0050] It should be noted that the step S21 also includes sub-steps S211, S212, and S213, which are specifically: S211, according to the preset time window length, obtain the data of the previous preset time window length with the corresponding data at the current time as the end point in the ERP potential abnormal data through the sliding window method to obtain the first target data sequence, and calculate the business data variation coefficient based on the data in the first target data sequence for: .

[0051] in is the first target data sequence data points, is the mean of all data in the first target data sequence, is the number of data in the first target data sequence, is the coefficient of variation of business data.

[0052] For example, set the preset time window length to 7, obtain the first 7 data of the sales quantity ERP potential abnormal data with the corresponding data at the current time as the end point, and obtain the first target data sequence as , then the mean , the business data variation coefficient can be further obtained as: .

[0053] S212: Calculate the business data trend degree based on the data in the first target data sequence for: .

[0054] in is the first target data sequence data points, It is the trend of business data.

[0055] For example: the first target data sequence above The business data trend is .

[0056] S213: Calculate the business data fluctuation index based on the data in the first target data sequence for: .

[0057] in is the first target data sequence The maximum value of the data point, is the first target data sequence The minimum value of the data point, It is the business data volatility index.

[0058] For example: the first target data sequence above middle , , then the business data volatility index .

[0059] It should be noted that the step S21 also includes sub-steps S214, S215, S216, S217, and S218, which are specifically: S214. Expand the first target data sequence to a minimum integer power of 2 of the number of data in the first target data sequence according to the preset time window length by using a zero-filling expansion method to obtain a second target data sequence.

[0060] For example: The first target data sequence is When the preset time window length is 7, the minimum integer power of 2 data greater than 7 according to the zero-filling extension method is 8, that is, the second target data sequence is obtained by expanding to 8 data and filling the end with zeros: .

[0061] S215, performing Fourier transform on the second target data sequence to obtain a spectrum sequence, and calculating the spectrum energy of each spectrum component in the spectrum sequence according to the spectrum sequence for: .

[0062] in is the frequency number and from start, is the first The spectral components corresponding to the frequency numbers are: is the first The real part of the spectral component corresponding to the frequency number, is the first The imaginary part of the spectral component corresponding to the frequency number, It is The spectral energy of the spectral component corresponding to the frequency sequence number.

[0063] The Fourier transform is to perform frequency domain transformation on the second target data sequence according to a frequency domain transformation formula, and the formula is as follows: ; .

[0064] in is the first data, is the frequency number and , is the number of data in the second target data sequence, is an imaginary unit, is the base of natural logarithms, is pi, is the rotation factor, are the spectral components.

[0065] For example: for the second target data sequence Available , when k=0, , when k=1, , when k=2, , we can get by running the calculation with Numpy 745, , , , , , , , we can further get the spectrum energy of each spectrum component , , similarly, , , , , , .

[0066] S216: Summarize the spectrum energy of each spectrum component to obtain a spectrum energy sequence, and calculate the main frequency intensity of the service data according to the spectrum energy sequence for: .

[0067] in is the spectrum energy sequence, is the maximum value in the spectrum energy sequence, is the number of data in the spectral energy sequence, It is the main frequency strength of business data.

[0068] For example: Summarize the spectrum energy of each spectrum component to obtain the spectrum energy sequence: , then the service data main frequency intensity .

[0069] S217, calculating the frequency concentration of service data according to the spectrum energy sequence for: .

[0070] in It is The frequency value of the spectral component corresponding to the frequency sequence number, It is the frequency concentration of business data.

[0071] For example: Calculate the frequency concentration of service data based on the above spectrum energy sequence .

[0072] S218, calculating the harmonic ratio of the service data according to the spectrum energy sequence for: .

[0073] in is the business data harmonic ratio.

[0074] For example: In the above spectrum energy sequence ,but , thus dividing the low-frequency part into , the high frequency part is , further calculate the harmonic ratio of business data .

[0075] It should be noted that the step S23 also includes sub-steps S231, S232, and S233, which are specifically: S231. Summarize the business data variation coefficient, business data trend, business data fluctuation index, business data main frequency intensity, business data frequency concentration, and business data harmonic ratio of different categories of ERP fluctuation data to obtain the business linkage initial feature matrix.

[0076] For example, taking two different categories of ERP fluctuation data as an example, assuming that sales quantity and purchase expenses are identified as ERP fluctuation data, the six characteristic data of sales quantity are , the six characteristic data of procurement cost are , then the initial feature matrix is .

[0077] S232. Calculate the business linkage covariance based on the business linkage initial feature matrix for: .

[0078] in is the first row of data of the initial feature matrix, is the second row of data of the initial feature matrix, is the mean of the first row of data in the initial feature matrix, is the mean of the second row of data in the initial feature matrix, is the number of columns of the initial feature matrix, is the business linkage covariance.

[0079] For example: According to the initial feature matrix, we can get , ,but .

[0080] S233. Get linkage strength based on business linkage covariance for: .

[0081] in is the standard deviation of the first row of data samples in the initial feature matrix, is the standard deviation of the data sample in the second row of the initial feature matrix, Linkage strength.

[0082] For example: In the above example , similarly, , then the linkage strength between sales quantity and purchasing cost , the closer the linkage strength is to The stronger the linkage, the closer , indicating that the linkage is weak.

[0083] It should be noted that the method of obtaining the target threshold of the business category through a dynamic threshold adjustment model according to the linkage strength, the time series characteristics and the frequency domain characteristics of the potential abnormal data of ERP includes: S31. Obtain comprehensive linkage strength based on the linkage strength between ERP fluctuation data and all business categories analyzed in linkage. for: .

[0084] in It is the ERP fluctuation data and The linkage strength of the business category with which the linkage analysis is conducted, is the number of business categories to be analyzed for linkage, It is the comprehensive linkage strength.

[0085] For example: Assume that the sales quantity is linked to three business categories, and the linkage strength with the purchase cost is , and the linkage strength with cash inflow is , and the linkage strength with the inventory quantity is , then the sales quantity comprehensive linkage strength .

[0086] S32, obtain the business data variation coefficient, business data fluctuation index and business data main frequency intensity in the time series characteristics of the ERP potential abnormal data, and modify the initial abnormality judgment threshold according to the business data variation coefficient, business data fluctuation index, business data main frequency intensity and comprehensive linkage intensity to obtain the business category target threshold: ; .

[0087] in is the upper limit of the initial abnormality judgment threshold, is the lower limit of the initial abnormality judgment threshold, is the coefficient of variation of business data, is the business data volatility index, is the main frequency strength of business data, is the natural logarithm function, is the upper threshold value of the business category target, is the lower limit of the target threshold of the business category.

[0088] For example: Assume that the initial abnormality determination threshold of sales quantity is Ten thousand, business data variation coefficient , business data volatility index , business data main frequency intensity , comprehensive linkage strength , then the upper limit of the target threshold of the business category , business category target threshold lower limit , then the target threshold of the business category after dynamic adjustment is .

[0089] It should be noted that, before obtaining the comprehensive linkage strength according to the linkage strength between the ERP fluctuation data and all the business categories analyzed in linkage therewith, the method further includes: S301. Compare the linkage strength of ERP fluctuation data with the preset inter-category linkage strength threshold list. When the linkage strength between the ERP fluctuation data and the business category analyzed in linkage is greater than or equal to the preset inter-category linkage strength threshold, mark the ERP fluctuation data as threshold data to be adjusted. The preset inter-category linkage strength threshold is set based on historical experience and is used to determine whether the corresponding business is at a critical point of normal fluctuation based on the linkage strength between each business category. For example, the inter-category linkage strength threshold of sales quantity and purchase cost is set to 0.65. When the linkage strength between sales data and inventory data is greater than 0.65, it indicates that the linkage between the two is strong and reaches the determination range of normal fluctuation, and is thus marked as normal fluctuation.

[0090] S302. When the linkage strength between the ERP fluctuation data and the business category analyzed in linkage is less than the preset linkage strength threshold between categories, the ERP fluctuation data is marked as abnormal fluctuation and the data abnormality warning mechanism is triggered. When the data is in normal periodic fluctuation, the correlation between businesses is strong and the linkage strength is large. When the ERP fluctuation data has periodic fluctuation but the linkage between businesses is weak, that is, the correlation between businesses is not in line with reality, this periodic fluctuation needs to be judged as abnormal fluctuation to trigger abnormal warning for further verification by personnel.

[0091] It should be noted that the training method of the fluctuation detection model includes: A1. Obtain historical ERP potential abnormal data and extract its time series features and frequency domain features to obtain sample feature vectors. Manually mark the fluctuation categories of the samples to obtain sample labels, including periodic fluctuations and irregular fluctuations. Use the sample feature vectors and sample labels to construct a fluctuation detection sample data set. Use 80% of the sample data set as training data to obtain a fluctuation detection training data set, and use 20% of the sample data set as verification data to obtain a fluctuation detection verification data set.

[0092] A2. Select radial basis kernel function as the kernel function, input the fluctuation detection training data set, fit the penalty factor and kernel function parameters through grid search method, evaluate the classification accuracy of periodic fluctuation and irregular fluctuation under different parameter combinations through k-fold cross validation, and select the parameter combination with the highest classification accuracy as the optimal parameter.

[0093] A3. Verify the model through the verification data set. When the classification accuracy of periodic fluctuations and irregular fluctuations is greater than the preset threshold, the fluctuation detection model is established. Otherwise, return to parameter optimization and retrain the model until the preset performance requirements are met.

[0094] Embodiment 2: Based on the same inventive concept, Figure 2 As shown, this embodiment provides an ERP data anomaly analysis system based on big data, and the system includes: a data processing module, an anomaly analysis module, and a threshold adjustment module, and the modules are connected in sequence.

[0095] The data processing module is used to obtain original business data of preset categories from the ERP system, perform data cleaning and denoising on the original business data to obtain standardized business data, compare the standardized business data with a preset initial abnormality determination threshold, and mark the standardized business data outside the initial abnormality determination threshold as potential abnormal data for ERP.

[0096] The anomaly analysis module is used to extract the time series characteristics and frequency domain characteristics of ERP potential anomaly data and input them into the fluctuation detection model to perform fluctuation identification on the ERP potential anomaly data to obtain ERP fluctuation data, and perform multi-dimensional linkage analysis on the ERP fluctuation data through covariance analysis to obtain the linkage strength between different categories of ERP fluctuation data.

[0097] The threshold adjustment module is used to obtain the business category target threshold through a dynamic threshold adjustment model according to the linkage strength, the time series characteristics and the frequency domain characteristics of the ERP potential abnormal data, and to make an abnormal judgment on the ERP fluctuation data according to the business category target threshold. When the ERP fluctuation data exceeds the target threshold range, it is marked as the first target abnormal data and triggers the data abnormality early warning mechanism.

[0098] It should be noted that, regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in Embodiment 1 of the method, and will not be elaborated here.

[0099] Finally, it should be noted that: Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. The ERP data anomaly analysis method based on big data is characterized by: The method comprises: Obtaining original business data of preset categories from the ERP system, cleaning and denoising the original business data to obtain standardized business data, comparing the standardized business data with the preset initial abnormality determination threshold, and marking the standardized business data outside the initial abnormality determination threshold as potential abnormal data of ERP; The time series characteristics and frequency domain characteristics of ERP potential abnormal data are extracted and input into the fluctuation detection model to identify the fluctuation of ERP potential abnormal data to obtain ERP fluctuation data. The ERP fluctuation data is subjected to multi-dimensional linkage analysis through covariance analysis to obtain the linkage strength between different categories of ERP fluctuation data. According to the linkage strength, the time series characteristics and frequency domain characteristics of ERP potential abnormal data, the business category target threshold is obtained through the dynamic threshold adjustment model. The ERP fluctuation data is judged as abnormal according to the business category target threshold. When the ERP fluctuation data exceeds the target threshold range, it is marked as the first target abnormal data and the data abnormality early warning mechanism is triggered.

2. The ERP data anomaly analysis method based on big data according to claim 1 is characterized in that: The method of extracting the time series characteristics and frequency domain characteristics of the ERP potential abnormal data and inputting them into the fluctuation detection model to perform fluctuation identification on the ERP potential abnormal data to obtain ERP fluctuation data, and performing multi-dimensional linkage analysis on the ERP fluctuation data through the covariance analysis method to obtain the linkage strength between different categories of ERP fluctuation data includes: Extract the time series features of ERP potential abnormal data, including business data variation coefficient, business data trend, and business data volatility index; convert ERP potential abnormal data into frequency domain data through Fourier transform and extract the frequency domain features of ERP potential abnormal data, including business data main frequency intensity, business data frequency concentration, and business data harmonic ratio; The business data variation coefficient, business data trend, business data fluctuation index, business data main frequency intensity, business data frequency concentration, and business data harmonic ratio of the data are summarized to obtain a potential abnormal data target feature vector, and the potential abnormal data target feature vector is input into a fluctuation detection model to perform fluctuation identification on the ERP potential abnormal data to obtain a fluctuation category, wherein the fluctuation category includes periodic fluctuation and irregular fluctuation, and the ERP potential abnormal data whose fluctuation category is irregular fluctuation is marked as the second target abnormal data and triggers the data abnormality early warning mechanism, and the ERP potential abnormal data whose fluctuation category is periodic fluctuation is marked as ERP fluctuation data; the fluctuation detection model is a mapping relationship between the potential abnormal data target feature vector and the fluctuation category established by a support vector machine algorithm based on the historical data of the ERP potential abnormal data; The ERP fluctuation data were subjected to multidimensional linkage analysis through covariance analysis to obtain the linkage strength between different categories of ERP fluctuation data.

3. The ERP data anomaly analysis method based on big data according to claim 2 is characterized in that: The method for extracting the time series characteristics of ERP potential abnormal data including business data variation coefficient, business data trend degree, and business data fluctuation index includes: According to the preset time window length, the sliding window method is used to obtain the data of the previous preset time window length with the corresponding data at the current time as the end point in the ERP potential abnormal data to obtain the first target data sequence, and the business data variation coefficient is calculated based on the data in the first target data sequence. for: ； in is the first target data sequence data points, is the mean of all data in the first target data sequence, is the number of data in the first target data sequence, is the coefficient of variation of business data; The business data trend degree is calculated based on the data in the first target data sequence for: ； in is the first target data sequence data points, It is the trend of business data; The business data fluctuation index is calculated based on the data in the first target data sequence for: ； in is the first target data sequence The maximum value of the data point, is the first target data sequence The minimum value of the data point, It is the business data volatility index.

4. The ERP data anomaly analysis method based on big data according to claim 2 is characterized in that: The method of converting the ERP potential abnormal data into frequency domain data through Fourier transform and extracting the frequency domain features of the ERP potential abnormal data including the main frequency intensity of the business data, the frequency concentration of the business data, and the harmonic ratio of the business data includes: According to the preset time window length, the first target data sequence is expanded by a zero-filling expansion method to a minimum integer power of 2 of the number of data in the first target data sequence to obtain a second target data sequence; Perform Fourier transform on the second target data sequence to obtain a spectrum sequence, and calculate the spectrum energy of each spectrum component in the spectrum sequence according to the spectrum sequence for: ； in is the frequency number and from start, is the first The spectral components corresponding to the frequency numbers are: is the first The real part of the spectral component corresponding to the frequency number, is the first The imaginary part of the spectral component corresponding to the frequency number, It is The spectrum energy of the spectrum component corresponding to the frequency sequence number; Summarize the spectrum energy of each spectrum component to obtain the spectrum energy sequence, and calculate the main frequency intensity of the service data based on the spectrum energy sequence for: ； in is the spectrum energy sequence, is the maximum value in the spectrum energy sequence, is the number of data in the spectral energy sequence, is the main frequency strength of business data; Calculate the frequency concentration of service data based on the spectrum energy sequence for: ； in It is The frequency value of the spectral component corresponding to the frequency sequence number, is the frequency concentration of business data; Calculate the harmonic ratio of service data based on the spectrum energy sequence for: ； in is the business data harmonic ratio.

5. The ERP data anomaly analysis method based on big data according to claim 2 is characterized in that: The method of performing multi-dimensional linkage analysis on the ERP fluctuation data by covariance analysis to obtain linkage strength between different categories of ERP fluctuation data includes: The business data variation coefficient, business data trend, business data fluctuation index, business data main frequency intensity, business data frequency concentration, and business data harmonic ratio of different categories of ERP fluctuation data are summarized to obtain the business linkage initial feature matrix; Calculate the business linkage covariance based on the business linkage initial feature matrix for: ； in is the first row of data of the initial feature matrix, is the second row of data of the initial feature matrix, is the mean of the first row of data in the initial feature matrix, is the mean of the second row of data in the initial feature matrix, is the number of columns of the initial feature matrix, is the business linkage covariance; Get linkage strength based on business linkage covariance for: ； in is the standard deviation of the first row of data samples in the initial feature matrix, is the standard deviation of the data sample in the second row of the initial feature matrix, Linkage strength.

6. The ERP data anomaly analysis method based on big data according to claim 3 is characterized in that: The method for obtaining the target threshold of the business category through a dynamic threshold adjustment model according to the linkage strength, the time series characteristics and the frequency domain characteristics of the ERP potential abnormal data includes: The comprehensive linkage strength is obtained based on the linkage strength between ERP fluctuation data and all business categories analyzed in linkage with it. for: ； in It is the ERP fluctuation data and The linkage strength of the business category with which the linkage analysis is conducted, is the number of business categories to be analyzed for linkage, is the comprehensive linkage strength; The business data variation coefficient, business data fluctuation index and business data main frequency intensity in the time series characteristics of ERP potential abnormal data are obtained. According to the business data variation coefficient, business data fluctuation index, business data main frequency intensity and comprehensive linkage intensity, the initial abnormality judgment threshold is corrected to obtain the business category target threshold: ； ； in is the upper limit of the initial abnormality judgment threshold, is the lower limit of the initial abnormality judgment threshold, is the coefficient of variation of business data, is the business data volatility index, is the main frequency strength of business data, is the natural logarithm function, is the upper threshold value of the business category target, is the lower limit of the target threshold of the business category.

7. The ERP data anomaly analysis method based on big data according to claim 6 is characterized in that: Before obtaining the comprehensive linkage strength according to the linkage strength between the ERP fluctuation data and all the business categories analyzed in linkage therewith, the method further includes: The linkage strength of ERP fluctuation data is compared with the preset linkage strength threshold list between categories. When the linkage strength between ERP fluctuation data and the business category to be analyzed with linkage is greater than or equal to the preset linkage strength threshold between categories, the ERP fluctuation data is marked as threshold data to be adjusted; When the linkage strength between ERP fluctuation data and the business category with which it is linked for analysis is less than the preset inter-category linkage strength threshold, the ERP fluctuation data is marked as abnormal fluctuation and the data abnormality warning mechanism is triggered.

8. The ERP data anomaly analysis method based on big data according to claim 2 is characterized in that: The training method of the fluctuation detection model includes: Obtain historical ERP potential abnormal data and extract its time series features and frequency domain features to obtain sample feature vectors. Manually mark the fluctuation categories of samples to obtain sample labels, including periodic fluctuations and irregular fluctuations. Use sample feature vectors and sample labels to construct a fluctuation detection sample data set. Use 80% of the sample data set as training data to obtain a fluctuation detection training data set, and use 20% of the sample data set as verification data to obtain a fluctuation detection verification data set. The radial basis kernel function is selected as the kernel function, and the fluctuation detection training data set is input. The penalty factor and kernel function parameters are fitted by the grid search method. The classification accuracy of periodic fluctuations and irregular fluctuations under different parameter combinations is evaluated by k-fold cross validation, and the parameter combination with the highest classification accuracy is selected as the optimal parameter. The model is verified through the verification data set. When the classification accuracy of periodic fluctuations and irregular fluctuations is greater than the preset threshold, the fluctuation detection model is established. Otherwise, the parameter optimization is returned to retrain the model until the preset performance requirements are met.

9. An ERP data anomaly analysis system based on big data, used to execute the method according to any one of claims 1 to 8, characterized in that: The system comprises: a data processing module, an abnormality analysis module, and a threshold adjustment module, wherein the modules are connected in sequence; The data processing module is used to obtain original business data of preset categories from the ERP system, perform data cleaning and denoising on the original business data to obtain standardized business data, compare the standardized business data with a preset initial abnormality determination threshold, and mark the standardized business data outside the initial abnormality determination threshold as potential abnormal data of ERP; The abnormality analysis module is used to extract the time series characteristics and frequency domain characteristics of the ERP potential abnormal data and input them into the fluctuation detection model to perform fluctuation identification on the ERP potential abnormal data to obtain ERP fluctuation data, and perform multi-dimensional linkage analysis on the ERP fluctuation data through the covariance analysis method to obtain the linkage strength between different categories of ERP fluctuation data; The threshold adjustment module is used to obtain the business category target threshold through a dynamic threshold adjustment model according to the linkage strength, the time series characteristics and the frequency domain characteristics of the ERP potential abnormal data, and to make an abnormal judgment on the ERP fluctuation data according to the business category target threshold. When the ERP fluctuation data exceeds the target threshold range, it is marked as the first target abnormal data and triggers the data abnormality early warning mechanism.

Images