Abnormal data identification method and device and storage medium

Abstract

The invention discloses an abnormal data identification method and device and a storage medium, which can be applied to various scenes such as cloud technology, artificial intelligence, smart traffic, Internet of Vehicles and the like, and the method comprises the following steps: obtaining a target business data sequence and a real-time business data sequence of a target object; performing spectrum feature extraction on the target service data sequence to obtain a target main frequency feature, a target amplitude distribution feature and a target power spectrum density feature; determining a target spectrum analysis feature corresponding to the target service data sequence; determining a predicted service data sequence of the target object in a second target time period according to the target service data sequence and the target spectrum analysis feature; and if the difference between the first service data in the real-time service data sequence and the second service data in the predicted service data sequence at the same moment is greater than a preset threshold, determining that the first service data is abnormal data. According to the invention, abnormal data can be accurately identified.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to an abnormal data identification method and device and storage medium. BACKGROUND

[0002] The current technology mainly includes basic statistical analysis methods, machine learning algorithms, etc., which are used for processing time series data and performing anomaly detection. These methods are often inefficient when dealing with large-scale, multivariate data, and are not strongly adaptable to new data patterns. The existing technology is difficult to efficiently process large-scale multivariate time series data; when facing data pattern changes, the prediction accuracy of the existing technology decreases. And the existing technology is not good at identifying complex abnormal patterns. SUMMARY

[0003] The present application provides an abnormal data identification method, device and storage medium, which can accurately predict the predicted business data sequence of the target object in the second target period, improve the identification accuracy and efficiency of abnormal data.

[0004] In one aspect, the present application provides an abnormal data identification method, which comprises:

[0005] Obtain the target business data sequence of the target object in the first target period and the real-time business data sequence in the second target period; the first target period is the historical period corresponding to the second target period;

[0006] Perform frequency spectrum feature extraction on the target business data sequence to obtain target main frequency features, target amplitude distribution features, and target power spectrum density features;

[0007] Determine the target spectrum analysis features corresponding to the target business data sequence based on the target main frequency features, target amplitude distribution features, and target power spectrum density features, wherein the target spectrum analysis features include main periodicity features, main frequency band features, and main energy band features;

[0008] Determine the predicted business data sequence of the target object in the second target period according to the target business data sequence and the target spectrum analysis features;

[0009] Obtain the first business data in the real-time business data sequence and the second business data in the predicted business data sequence at the same time;

[0010] If the difference between the first business data and the second business data is greater than a preset threshold, determine that the first business data is abnormal data.

[0011] Another aspect provides an abnormal data identification device, which comprises:

[0012] The first data acquisition module is configured to acquire a target service data sequence of a target object in a first target period and a real-time service data sequence in a second target period; the first target period is a historical period corresponding to the second target period;

[0013] The spectrum feature extraction module is configured to perform spectrum feature extraction on the target service data sequence to obtain a target main frequency feature, a target amplitude distribution feature, and a target power spectral density feature.

[0014] The spectrum analysis module is configured to determine a target spectrum analysis feature corresponding to the target service data sequence based on the target main frequency feature, the target amplitude distribution feature, and the target power spectral density feature; the target spectrum analysis feature includes a main periodicity feature, a main frequency band feature, and a main energy band feature.

[0015] The prediction data determination module is configured to determine a predicted service data sequence of the target object in the second target period according to the target service data sequence and the target spectrum analysis feature.

[0016] The second data acquisition module is configured to acquire first service data in the real-time service data sequence and second service data in the predicted service data sequence at the same time.

[0017] The abnormal data determination module is configured to determine the first service data as abnormal data if a difference between the first service data and the second service data is greater than a preset threshold.

[0018] In an example embodiment, the spectrum analysis module includes:

[0019] The first analysis unit is configured to analyze the target main frequency feature to obtain a main periodicity feature corresponding to the target service data sequence.

[0020] The second analysis unit is configured to analyze the target amplitude distribution feature to obtain a main frequency band feature corresponding to a main energy.

[0021] The third analysis unit is configured to analyze the target power spectral density feature to obtain the main energy band feature.

[0022] The feature determination unit is configured to determine a target spectrum analysis feature corresponding to the target service data sequence based on the main periodicity feature, the main frequency band feature, and the main energy band feature.

[0023] In an example embodiment, the first analysis unit includes:

[0024] a feature analysis subunit, configured to analyze the target main frequency feature to obtain a target main cycle feature and a target seasonal cycle feature;

[0025] a feature determination subunit, configured to determine the target main cycle feature and the target seasonal cycle feature as a main periodicity feature corresponding to the target service data sequence.

[0026] In an example implementation, the spectrum feature extraction module comprises:

[0027] a target processing unit, configured to perform data preprocessing on the target service data sequence to obtain target processing data;

[0028] a data switching unit, configured to convert the target processing data from a time domain to a frequency domain through Fourier transform to obtain target time domain data;

[0029] a main frequency identification unit, configured to identify a main frequency component in the target time domain data to obtain the target main frequency feature;

[0030] an amplitude identification unit, configured to identify the target amplitude distribution feature corresponding to the target time domain data; the target amplitude distribution feature represents amplitude distribution of each frequency component in the target time domain data;

[0031] a power feature determination unit, configured to determine energy distribution of the target time domain data in the frequency domain to obtain the target power spectral density feature.

[0032] In an example implementation, the apparatus further comprises:

[0033] a phase acquisition module, configured to acquire target phase information of the target time domain data;

[0034] In an example implementation, the data switching unit comprises:

[0035] an initial data determination subunit, configured to convert the target processing data from the time domain to the frequency domain through Fourier transform to obtain initial time domain data;

[0036] a target data determination subunit, configured to remove noise information in the initial time domain data based on the target phase information to obtain the target time domain data.

[0037] In an example implementation, the prediction data determination module comprises:

[0038] a model acquisition unit, configured to acquire a data prediction model; the data prediction model is obtained by training a preset model based on a sample service data sequence of a sample object;

[0039] The prediction unit is used to input the target service data sequence and the target spectrum analysis feature into a data prediction model to perform data prediction, so as to obtain the predicted service data sequence of the target object in the second target time period.

[0040] In an exemplary embodiment, the apparatus further comprises:

[0041] a sample sequence acquisition module, configured to acquire a sample business data sequence of the sample object in a first preset time period; the sample business data sequence is labeled with a sample real-time data sequence tag; the sample real-time data sequence tag is a real-time business data sequence of the sample object in a second preset time period; the first preset time period is a historical time period corresponding to the second preset time period;

[0042] An analytical feature determination module, configured to determine a sample spectrum analytical feature corresponding to the sample service data sequence, wherein the sample spectrum analytical feature includes a sample main periodicity feature, a sample main frequency band feature, and a sample main energy band feature;

[0043] A sample real-time result determination module is used to input the sample business data sequence and the sample spectrum analysis characteristics into a preset model for data prediction processing to obtain a sample real-time data sequence result;

[0044] The training module is used to train the preset model based on the difference between the sample real-time data sequence result and the sample real-time data sequence label to obtain the data prediction model.

[0045] In an exemplary embodiment, the analytical feature determination module includes:

[0046] A spectrum feature extraction unit is used to extract spectrum features of the sample service data sequence to obtain sample main frequency features, sample amplitude distribution features and sample power spectrum density features;

[0047] an analytical feature determination unit, configured to determine a sample spectrum analytical feature based on the sample main frequency feature;

[0048] a sample main period determination unit, configured to determine a sample main periodicity feature based on the sample power spectrum density feature;

[0049] a sample main energy determination unit, configured to determine a sample main energy band feature based on the sample power spectrum density feature;

[0050] The analytical feature determination unit is used to determine the sample spectrum analytical feature, the sample main periodicity feature and the sample main energy band feature as the sample spectrum analytical feature corresponding to the sample business data sequence.

[0051] In an example implementation, the training module comprises:

[0052] a target loss determination unit configured to determine target loss data based on a difference between the sample real-time data sequence result and the sample real-time data sequence label;

[0053] an initial model determination unit configured to adjust model parameters of the preset model based on the target loss data to obtain an initial data prediction model;

[0054] a test sequence acquisition unit configured to acquire a test service data sequence of a test object;

[0055] a parameter determination unit configured to test a performance parameter of the initial data prediction model based on the test service data sequence;

[0056] a model determination unit configured to determine the initial data prediction model as the data prediction model if the performance parameter of the initial data prediction model meets a preset condition.

[0057] In an example implementation, the parameter determination unit comprises:

[0058] a data test subunit configured to test an accuracy, a recall rate and an F1 score of the initial data prediction model based on the test service data sequence;

[0059] a weight determination subunit configured to determine a first weight corresponding to the accuracy, a second weight corresponding to the recall rate and a third weight corresponding to the F1 score;

[0060] a parameter determination subunit configured to determine the performance parameter of the initial data prediction model based on a product of the accuracy and the first weight, a product of the recall rate and the second weight, and a product of the F1 score and the third weight.

[0061] In an example implementation, the apparatus further comprises:

[0062] a context acquisition module configured to acquire context information corresponding to the abnormal data;

[0063] an abnormal reason determination module configured to determine an abnormal reason corresponding to the abnormal data based on the context information of the abnormal data;

[0064] a strategy construction module configured to construct an abnormal processing strategy based on the abnormal reason.

[0065] Another aspect provides an electronic device, the device comprising a processor and a memory having stored therein at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the abnormal data identification method as described above.

[0066] Another aspect provides a computer storage medium having stored therein at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by a processor to implement the abnormal data identification method as described above.

[0067] Another aspect provides a computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform to implement the abnormal data identification method as described above.

[0068] The abnormal data identification method, device and storage medium provided by the present application have the following technical effects:

[0069] The present application obtains a target service data sequence of a target object in a first target period and a real-time service data sequence in a second target period; the first target period is a historical period corresponding to the second target period; a target main frequency feature, a target amplitude distribution feature and a target power spectral density feature are obtained by performing frequency spectrum feature extraction on the target service data sequence; a target spectrum analysis feature corresponding to the target service data sequence is determined based on the target main frequency feature, the target amplitude distribution feature and the target power spectral density feature, and the target spectrum analysis feature includes a main periodicity feature, a main frequency band feature and a main energy band feature; a predicted service data sequence of the target object in the second target period is determined according to the target service data sequence and the target spectrum analysis feature; in the data prediction process, the spectrum analysis feature is introduced, so that the dynamic change of the data can be more accurately captured and predicted; the present application predicts the predicted service data sequence of the target object in the second target period by combining the target service data sequence and the target spectrum analysis feature, thereby improving the accuracy of the predicted service data sequence; a first service data in the real-time service data sequence and a second service data in the predicted service data sequence at the same time are obtained; if the difference between the first service data and the second service data is greater than a preset threshold, the first service data is determined as abnormal data, so that the abnormal data can be accurately identified by combining the target service data sequence and the target spectrum analysis feature, and the false positives and false negatives of the abnormal data are reduced. BRIEF DESCRIPTION OF DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present specification or the prior art and the advantages, the drawings required to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0071] Figure 1 is a schematic diagram of an abnormal data identification system provided by an embodiment of the present specification;

[0072] Figure 2 is a flowchart of an abnormal data identification method provided by an embodiment of the present specification;

[0073] Figure 3 is a flowchart of a method for extracting the target frequency spectrum feature, target amplitude distribution feature and target power spectral density feature from the target service data sequence provided by an embodiment of the present specification;

[0074] Figure 4 is a flowchart of a method for determining the target frequency spectrum analysis feature corresponding to the target service data sequence provided by an embodiment of the present specification;

[0075] Figure 5 is a flowchart of a training method of a data prediction model provided by an embodiment of the present specification;

[0076] Figure 6 is a flowchart of a method for determining the sample frequency spectrum analysis feature corresponding to the sample service data sequence provided by an embodiment of the present specification;

[0077] Figure 7 is a flowchart of a method for training the preset model based on the difference between the sample real-time data sequence result and the sample real-time data sequence label, and obtaining the data prediction model provided by an embodiment of the present specification;

[0078] Figure 8 is a structural diagram of an abnormal data identification device provided by an embodiment of the present specification;

[0079] Figure 9 is a structural diagram of a server provided by an embodiment of the present specification. DETAILED DESCRIPTION

[0080] With reference to the drawings of the embodiments of the present specification, the technical solutions in the embodiments of the present specification will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work belong to the scope of protection of the present application.

[0081] It can be understood that in the specific embodiments of the present application, data related to user information, service data sequences and the like are involved. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent needs to be obtained, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of countries and regions.

[0082] Artificial intelligence (AI) is to use digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which tries to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machine has the functions of perception, reasoning and decision-making.

[0083] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or server including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0084] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works with other related parts to achieve a predetermined target, and can be implemented entirely or partially by using software, hardware (such as processing circuit or memory) or combination thereof. Similarly, one processor (or multiple processors or memory) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an integral module or unit that includes the functions of the module or unit.

[0085] Artificial intelligence technology is a comprehensive discipline involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include, for example, sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-training model technology, operation / interaction systems, mechatronics, etc. Among them, the pre-training model, also known as the large model or the basic model, can be widely applied to downstream tasks in various directions of artificial intelligence after fine-tuning. Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0086] Machine learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, etc. It is a subject that studies how computers simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structure to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent, and its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning generally include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and inductive learning, etc. Pre-training models are the latest developments in deep learning, integrating the above technologies.

[0087] With the research and progress of artificial intelligence technology, artificial intelligence technology has been researched and applied in many fields, such as common smart home, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned vehicles, autonomous vehicles, drones, digital twins, virtual humans, robots, artificial intelligence generated content (AIGC), conversational interaction, intelligent medical care, intelligent customer service, game AI, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0088] The scheme provided by the embodiments of the present application relates to machine learning and other technologies of artificial intelligence, which is specifically explained by the following embodiments.

[0089] Please refer to Figure 1 , Figure 1 is a schematic diagram of an abnormal data identification system provided by an embodiment of the present application, as Figure 1 shown, the abnormal data identification system can at least include a server 01 and a client 02.

[0090] Specifically, in the embodiments of the present specification, the server 01 can include a standalone server, or a distributed server, or a server cluster composed of multiple servers, and can also be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms. The server 01 can include a network communication unit, a processor, a memory, and the like. Specifically, the server 01 can be configured to determine a target spectrum analysis feature corresponding to the target service data sequence based on the target main frequency feature, the target amplitude distribution feature, and the target power spectral density feature, the target spectrum analysis feature including a main periodicity feature, a main frequency band feature, and a main energy band feature; and determine a predicted service data sequence of the target object in the second target period based on the target service data sequence and the target spectrum analysis feature.

[0091] Specifically, in the embodiments of the present specification, the client 02 can include a smart phone, a desktop computer, a tablet computer, a notebook computer, a digital assistant, a smart wearable device, a smart speaker, a vehicle-mounted terminal, a smart television, and the like. The client 02 can also include software running in an entity device, such as a web page provided by a service provider to a user, and an application provided by the service provider to the user. Specifically, the client 02 can be configured to query abnormal data in the real-time service data sequence.

[0092] The embodiments of the present application can be applied to various scenes such as cloud technology, artificial intelligence, intelligent transportation, and assisted driving. The following describes an abnormal data identification method according to the present application, Figure 2 is a flowchart of an abnormal data identification method according to an embodiment of the present specification. The present specification provides method operation steps as described in the embodiments or flowcharts, but more or fewer operation steps can be included based on conventional or non-creative labor. The order of steps listed in the embodiments is only one of the many execution orders, and does not represent the only execution order. In actual system or server product execution, the method order shown in the embodiments or the drawings can be executed in sequence or in parallel (for example, in a parallel processor or multi-threaded processing environment). Specifically, as shown in Figure 2 The above method can include:

[0093] S201: Obtain a target service data sequence of a target object in a first target period and a real-time service data sequence in a second target period. The first target period is a historical period corresponding to the second target period.

[0094] In the embodiments of the present specification, the target object can include but is not limited to a user, a merchant, a store, etc. in a business platform. The target business data sequence and the real-time business data sequence are time series data of the same type, and the target business data sequence and the real-time business data sequence are both business data corresponding to the target object. The business data of the target object can include but is not limited to the sales volume, the visitor volume, the resource transfer data (transaction amount), etc. of the electronic store of the target object. The first target period is a historical period corresponding to the second target period; for example, the first target period can be the past three months, and the second target period can be the current month, so that the predicted business data of the current month can be predicted according to the business data of the past three months, and the predicted business data is compared with the real-time business data of the current month, so as to identify the abnormal data.

[0095] S203: performing spectrum feature extraction on the target business data sequence to obtain a target main frequency feature, a target amplitude distribution feature, and a target power spectral density feature.

[0096] In the embodiments of the present specification, the spectrum features in the target business data sequence can be extracted to obtain the target main frequency feature, the target amplitude distribution feature, and the target power spectral density feature.

[0097] In the embodiments of the present specification, as shown in Figure 3 the spectrum feature extraction on the target business data sequence to obtain the target main frequency feature, the target amplitude distribution feature, and the target power spectral density feature includes:

[0098] S2031: performing data preprocessing on the target business data sequence to obtain target processed data.

[0099] In some embodiments, the data preprocessing on the target business data sequence to obtain the target processed data can include: processing missing values and abnormal values in the target business data sequence to obtain initial processed data; and performing standardization and normalization processing on the target processed data to obtain the target processed data.

[0100] In the embodiments of the present specification, data preprocessing is a key step for any data analysis and model construction. It involves converting raw multivariate time series data into a clean, standardized format, making it suitable for spectrum analysis and subsequent model training. The following is a detailed description of data preprocessing:

[0101] (1) Data cleaning:

[0102] Missing value processing: identify missing values in the data and decide how to handle them. Common processing methods include deleting rows containing missing values, filling missing values with average or median, etc.

[0103] Outlier treatment: Detect and handle outliers, which can be caused by entry errors or other factors. A common method is to calculate the Z-score, and then delete data points that exceed a certain threshold, the Z-score formula is as follows:

[0104] Z = σ (X - μ)

[0105] Where X is the observation value, μ and σ are the mean and standard deviation of the data respectively.

[0106] (2) Data transformation:

[0107] Standardization: Convert data into a format with zero mean and unit variance, which is necessary for many algorithms, especially distance-based algorithms. The standardization formula is as follows:

[0108]

[0109] Where, is the sample mean, s is the sample standard deviation.

[0110] Normalization: Scale data to a specific range (usually 0 to 1). This is useful for features of different magnitudes, the normalization formula is as follows:

[0111]

[0112] Input: Target business data sequence, which may contain various types of business indicators such as sales volume, visitor volume, transaction amount, etc.

[0113] Output: Cleaned, standardized or normalized data set, ready for further spectral analysis and prediction model construction.

[0114] Data preprocessing steps need to be adjusted according to specific data sets and business scenarios. For example, for some time series data, seasonal adjustment or time smoothing may need to be considered. The treatment strategy for outliers and missing values should be determined according to the nature of the data and the analysis target. In some cases, directly deleting outliers may not be the best choice, as this may result in the loss of important information. In this embodiment, data processing can improve the accuracy of target processing data.

[0115] S2033: Convert the above target processing data from time domain to frequency domain by Fourier transform to obtain target time domain data;

[0116] In some embodiments, the above method further comprises:

[0117] Obtain the target phase information of the above target time domain data;

[0118] For example, the target processing data is converted from the time domain to the frequency domain by Fourier transform to obtain target time domain data, including:

[0119] The target processing data is converted from the time domain to the frequency domain by Fourier transform to obtain initial time domain data;

[0120] Based on the target phase information, the noise information in the initial time domain data is removed to obtain the target time domain data.

[0121] In the embodiments of the present specification, the noise can be determined through the target phase information of the target time domain data, and the noise information in the initial time domain data is removed to obtain the target time domain data, thereby improving the accuracy of the target time domain data.

[0122] In the embodiments of the present specification, in the spectral analysis, the following aspects can be focused on:

[0123] a. Main frequency components: find the frequency components with higher amplitudes, which usually represent the main periodic characteristics in the signal. For example, in meteorological data, there may be obvious annual and seasonal cycles; in economic data, there may be monthly, quarterly, annual, etc. b. Amplitude: analyze the amplitudes of each frequency component to understand their importance in the signal. Frequency components with larger amplitudes contribute more to the signal and reflect the main characteristics of the signal. c. Power spectral density (PSD): calculate the power spectral density of each frequency component to understand the energy distribution of the signal at each frequency. PSD can help identify significant features and potential laws in the signal. d. Phase information: in some application scenarios, the phase information of the frequency component is also important. For example, in signal reconstruction or denoising, the phase information needs to be preserved to ensure that the restored signal is as close as possible to the original signal.

[0124] S2035: Identify the main frequency components in the target time domain data to obtain the target main frequency characteristics;

[0125] S2037: Identify the target amplitude distribution characteristics corresponding to the target time domain data; the target amplitude distribution characteristics represent the amplitude distribution of each frequency component in the target time domain data;

[0126] S2039: Determine the energy distribution of the target time domain data in the frequency domain to obtain the target power spectral density characteristics.

[0127] In the embodiments of the present specification, the frequency component refers to each component of a signal in the frequency domain. A signal can be regarded as being composed of sine waves of different frequencies, which are the frequency components of the signal. Common frequency components include the fundamental frequency (the lowest frequency in the signal) and harmonics (integer multiples of the fundamental frequency). The frequency amplitude refers to the size of each frequency component of a signal in the frequency domain. In signal analysis, the Fourier Transform is usually used to convert the time-domain signal into the frequency-domain signal to obtain the amplitude of each frequency component. The amplitude reflects the importance of the frequency component in the entire signal. The larger the amplitude, the greater the contribution of the frequency component to the signal. For example, in audio processing, the frequency amplitude can be used to reflect the strength of the tone. When performing spectral analysis on time series data, the time-domain signal can be converted into the frequency-domain signal by Fourier Transform or other frequency-domain analysis methods (such as short-time Fourier Transform, wavelet transform, etc.). In the frequency domain, each frequency component and its corresponding amplitude can be observed. By analyzing these frequency components and amplitudes, the spectral features of the signal can be extracted. For example, the frequency components with the largest amplitudes, i.e., the main frequency components, can be found, which may represent the main periodicity of the signal; the power spectral density (PSD) can also be calculated, which describes the energy distribution of the signal at each frequency, and helps to better understand the spectral characteristics of the signal.

[0128] S205: Based on the target main frequency feature, the target amplitude distribution feature, and the target power spectral density feature, determine the target spectral analysis feature corresponding to the target service data sequence, wherein the target spectral analysis feature includes a main periodicity feature, a main frequency band feature, and a main energy band feature.

[0129] In the embodiments of the present specification, as shown in Figure 4 Based on the target main frequency feature, the target amplitude distribution feature, and the target power spectral density feature, determining the target spectral analysis feature corresponding to the target service data sequence includes:

[0130] S2051: Analyzing the target main frequency feature to obtain the main periodicity feature corresponding to the target service data sequence.

[0131] In the embodiments of the present specification, analyzing the target main frequency feature to obtain the main periodicity feature corresponding to the target service data sequence includes:

[0132] Analyzing the target main frequency feature to obtain the target main cycle feature and the target seasonal cycle feature.

[0133] The target main cycle feature and the target seasonal cycle feature are determined as the main periodicity feature corresponding to the target business data sequence.

[0134] In the embodiments of the present specification, the target main frequency feature can be analyzed to obtain the target main cycle feature and the target seasonal cycle feature, thereby obtaining the main periodicity feature corresponding to the target business data sequence. The main period (corresponding to the target main cycle feature) and the seasonal pattern (corresponding to the target seasonal cycle feature) are important features in time series data, which reflect the regular changes of data within a certain time range. The main period refers to the main cycle pattern in time series data. For example, in the stock market, there may be a certain cycle of rise and fall; in the field of biology, the circadian rhythm is a typical main period phenomenon. The seasonal pattern refers to the regular change of time series data within a year. Such changes are usually related to seasonal changes, such as temperature, rainfall, and the impact of holidays on sales.

[0135] The following illustrates how to analyze the main period and the seasonal pattern:

[0136] Suppose there is a set of daily sales time series data, and we want to analyze the main period and the seasonal pattern of this set of data. First, the data can be preprocessed, such as removing trends and outliers. Then, Fourier transform can be used to convert time domain data to frequency domain data to obtain a frequency spectrum. On the frequency spectrum, different frequency components and their corresponding amplitudes can be observed. If a certain frequency component has a large amplitude, it may correspond to the main period of the data. For example, if this frequency component corresponds to a period of 30 days, it can be inferred that there may be a monthly cycle in the data.

[0137] S2053: Analyzing the target amplitude distribution feature to obtain a main frequency band feature corresponding to the main energy;

[0138] S2055: Analyzing the target power spectral density feature to obtain the main energy band feature;

[0139] S2057: Based on the main periodicity feature, the main frequency band feature, and the main energy band feature, determining the target spectrum analysis feature corresponding to the target business data sequence.

[0140] In the embodiments of the present specification, the analysis of spectral features is mainly carried out by observing and analyzing the frequency spectrum. The frequency spectrum is obtained after converting the signal from the time domain to the frequency domain, which shows the strength (amplitude or energy) of the signal at each frequency. On the frequency spectrum, we can see the frequency components and their corresponding amplitudes, and by analyzing this information, we can extract the spectral features of the signal. The main frequency band feature is used to represent the frequency band corresponding to the main energy as low or high frequency; we can find the frequency components with higher amplitude, which usually represent the main periodic characteristics in the signal. For example, in meteorological data, there may be obvious annual and seasonal cycles; in economic data, there may be monthly, quarterly, annual, etc. periodicity. We can analyze the amplitudes of each frequency component to understand their importance in the signal. The frequency components with larger amplitudes contribute more to the signal, reflecting the main characteristics of the signal. We can also calculate the power spectral density of each frequency component to understand the energy distribution of the signal at each frequency. PSD can help identify significant features and potential patterns in the signal.

[0141] In the embodiments of the present specification, on the frequency spectrum, the frequency points with larger amplitudes usually represent the main frequency components of the signal. These frequency components correspond to the periodic characteristics that may be important information in the signal. For example, when processing audio signals, the main frequency components may correspond to the pitch of the music; when processing economic data, the main frequency components may correspond to the periodic fluctuations of economic activity.

[0142] Analyze the amplitude distribution: In addition to the main frequency components, the amplitude distribution of other frequency components is also important. The amplitude distribution can reflect the energy distribution of the signal in the frequency domain, helping to understand the overall characteristics of the signal. For example, if most of the energy of a signal is concentrated in the low frequency region on the frequency spectrum, it may mean that the signal is mainly composed of slow-changing components.

[0143] Calculate the power spectral density: Power spectral density (Power Spectral Density, PSD) is an important indicator to describe the energy distribution of the signal in the frequency domain. PSD can help understand the spectral characteristics of the signal in more detail, such as finding the main energy band of the signal, identifying noise in the signal, etc.

[0144] For different types of spectral features, such as main frequency components, amplitude distribution and PSD, etc., they need to be analyzed respectively. This is because these features reflect different aspects of information about the signal, and by comprehensively analyzing these features, we can more comprehensively understand the spectral characteristics of the signal.

[0145] In some embodiments, the amplitude of each frequency component corresponding to the target object can also be determined based on the above-mentioned target amplitude distribution characteristics; the frequency components with amplitudes greater than a preset amplitude threshold are obtained to obtain target frequency components; and the period corresponding to the target frequency components is determined as the main periodic characteristic. The effectiveness of spectral analysis is highly dependent on the quality of the data and the accuracy of the preprocessing. Special attention should be paid when analyzing the spectral characteristics, because some frequency components may be caused by noise or non-periodic changes. In some cases, a window function may need to be applied to reduce edge effects, especially when dealing with non-stationary time series data.

[0146] S207: Determine the predicted business data sequence of the target object in the second target period according to the target business data sequence and the target spectral analysis characteristics.

[0147] In the embodiments of the present specification, determining the predicted business data sequence of the target object in the second target period according to the target business data sequence and the target spectral analysis characteristics comprises:

[0148] Obtaining a data prediction model; the data prediction model is obtained by training a preset model based on a sample business data sequence of a sample object;

[0149] Inputting the target business data sequence and the target spectral analysis characteristics into the data prediction model for data prediction to obtain the predicted business data sequence of the target object in the second target period.

[0150] In the embodiments of the present specification, the data prediction model for predicting the business data sequence can be obtained by pre-training, and then the target business data sequence and the target spectral analysis characteristics are input into the data prediction model for data prediction, so that the predicted business data sequence of the target object in the second target period can be quickly and accurately obtained.

[0151] For example, the target business data sequence and the target spectral analysis characteristics can be first processed by a principal component analysis method to select features and reduce dimensions, to obtain target input data; and then the target input data is input into the data prediction model for data prediction. Principal component analysis (PCA) is used to explore the correlation between multiple possible related variables (such as the relationship between weight and height), find the maximum or minimum correlation direction, and achieve data compression or denoising purposes (dimensionality reduction). For a data set X, PCA aims to find a linear transformation Y=XP, where P is the principal component extracted from the covariance matrix of the data. The principal component analysis method can reduce the number of features and improve the data processing efficiency of the model.

[0152] In some embodiments, asFigure 5 As shown, the training method of the above data prediction model includes:

[0153] S501: Obtain a sample business data sequence of the sample object in a first preset time period; the sample business data sequence is labeled with a sample real-time data sequence label; the sample real-time data sequence label is a real-time business data sequence of the sample object in a second preset time period; the first preset time period is a historical period corresponding to the second preset time period;

[0154] S503: Determine sample spectrum analysis features corresponding to the sample service data sequence, where the sample spectrum analysis features include sample main periodicity features, sample main frequency band features, and sample main energy band features;

[0155] S505: Inputting the sample service data sequence and the sample spectrum analysis features into a preset model for data prediction processing to obtain a sample real-time data sequence result;

[0156] S507: Based on the difference between the sample real-time data sequence result and the sample real-time data sequence label, the preset model is trained to obtain the data prediction model.

[0157] In the embodiment of this specification, the sample object and the target object may be objects of the same type, and the sample business data sequence, the sample real-time data sequence, the target business data sequence, and the real-time business data sequence are data of the same type.

[0158] In the embodiments of this specification, the spectrum feature extraction and analysis of the sample business data sequence are similar to the spectrum feature extraction and analysis of the target business data sequence; the sample main periodicity feature and the main periodicity feature in the target spectrum analysis feature are the same type of features, the sample main frequency band feature and the main frequency band feature in the target spectrum analysis feature are the same type of features, and the sample main energy band feature and the main energy band feature in the target spectrum analysis feature are the same type of features.

[0159] In an embodiment of the present specification, a preset model can be trained based on a sample business data sequence of a sample object in a first preset time period to obtain a data prediction model; wherein the preset model can select a suitable model based on data characteristics, such as a random forest, a support vector machine (SVM), a neural network, etc.

[0160] In the embodiments of this specification, Figure 6 As shown, determining the sample spectrum analysis features corresponding to the sample service data sequence includes:

[0161] S5031: Extract spectrum features from the sample service data sequence to obtain sample main frequency features, sample amplitude distribution features, and sample power spectrum density features;

[0162] In the embodiments of the present specification, the main frequency component in the sample time domain data can be identified to obtain the sample main frequency feature; the sample amplitude distribution feature corresponding to the sample time domain data is identified; the sample amplitude distribution feature represents the amplitude distribution of each frequency component in the sample time domain data; the energy distribution of the sample time domain data in the frequency domain is determined to obtain the sample power spectral density feature.

[0163] S5033: Determine the sample spectrum analysis feature based on the sample main frequency feature;

[0164] S5035: Determine the sample main periodicity feature based on the sample power spectral density feature;

[0165] S5037: Determine the sample main energy band feature based on the sample power spectral density feature;

[0166] S5039: Determine the sample spectrum analysis feature, the sample main periodicity feature and the sample main energy band feature as the sample spectrum analysis feature corresponding to the sample service data sequence.

[0167] In the embodiments of the present specification, the spectrum feature extraction method of the sample service data sequence is similar to the spectrum feature extraction method of the target service data sequence. The sample main frequency feature can be analyzed to determine the sample spectrum analysis feature; the sample power spectral density feature is determined to determine the sample main periodicity feature; the sample power spectral density feature is determined to determine the sample main energy band feature; and then the sample spectrum analysis feature, the sample main periodicity feature and the sample main energy band feature are determined as the sample spectrum analysis feature corresponding to the sample service data sequence.

[0168] In the embodiments of the present specification, the sample service data sequence and the sample spectrum analysis feature are input into a preset model for data prediction processing to obtain a sample real-time data sequence result, including:

[0169] Based on the principal component analysis method, the sample service data sequence and the sample spectrum analysis feature are subjected to feature selection and feature dimension reduction processing to obtain sample input data;

[0170] The sample input data is input into a preset model for data prediction processing to obtain a sample real-time data sequence result.

[0171] In the embodiments of the present specification, principal component analysis (PCA) is used to explore the correlation between multiple possible relevant variables (such as the relationship between weight and height), find the maximum or minimum correlation direction, and achieve the purpose of data compression or denoising (dimensionality reduction). For a data set X, PCA aims to find a linear transformation Y = XP, where P is the principal component extracted from the covariance matrix of the data. The number of features can be reduced by the principal component analysis method, and the training efficiency of the model can be improved.

[0172] In the embodiments of the present specification, as shown in Figure 7 the preset model is trained based on the difference between the sample real-time data sequence result and the sample real-time data sequence label, and the data prediction model is obtained, including:

[0173] S5071: Determine the target loss data based on the difference between the sample real-time data sequence result and the sample real-time data sequence label;

[0174] S5073: Adjust the model parameters of the preset model based on the target loss data to obtain an initial data prediction model;

[0175] In the embodiments of the present specification, the model parameters of the preset model can be adjusted according to the target loss data until the training end condition is met, and the preset model at the end of training is determined as the initial data prediction model. The training end condition can be that the target loss data is less than a preset loss threshold, or the number of training iterations reaches a preset number.

[0176] In the model training process, hyperparameter tuning can be performed by methods such as grid search, random search, or Bayesian optimization to find the best parameter combination. Hyperparameter tuning is an important task during model training, and its goal is to find a set of hyperparameters that make the model perform best on the validation set. Common hyperparameter tuning methods include grid search, random search, and Bayesian optimization.

[0177] Grid search (Grid Search): This is the most direct parameter search method, which finds the best parameters by traversing all possible parameter combinations. For example, if there are two parameters to be adjusted, each parameter has 3 possible values, then 3*3 = 9 models need to be trained, and their performance needs to be compared. The advantage of grid search is that it can exhaust all possible parameter combinations, but the disadvantage is that the calculation is large and the efficiency is low.

[0178] Random Search: Random search is a more efficient method of parameter search. Unlike grid search, which exhausts all possible combinations of parameters, random search randomly samples a portion of parameter combinations from the parameter space for training. Although random search cannot guarantee finding the optimal parameters, in practice, it often finds better parameters in a shorter time.

[0179] Bayesian Optimization: Bayesian optimization is a more advanced method of parameter search, which optimizes by building a probabilistic model of the objective function (i.e., model performance). At each step of optimization, it selects the parameters that are most likely to improve model performance for the next step of training. The advantage of Bayesian optimization is that it can find better parameters in fewer steps, but the disadvantage is that it is complex to implement and computationally intensive.

[0180] When performing hyperparameter tuning, the data is usually divided into training set, validation set and test set. The training set is used to train the model, the validation set is used for hyperparameter tuning, and the test set is used to evaluate the performance of the final model.

[0181] In some embodiments, after the training of the initial data prediction model is completed, the validation business data sequence set can be obtained, and the validation business data sequences in the validation business data sequence set can be divided into multiple data subsets; then the model is cross-validated according to the multiple data subsets, so as to obtain the performance parameters of the initial data prediction model on the multiple data subsets.

[0182] S5075: Obtain a test business data sequence of a test object;

[0183] S5077: Test the performance parameters of the initial data prediction model based on the test business data sequence.

[0184] S5079: If the performance parameters of the initial data prediction model meet the preset conditions, the initial data prediction model is determined as the data prediction model.

[0185] In the embodiments of the present application, a test service data sequence of a test object in a first period can be obtained; the test service data sequence is labeled with a test real-time data sequence label; the test real-time data sequence label is a real-time service data sequence of the test object in a second period; the first period is a historical period corresponding to the second period; the test service data sequence is input into the initial data prediction model to obtain a test real-time data sequence result; and a performance parameter of the initial data prediction model is determined based on the difference between the test real-time data sequence result and the test real-time data sequence label. The performance parameter includes but is not limited to the accuracy, recall rate and F1 score of the model. The preset condition can be determined according to a preset performance parameter threshold, for example, the preset condition can be that the performance parameter is less than the preset performance parameter threshold.

[0186] In some embodiments, testing the performance parameter of the initial data prediction model based on the test service data sequence includes:

[0187] testing the accuracy, recall rate and F1 score of the initial data prediction model based on the test service data sequence;

[0188] determining a first weight corresponding to the accuracy, a second weight corresponding to the recall rate and a third weight corresponding to the F1 score;

[0189] determining the performance parameter of the initial data prediction model based on the product of the accuracy and the first weight, the product of the recall rate and the second weight, and the product of the F1 score and the third weight.

[0190] In the embodiments of the present application, the F1 score is an index widely used in the fields of statistics and machine learning, which is used to measure the performance of a classification model and takes into account both the precision and recall of the model. The accuracy, recall rate and F1 score of the initial data prediction model can be obtained, and then the weights corresponding to the accuracy, recall rate and F1 score are determined according to actual needs; the product of the accuracy and the first weight is calculated to obtain first data, the product of the recall rate and the second weight is calculated to obtain second data, the product of the F1 score and the third weight is calculated to obtain third data, and finally the sum of the first data, second data and third data is calculated to obtain the performance parameter of the initial data prediction model.

[0191] In some embodiments, the method further includes:

[0192] determining a preset threshold based on historical service data of a target object;

[0193] determining the deviation between the first service data and the second service data;

[0194] The deviation between the first service data and the second service data is determined as the difference between the first service data and the second service data.

[0195] In the embodiments of the present disclosure, the preset threshold can be determined according to the historical service data of the target object.

[0196] S209: Obtain the first service data in the real-time service data sequence and the second service data in the predicted service data sequence at the same time.

[0197] In the embodiments of the present disclosure, the real-time service data sequence and the predicted service data sequence are data in the same time period, and in the process of identifying abnormal data, the first service data in the real-time service data sequence and the second service data in the predicted service data sequence at the same time can be obtained respectively.

[0198] S2011: If the difference between the first service data and the second service data is greater than the preset threshold, it is determined that the first service data is abnormal data.

[0199] In the embodiments of the present disclosure, the preset threshold can be set according to actual conditions. For example, the preset threshold can be determined according to the historical service data of the target object. The data identification result of the first service data can be determined according to the difference between the first service data and the second service data. The data identification result represents whether the first service data is abnormal data. If the difference between the first service data and the second service data is greater than the preset threshold, it is determined that the first service data is abnormal data. If the difference between the first service data and the second service data is less than or equal to the preset threshold, it is determined that the first service data is normal data. Thus, abnormal data in the real-time service data sequence can be quickly and accurately screened out, and the cause of the abnormal data can be analyzed, so as to identify the abnormal behavior of the target object, such as brushing orders.

[0200] In the application process of the model, the performance of the model in actual application can be fed back to the training process. The model is continuously adjusted and optimized according to the latest data and business changes. The optimization of the model is a continuous process that needs to be evaluated and adjusted regularly. When optimizing the model, the relationship between different performance indicators needs to be balanced, for example, there is often a trade-off between accuracy and recall rate. Data drift and concept drift problems also need to be considered, which may change over time and affect the performance of the model.

[0201] In the embodiments of the present disclosure, after it is determined that the first service data is abnormal data if the difference between the first service data and the second service data is greater than the preset threshold, the method further comprises:

[0202] obtaining context information corresponding to the abnormal data;

[0203] determining an abnormal reason corresponding to the abnormal data based on the context information of the abnormal data;

[0204] constructing an abnormal processing strategy based on the abnormal reason.

[0205] In the embodiments of the present specification, the abnormal reason corresponding to the abnormal data can be determined in combination with the context information of the abnormal data, and the abnormal processing strategy can be constructed according to the abnormal reason, so as to realize the rapid processing of the abnormal data.

[0206] In the embodiments of the present specification, the spectral analysis reveals the periodicity and frequency characteristics in the data, which can be ignored in traditional time series prediction models. By incorporating these characteristics into the prediction model, we can more accurately capture and predict the dynamic changes of the data.

[0207] The spectral features provide the model with a deep understanding of different aspects of the data, which can help the model perform better when facing unseen data. This generalization ability is crucial for dealing with data changes and uncertainties, enhancing the model's generalization ability; the prediction model combining traditional time series features and spectral features in the present embodiment can more effectively identify abnormal patterns, reduce false positives and false negatives, and improve the accuracy and sensitivity of abnormal data detection. Due to the inclusion of result optimization and feedback mechanism, the entire system can adapt to changes in new data and environment, ensuring long-term effectiveness, adaptability and flexibility. The automated data processing and analysis process of the present embodiment reduces the need for human intervention, improves operational efficiency and reduces the likelihood of human error, improving data processing efficiency.

[0208] It can be seen from the technical solutions provided by the embodiments of the present specification that the embodiments of the present specification obtain a target service data sequence of a target object in a first target period and a real-time service data sequence in a second target period; the first target period is a historical period corresponding to the second target period; then, spectral feature extraction is performed on the target service data sequence to obtain a target main frequency feature, a target amplitude distribution feature, and a target power spectral density feature; based on the target main frequency feature, the target amplitude distribution feature, and the target power spectral density feature, a target spectral analysis feature corresponding to the target service data sequence is determined, and the target spectral analysis feature includes a main periodicity feature, a main frequency band feature, and a main energy band feature; according to the target service data sequence and the target spectral analysis feature, a predicted service data sequence of the target object in the second target period is determined; in the data prediction process, the spectral analysis feature is introduced, so that the dynamic change of the data can be more accurately captured and predicted; by combining the target service data sequence and the target spectral analysis feature, the predicted service data sequence of the target object in the second target period is predicted, and the accuracy of the predicted service data sequence is improved; the first service data in the real-time service data sequence and the second service data in the predicted service data sequence at the same time are obtained; if the difference between the first service data and the second service data is greater than a preset threshold, the first service data is determined as abnormal data, so that the target service data sequence and the target spectral analysis feature can be combined to accurately identify abnormal data and reduce false positives and false negatives of abnormal data.

[0209] The embodiments of the present specification also provide an abnormal data identification device, as shown in Figure 8 The device comprises:

[0210] The first data acquisition module 810 is configured to acquire a target service data sequence of a target object in a first target period and a real-time service data sequence in a second target period; the first target period is a historical period corresponding to the second target period;

[0211] The spectral feature extraction module 820 is configured to perform spectral feature extraction on the target service data sequence to obtain a target main frequency feature, a target amplitude distribution feature, and a target power spectral density feature.

[0212] The spectral analysis module 830 is configured to determine a target spectral analysis feature corresponding to the target service data sequence based on the target main frequency feature, the target amplitude distribution feature, and the target power spectral density feature; the target spectral analysis feature includes a main periodicity feature, a main frequency band feature, and a main energy band feature.

[0213] The predicted data determination module 840 is configured to determine a predicted service data sequence of the target object in the second target period according to the target service data sequence and the target spectral analysis feature.

[0214] The second data acquisition module 850 is configured to acquire first service data in the real-time service data sequence and second service data in the predicted service data sequence at the same time point.

[0215] The abnormal data determination module 860 is configured to determine that the first service data is abnormal data if a difference between the first service data and the second service data is greater than a preset threshold.

[0216] In an example embodiment, the spectrum analysis module comprises:

[0217] The first analysis unit is configured to analyze the target main frequency feature to obtain a main periodicity feature corresponding to the target service data sequence.

[0218] The second analysis unit is configured to analyze the target amplitude distribution feature to obtain a main energy band feature corresponding to a main energy.

[0219] The third analysis unit is configured to analyze the target power spectral density feature to obtain a main energy band feature.

[0220] The feature determination unit is configured to determine a target spectrum analysis feature corresponding to the target service data sequence based on the main periodicity feature, the main energy band feature, and the main energy band feature.

[0221] In an example embodiment, the first analysis unit comprises:

[0222] The feature analysis subunit is configured to analyze the target main frequency feature to obtain a target main cycle feature and a target seasonal cycle feature.

[0223] The feature determination subunit is configured to determine the target main cycle feature and the target seasonal cycle feature as the main periodicity feature corresponding to the target service data sequence.

[0224] In an example embodiment, the spectrum feature extraction module comprises:

[0225] The target processing unit is configured to perform data preprocessing on the target service data sequence to obtain target processing data.

[0226] The data switching unit is configured to convert the target processing data from a time domain to a frequency domain by Fourier transform to obtain target time domain data.

[0227] The main frequency identification unit is configured to identify a main frequency component in the target time domain data to obtain the target main frequency feature.

[0228] An amplitude recognition unit is configured to recognize the target amplitude distribution feature corresponding to the target time domain data, wherein the target amplitude distribution feature represents amplitude distribution of each frequency component in the target time domain data.

[0229] A power feature determination unit is configured to determine energy distribution of the target time domain data in the frequency domain, and obtain the target power spectral density feature.

[0230] In an example embodiment, the device further comprises:

[0231] A phase acquisition module is configured to acquire target phase information of the target time domain data.

[0232] In an example embodiment, the data switching unit comprises:

[0233] An initial data determination sub-unit is configured to convert the target processing data from the time domain to the frequency domain by Fourier transform, and obtain initial time domain data.

[0234] A target data determination sub-unit is configured to remove noise information in the initial time domain data based on the target phase information, and obtain the target time domain data.

[0235] In an example embodiment, the prediction data determination module comprises:

[0236] A model acquisition unit is configured to acquire a data prediction model, wherein the data prediction model is obtained by training a preset model based on a sample object and a sample service data sequence of the sample object.

[0237] A prediction unit is configured to input the target service data sequence and the target spectrum analysis feature into the data prediction model for data prediction, and obtain a predicted service data sequence of the target object in the second target time period.

[0238] In an example embodiment, the device further comprises:

[0239] A sample sequence acquisition module is configured to acquire a sample service data sequence of the sample object in a first preset time period, wherein the sample service data sequence is labeled with a sample real-time data sequence label, the sample real-time data sequence label is a real-time service data sequence of the sample object in a second preset time period, and the first preset time period is a historical time period corresponding to the second preset time period.

[0240] An analysis feature determination module is configured to determine a sample spectrum analysis feature corresponding to the sample service data sequence, wherein the sample spectrum analysis feature comprises a sample main periodicity feature, a sample main frequency band feature, and a sample main energy band feature.

[0241] A sample real-time result determination module is configured to input the sample service data sequence and the sample spectrum analysis feature into a preset model to perform data prediction processing, and obtain a sample real-time data sequence result.

[0242] A training module is configured to train the preset model based on a difference between the sample real-time data sequence result and a sample real-time data sequence label, and obtain the data prediction model.

[0243] In an example embodiment, the analysis feature determination module includes:

[0244] A spectrum feature extraction unit is configured to perform spectrum feature extraction on the sample service data sequence, and obtain a sample main frequency feature, a sample amplitude distribution feature, and a sample power spectrum density feature.

[0245] An analysis feature determination unit is configured to determine a sample spectrum analysis feature based on the sample main frequency feature.

[0246] A sample main periodicity determination unit is configured to determine a sample main periodicity feature based on the sample power spectrum density feature.

[0247] A sample main energy determination unit is configured to determine a sample main energy band feature based on the sample power spectrum density feature.

[0248] The analysis feature determination unit is configured to determine the sample spectrum analysis feature, the sample main periodicity feature, and the sample main energy band feature as the sample spectrum analysis feature corresponding to the sample service data sequence.

[0249] In an example embodiment, the training module includes:

[0250] A target loss determination unit is configured to determine target loss data based on a difference between the sample real-time data sequence result and the sample real-time data sequence label.

[0251] An initial model determination unit is configured to adjust model parameters of the preset model based on the target loss data, and obtain an initial data prediction model.

[0252] A test sequence acquisition unit is configured to acquire a test service data sequence of a test object.

[0253] A parameter determination unit is configured to test a performance parameter of the initial data prediction model based on the test service data sequence.

[0254] A model determination unit is configured to determine the initial data prediction model as the data prediction model if the performance parameter of the initial data prediction model meets a preset condition.

[0255] In an example implementation, the parameter determination unit comprises:

[0256] a data test subunit configured to test accuracy, recall rate and F1 score of the initial data prediction model based on the test service data sequence;

[0257] a weight determination subunit configured to determine a first weight corresponding to the accuracy, a second weight corresponding to the recall rate, and a third weight corresponding to the F1 score;

[0258] a parameter determination subunit configured to determine a performance parameter of the initial data prediction model based on a product of the accuracy and the first weight, a product of the recall rate and the second weight, and a product of the F1 score and the third weight.

[0259] In an example implementation, the apparatus further comprises:

[0260] a context acquisition module configured to acquire context information corresponding to the abnormal data;

[0261] an abnormal reason determination module configured to determine an abnormal reason corresponding to the abnormal data based on the context information of the abnormal data;

[0262] a strategy construction module configured to construct an abnormal processing strategy based on the abnormal reason.

[0263] The apparatus in the apparatus embodiment and the method embodiment are based on the same inventive concept.

[0264] The electronic device provided by the embodiments of the present disclosure includes a processor and a memory, and the memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement the abnormal data identification method provided by the above method embodiments.

[0265] The embodiments of the present disclosure also provide a computer storage medium, which can be arranged in a terminal to save at least one instruction or at least one program related to the abnormal data identification method in the method embodiments, and the at least one instruction or at least one program is loaded and executed by the processor to implement the abnormal data identification method provided by the above method embodiments.

[0266] The embodiments of the present disclosure also provide a computer program product or a computer program, which includes computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to make the computer device perform the abnormal data identification method provided by the above method embodiments.

[0267] Optionally, in the embodiments of the present specification, the storage medium can be located in at least one of the plurality of network servers of the computer network. Optionally, in the embodiments, the storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0268] The memory in the embodiments of the present specification can be used to store software programs and modules, and the processor executes various function applications and data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required for functions, etc.; and the data storage area can store data created according to the use of the device, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the memory can also include a memory controller to provide the processor with access to the memory.

[0269] The abnormal data identification method provided by the embodiments of the present specification can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Taking the case of running on a server as an example, Figure 9 is a hardware structure block diagram of a server of an abnormal data identification method provided by the embodiments of the present specification. As shown in Figure 9As shown, the server 900 can vary greatly in configuration and performance, and can include one or more Central Processing Units (CPU) 910 (the CPU 910 can include, but is not limited to, a microprocessor, a programmable logic device, a processing device, etc.), a memory 930 for storing data, one or more storage media 920 (e.g., one or more mass storage devices) for storing applications 923 or data 922. The memory 930 and the storage media 920 can be of the temporary or persistent storage variety. The programs stored in the storage media 920 can include one or more modules, each of which can include a series of instructions for operating on the server. Further, the CPU 910 can be configured to communicate with the storage media 920 to execute a series of instructions operating on the server 900 in the storage media 920. The server 900 can also include one or more power supplies 960, one or more wired or wireless network interfaces 950, one or more input / output interfaces 940, and / or one or more operating systems 921, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0270] The input / output interface 940 can be configured to receive or transmit data via a network. Examples of the network can include a wireless network provided by a communication provider of the server 900. In one example, the input / output interface 940 includes a network interface controller (NIC) that can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the input / output interface 940 can be a radio frequency (RF) module configured to communicate with the Internet through a wireless manner.

[0271] Those of ordinary skill in the art can understand that, Figure 9 The structure shown is merely illustrative and does not limit the structure of the electronic device described above. For example, the server 900 can include more or fewer components than those shown in Figure 9 or have a different configuration than that shown in Figure 9 .

[0272] It can be seen from the embodiments of the abnormal data identification method, device, equipment or storage medium provided in the application that the target object is obtained in the first target period and the real-time service data sequence in the second target period; the first target period is a historical period corresponding to the second target period; the target service data sequence is subjected to frequency spectrum feature extraction to obtain a target main frequency feature, a target amplitude distribution feature and a target power spectrum density feature; based on the target main frequency feature, the target amplitude distribution feature and the target power spectrum density feature, a target spectrum analysis feature corresponding to the target service data sequence is determined, and the target spectrum analysis feature includes a main periodicity feature, a main frequency band feature and a main energy band feature; according to the target service data sequence and the target spectrum analysis feature, a predicted service data sequence of the target object in the second target period is determined; in the data prediction process, the spectrum analysis feature is introduced, so that the dynamic change of the data can be more accurately captured and predicted; the application predicts the predicted service data sequence of the target object in the second target period by combining the target service data sequence and the target spectrum analysis feature, thereby improving the accuracy of the predicted service data sequence; the first service data in the real-time service data sequence and the second service data in the predicted service data sequence at the same time are obtained; if the difference between the first service data and the second service data is greater than a preset threshold, the first service data is determined as abnormal data, so that the abnormal data can be accurately identified by combining the target service data sequence and the target spectrum analysis feature, and the false positives and false negatives of the abnormal data are reduced.

[0273] It should be noted that the above-described order of the embodiments of the present specification is only for description, and does not represent the advantages and disadvantages of the embodiments. The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are possible or can be advantageous.

[0274] Each embodiment in the present specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other, and each embodiment mainly describes the difference from other embodiments. In particular, for the device, equipment and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the related parts refer to the part of the method embodiment.

[0275] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or can be instructed by a program to complete the related hardware, and the program can be stored in a computer storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0276] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for identifying abnormal data, characterized in that: The method comprises: Obtain a target business data sequence of a target object in a first target period and a real-time business data sequence in a second target period; the first target period is a historical period corresponding to the second target period; Extracting spectrum features of the target service data sequence to obtain target main frequency features, target amplitude distribution features, and target power spectrum density features; Determine target spectrum analysis features corresponding to the target service data sequence based on the target main frequency features, target amplitude distribution features, and target power spectrum density features, wherein the target spectrum analysis features include main periodicity features, main frequency band features, and main energy band features; determining a predicted service data sequence of the target object in the second target time period according to the target service data sequence and the target spectrum analysis feature; Acquire first business data in the real-time business data sequence and second business data in the predicted business data sequence at the same time; If the difference between the first business data and the second business data is greater than a preset threshold, the first business data is determined to be abnormal data.

2. The method according to claim 1, characterized in that The determining, based on the target main frequency characteristic, the target amplitude distribution characteristic, and the target power spectrum density characteristic, a target spectrum analysis characteristic corresponding to the target service data sequence includes: Analyzing the target main frequency characteristics to obtain the main periodic characteristics corresponding to the target service data sequence; Analyzing the target amplitude distribution characteristics to obtain the main frequency band characteristics corresponding to the main energy; Analyzing the target power spectrum density characteristics to obtain the main energy band characteristics; Based on the main periodicity characteristics, the main frequency band characteristics and the main energy band characteristics, the target spectrum analysis characteristics corresponding to the target service data sequence are determined.

3. The method according to claim 2, characterized in that The analyzing the target main frequency feature to obtain the main periodic feature corresponding to the target service data sequence includes: Analyzing the target main frequency characteristics to obtain target main cycle characteristics and target seasonal cycle characteristics; The target main cycle characteristics and the target seasonal cycle characteristics are determined as the main periodic characteristics corresponding to the target business data sequence.

4. The method according to claim 2, characterized in that The extracting of spectrum features of the target service data sequence to obtain target main frequency features, target amplitude distribution features, and target power spectrum density features includes: Performing data preprocessing on the target service data sequence to obtain target processed data; Converting the target processed data from the time domain to the frequency domain by Fourier transform to obtain target time domain data; Identifying the main frequency component in the target time domain data to obtain the target main frequency feature; Identifying the target amplitude distribution feature corresponding to the target time domain data; the target amplitude distribution feature represents the amplitude distribution of each frequency component in the target time domain data; The energy distribution of the target time domain data in the frequency domain is determined to obtain the target power spectrum density feature.

5. The method according to claim 4, characterized in that The method further comprises: Acquiring target phase information of the target time domain data; The method of converting the target processed data from the time domain to the frequency domain by Fourier transform to obtain target time domain data includes: Converting the target processed data from the time domain to the frequency domain by Fourier transform to obtain initial time domain data; Based on the target phase information, noise information in the initial time domain data is removed to obtain the target time domain data.

6. The method according to claim 1, characterized in that The determining, based on the target service data sequence and the target spectrum analysis feature, the predicted service data sequence of the target object in the second target time period includes: Obtaining a data prediction model; the data prediction model is obtained by training a preset model based on a sample business data sequence of a sample object; The target service data sequence and the target spectrum analysis features are input into a data prediction model to perform data prediction, so as to obtain a predicted service data sequence of the target object in the second target time period.

7. The method according to claim 6, characterized in that The training method of the data prediction model includes: Acquire a sample business data sequence of the sample object in a first preset time period; the sample business data sequence is marked with a sample real-time data sequence label; the sample real-time data sequence label is a real-time business data sequence of the sample object in a second preset time period; the first preset time period is a historical time period corresponding to the second preset time period; Determine a sample spectrum analysis feature corresponding to the sample service data sequence, the sample spectrum analysis feature including a sample main periodicity feature, a sample main frequency band feature, and a sample main energy band feature; Input the sample business data sequence and the sample spectrum analysis features into a preset model for data prediction processing to obtain a sample real-time data sequence result; Based on the difference between the sample real-time data sequence result and the sample real-time data sequence label, the preset model is trained to obtain the data prediction model.

8. The method according to claim 7, characterized in that The determining of the sample spectrum analysis feature corresponding to the sample service data sequence includes: Extracting spectrum features of the sample service data sequence to obtain sample main frequency features, sample amplitude distribution features, and sample power spectrum density features; Determining a sample spectrum analysis feature based on the sample main frequency feature; Determining a main periodicity feature of the sample based on the power spectrum density feature of the sample; Determining a main energy band characteristic of the sample based on the power spectrum density characteristic of the sample; The sample spectrum analysis feature, the sample main periodicity feature, and the sample main energy band feature are determined as the sample spectrum analysis feature corresponding to the sample business data sequence.

9. The method according to claim 7, characterized in that The step of training the preset model based on the difference between the sample real-time data sequence result and the sample real-time data sequence label to obtain the data prediction model includes: determining target loss data based on a difference between the sample real-time data sequence result and the sample real-time data sequence label; Adjusting the model parameters of the preset model based on the target loss data to obtain an initial data prediction model; Obtain the test business data sequence of the test object; Testing the performance parameters of the initial data prediction model based on the test business data sequence; If the performance parameters of the initial data prediction model meet the preset conditions, the initial data prediction model is determined as the data prediction model.

10. The method according to claim 9, characterized in that The testing of the performance parameters of the initial data prediction model based on the test business data sequence includes: Testing the accuracy, recall, and F1 score of the initial data prediction model based on the test business data sequence; Determine a first weight corresponding to the precision rate, a second weight corresponding to the recall rate, and a third weight corresponding to the F1 score; Performance parameters of the initial data prediction model are determined based on the product of the precision and the first weight, the product of the recall and the second weight, and the product of the F1 score and the third weight.

11. The method according to claim 1, wherein If the difference between the first business data and the second business data is greater than a preset threshold, after determining that the first business data is abnormal data, the method further includes: Obtaining context information corresponding to the abnormal data; Determining the abnormal cause corresponding to the abnormal data based on the context information of the abnormal data; Based on the cause of the exception, an exception handling strategy is constructed.

12. An abnormal data identification device, characterized in that: The device comprises: A first data acquisition module is configured to acquire a target business data sequence of a target object in a first target period and a real-time business data sequence in a second target period; the first target period is a historical period corresponding to the second target period; A spectrum feature extraction module is used to extract spectrum features of the target service data sequence to obtain target main frequency features, target amplitude distribution features and target power spectrum density features; A spectrum analysis module is used to determine target spectrum analysis features corresponding to the target service data sequence based on the target main frequency features, target amplitude distribution features, and target power spectrum density features, wherein the target spectrum analysis features include main periodicity features, main frequency band features, and main energy band features; a predicted data determination module, configured to determine a predicted service data sequence of the target object in the second target time period according to the target service data sequence and the target spectrum analysis characteristics; A second data acquisition module is used to acquire the first business data in the real-time business data sequence and the second business data in the predicted business data sequence at the same time; The abnormal data determination module is configured to determine that the first business data is abnormal data if the difference between the first business data and the second business data is greater than a preset threshold.

13. An electronic device, characterized in that: The device includes: a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the abnormal data identification method according to any one of claims 1-11.

14. A computer storage medium, characterized in that The computer storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by a processor to implement the abnormal data identification method according to any one of claims 1 to 11.

15. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the abnormal data identification method according to any one of claims 1 to 11 is implemented.

Images