Database anomaly detection method and device, equipment, medium and program product
By performing timing prediction and abnormal detection of the active session list of the database, the monitoring alarm coverage and accuracy problems caused by relying on the understanding and familiarity of operation and maintenance personnel in the prior art are solved, and efficient and accurate database abnormal detection and alarm are achieved.
Patent Information
- Application Number
- CN202411781856.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-05
- Publication Date
- 2025-03-21
AI Technical Summary
In the existing technology, relying on operation and maintenance personnel to understand database business and familiarity with database principles leads to the risk of subjective and untimely updates of monitoring alarm coverage and accuracy, which may lead to failure to alert database business problems in time and the impact is uncontrollable.
Provide a database exception detection method, which obtains original data by querying the active session list, performs serialization preprocessing, uses the pretrained timing prediction model to perform timing prediction, obtains the timing point deviation value, and uses the pretrained target exception detection model to perform exception detection, achieving high-level alarm coverage and accuracy.
Through dynamic timing prediction and abnormal detection, the alarm threshold and alarm dimension are optimized, the subjective dependence on operation and maintenance personnel is reduced, and the high-level alarm coverage and accuracy are achieved, and the risk of database business impact is reduced.
Smart Images
Figure CN119668994A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of database technology, and in particular to a database anomaly detection method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art
[0002] With the development of database technology, how to detect database load anomalies has become a key part of database technology. In the existing technology, data collection and threshold detection are performed by using monitoring and alarms. For the database, monitoring and alarms are the "eyes" of observing the database, and their coverage and accuracy are the prerequisites for load anomaly detection. However, in the existing technology, the operation and maintenance personnel's understanding of the database business and their familiarity with the principles of the database are relied on to ensure the coverage and accuracy of monitoring alarms. However, this maintenance method has risk factors such as subjectivity and untimely updates, and in some scenarios, there may be no alarm prompts when problems occur in the database business, and there is a risk of uncontrollable impact on the database business. Summary of the invention
[0003] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product that can perform time series prediction in real time to obtain time series point deviation values for measuring the load data of the database, and combine the anomaly detection method to achieve a high level of alarm coverage and accuracy.
[0004] In a first aspect, the present application provides a database anomaly detection method, comprising:
[0005] In one embodiment, the method comprises:
[0006] Query the active session list to obtain original data; the active session list is used to record the load data of the database;
[0007] Performing serialization preprocessing on the original data to obtain a preprocessing sequence;
[0008] Performing time series prediction on the preprocessed sequence according to a pre-trained time series prediction model to obtain a time series point deviation value;
[0009] Anomaly detection is performed on the time sequence point deviation value according to a pre-trained target anomaly detection model to obtain an anomaly detection result on the database.
[0010] In one embodiment, the serialization preprocessing of the original data to obtain a preprocessed sequence includes:
[0011] Performing data cleaning on the original data to obtain candidate data;
[0012] Perform data conversion on the candidate data to obtain the preprocessing sequence.
[0013] In one embodiment, performing timing prediction on the preprocessed sequence according to the pre-trained timing prediction model to obtain the timing point deviation value includes:
[0014] Performing database load analysis on the preprocessing sequence according to the time series prediction model to obtain a database load baseline;
[0015] The residual calculation is performed on the preprocessing sequence according to the database load baseline to obtain the timing point deviation value.
[0016] In one embodiment, performing anomaly detection on the time sequence point deviation value according to the pre-trained target anomaly detection model to obtain an anomaly detection result on the database includes:
[0017] Performing path length prediction on the time series data corresponding to the time series point deviation value according to the target anomaly detection model to obtain the path length of the time series data;
[0018] Anomaly detection is performed on the time series data according to the target anomaly detection model and the path length to obtain the anomaly detection result.
[0019] In one embodiment, the method further comprises:
[0020] Acquiring training load data, and preprocessing the training load data to obtain training preprocessing data;
[0021] Perform time series prediction on the training preprocessed data according to the original time series prediction model to obtain a time series point deviation value;
[0022] The model parameters of the original time series prediction model are adjusted according to the verification data corresponding to the training load data and the time series point deviation value to obtain the time series prediction model.
[0023] In one embodiment, after performing anomaly detection on the time sequence point deviation value according to the pre-trained target anomaly detection model to obtain an anomaly detection result on the database, the method further includes:
[0024] Obtain business scenario data;
[0025] Anomaly point analysis is performed on the anomaly detection result according to the business scenario data to obtain a target anomaly point set.
[0026] In a second aspect, the present application also provides a database anomaly detection device, comprising:
[0027] A query module, used to query an active session list to obtain raw data; the active session list is used to record database load data;
[0028] A preprocessing module, used for performing serialization preprocessing on the original data to obtain a preprocessing sequence;
[0029] A timing prediction module, used to perform timing prediction on the preprocessed sequence according to a pre-trained timing prediction model to obtain a timing point deviation value;
[0030] The anomaly detection module is used to perform anomaly detection on the time point deviation value according to a pre-trained target anomaly detection model to obtain anomaly detection results about the database.
[0031] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0032] Query the active session list to obtain original data; the active session list is used to record the load data of the database;
[0033] Performing serialization preprocessing on the original data to obtain a preprocessing sequence;
[0034] Performing time series prediction on the preprocessed sequence according to a pre-trained time series prediction model to obtain a time series point deviation value;
[0035] Anomaly detection is performed on the time sequence point deviation value according to a pre-trained target anomaly detection model to obtain an anomaly detection result on the database.
[0036] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:
[0037] Query the active session list to obtain original data; the active session list is used to record the load data of the database;
[0038] Performing serialization preprocessing on the original data to obtain a preprocessing sequence;
[0039] Performing time series prediction on the preprocessed sequence according to a pre-trained time series prediction model to obtain a time series point deviation value;
[0040] Anomaly detection is performed on the time sequence point deviation value according to a pre-trained target anomaly detection model to obtain an anomaly detection result on the database.
[0041] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:
[0042] Query the active session list to obtain original data; the active session list is used to record the load data of the database;
[0043] Performing serialization preprocessing on the original data to obtain a preprocessing sequence;
[0044] Performing time series prediction on the preprocessed sequence according to a pre-trained time series prediction model to obtain a time series point deviation value;
[0045] Anomaly detection is performed on the time sequence point deviation value according to a pre-trained target anomaly detection model to obtain an anomaly detection result on the database.
[0046] The above-mentioned database anomaly detection method, device, computer equipment, computer-readable storage medium and computer program product obtain the original data by querying the active session list; the active session list is used to record the load data of the database; the original data is serialized and preprocessed to obtain the preprocessed sequence; the preprocessed sequence is time-series predicted according to the pre-trained time series prediction model to obtain the time series point deviation value; the time series point deviation value is detected according to the pre-trained target anomaly detection model to obtain the anomaly detection result about the database. Therefore, by querying the active session list to obtain the original data, the monitoring item value path is optimized, the number of queries to the business database is greatly reduced, and after the original data is serialized and preprocessed to obtain the preprocessed sequence, the preprocessed sequence is time-series predicted according to the pre-trained time series prediction model, the time series deviation value can be predicted in real time based on the preprocessed sequence, and the time series point deviation value can be detected according to the pre-trained target anomaly detection model, that is, the time series point deviation value and anomaly detection are obtained through dynamic time series prediction, which can optimize the alarm threshold and alarm dimension, thereby reducing the subjective dependence on operation and maintenance personnel and achieving a high level of alarm coverage and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0048] Figure 1 An application environment diagram of a database anomaly detection method in one embodiment;
[0049] Figure 2 A schematic diagram of a flow chart of a database anomaly detection method in one embodiment;
[0050] Figure 3A schematic diagram of a flow chart of a database anomaly detection method in one embodiment;
[0051] Figure 4 A complete flowchart of a database anomaly detection method in one embodiment;
[0052] Figure 5 A schematic diagram of a model training process of a database anomaly detection method in one embodiment;
[0053] Figure 6 is a structural block diagram of a database anomaly detection device in one embodiment;
[0054] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0055] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0056] The database anomaly detection method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The server 104 obtains the original data by querying the active session list; the active session list is used to record the load data of the database; the original data is serialized and preprocessed to obtain the preprocessed sequence; the preprocessed sequence is predicted according to the pre-trained time series prediction model to obtain the time series point deviation value; the time series point deviation value is detected according to the pre-trained target anomaly detection model to obtain the anomaly detection result about the database. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.
[0057] In an exemplary embodiment, Figure 2As shown, a database anomaly detection method is provided, which is applied to Figure 1 The server 104 in the example is used as an example to illustrate the method, which includes the following steps S202 to S208. Among them:
[0058] Step S202, query the active session list to obtain original data; the active session list is used to record the load data of the database.
[0059] Among them, the active session list includes information such as the source IP of the database request, the operating database, the executed SQL, the SQL duration, the waiting event, the operating user, etc. One query can replace at least more than 10 traditional monitoring items and alarms, such as monitoring the number of active sessions and the SQL with the longest execution time in active sessions.
[0060] In some embodiments, a database active session collection module may be set to query the current active session list from the business database, and JDBC, AGENT and other solutions may be used to connect to the database to obtain raw data, but the present invention is not limited thereto.
[0061] In some embodiments, a raw data storage module may be provided to receive raw data sent by an active session list collection service. The table structure fields for storing the raw data are the same as those in the business database, thereby ensuring consistency between the data structure of the session storage and the session information in the business database.
[0062] In this embodiment, by querying the active session list, the method of obtaining the original data can solve the problem of the impact on database performance caused by the increase of monitoring items, optimize the monitoring item value path, and greatly reduce the number of queries to the business database. Among them, the monitoring item can be the original data.
[0063] Step S204, performing serialization preprocessing on the original data to obtain a preprocessing sequence.
[0064] Among them, serialization preprocessing refers to performing conventional data preprocessing on the data and then converting the processed data into serialized data to obtain sequence data.
[0065] In some embodiments, the original data is subjected to serialization preprocessing to obtain a preprocessing sequence, including: performing data cleaning on the original data to obtain candidate data; and performing data conversion on the candidate data to obtain a preprocessing sequence.
[0066] Among them, the data cleaning process can be carried out by removing null values, processing outliers, etc.
[0067] In some embodiments, the original data is processed by removing null values, processing outliers, etc. to obtain candidate data, and then features are extracted from the candidate data. Technical features that have a relevant impact on the database load of the current business are extracted from the original data of the active session list, and the extracted technical features are normalized to meet the input requirements of the subsequent time series prediction model to improve the accuracy of the subsequent time series prediction. Finally, the normalized data is converted into serialized data to obtain a preprocessed sequence.
[0068] The extracted technical features may be Schema_name (database name), Digest (SQL fingerprint), Time (execution duration), etc., and the preprocessing sequence may be time window sequence data, but are not limited thereto.
[0069] Step S206, performing timing prediction on the preprocessed sequence according to the pre-trained timing prediction model to obtain timing point deviation values.
[0070] Among them, the pre-trained time series prediction model can adopt LSTM (Long Short-Term Memory), which is a special recurrent neural network (RNN) architecture. It is very good at processing and predicting time intervals and long-term dependencies in time series data. As a variant of recurrent neural network, the LSTM model can learn long-term dependencies and short-term fluctuations in time series data, and perform time series prediction based on these learning results. Time series prediction refers to the process of analyzing the past behavior of time series data to predict future values.
[0071] In some embodiments, timing prediction is performed on a preprocessed sequence according to a pretrained timing prediction model to obtain timing point deviation values, including: performing database load analysis on the preprocessed sequence according to the timing prediction model to obtain a database load baseline; performing residual calculation on the preprocessed sequence according to the database load baseline to obtain timing point deviation values.
[0072] In some embodiments, a database load analysis is performed on the preprocessing sequence according to a timing prediction model to obtain a database load baseline, that is, a performance baseline of indicators of each dimension in the active session list is drawn according to the preprocessing sequence. The performance baseline can be used to accurately measure the residual between the distribution of actual indicator points of each dimension in the preprocessing sequence and the performance baseline, thereby obtaining a timing point deviation value, and based on this timing point deviation value, a more effective input is provided to the subsequent anomaly detection step S.
[0073] In this embodiment, by performing timing prediction on the preprocessing sequence according to the pre-trained timing prediction model to obtain the timing point deviation value, the problem of insufficient granularity of monitoring alarm coverage can be solved. For example, for different SQLs, dynamic performance baselines and anomaly detection can be used to more accurately detect changes in SQL-level load data.
[0074] Step S208, performing anomaly detection on the time point deviation value according to the pre-trained target anomaly detection model to obtain anomaly detection results about the database.
[0075] Among them, the target anomaly detection model can adopt the isolation forest model, but is not limited to this.
[0076] In some embodiments, anomaly detection is performed on the timing point deviation value according to a pre-trained target anomaly detection model to obtain anomaly detection results about the database, including: predicting the path length of the timing data corresponding to the timing point deviation value according to the target anomaly detection model to obtain the path length of the timing data; performing anomaly detection on the timing data according to the target anomaly detection model and the path length to obtain anomaly detection results.
[0077] Among them, path length prediction is the process of predicting the path length of the time series data corresponding to the deviation value of the time point in the decision tree. It should be noted that the target anomaly detection model will isolate the time series data by building a decision tree based on the isolation forest algorithm (Isolation Forest) during the pre-training stage. Among them, the path lengths of normal data and abnormal data in the decision tree are different.
[0078] In some embodiments, the path length of the time series data corresponding to the time point deviation value is predicted according to the target anomaly detection model to obtain the path length of the time series data in the decision tree, which is used to perform anomaly detection on the time series data according to the target anomaly detection model and the path length, thereby accurately and quickly completing the detection of abnormal data in the load data in the database and obtaining anomaly detection results.
[0079] In this embodiment, anomaly detection is performed on the time series point deviation value according to the pre-trained target anomaly detection model to obtain anomaly detection results about the database. The target anomaly detection model trained by this machine learning algorithm can automatically identify anomalies without manually defining thresholds. It can automatically adapt to the distribution of data and can process multi-dimensional data. It has better generalization ability and robustness.
[0080] In some embodiments, after performing anomaly detection on the timing point deviation value according to the pre-trained target anomaly detection model to obtain anomaly detection results about the database, the method also includes: acquiring business scenario data; performing anomaly point analysis on the anomaly detection results according to the business scenario data to obtain a target anomaly point set.
[0081] The business scenario data may be input data set according to a specific business scenario, so as to distinguish between planned and unplanned abnormal point sets.
[0082] In some embodiments, a maintenance window can be defined to obtain the time range of the maintenance window, and the time range of the maintenance window can be compared with the timestamp in the anomaly detection result to obtain a comparison result, and anomaly point analysis can be performed based on the comparison result. The anomaly points in the maintenance window can be marked as 0, that is, the anomaly detection results are traversed, and the anomaly points in the maintenance window are discarded to obtain a target anomaly point set.
[0083] In some embodiments, the target abnormal point set is used for error reporting service, that is, after abnormal point analysis and filtering based on the business scenario data, the target abnormal point set obtained is used for error reporting.
[0084] In the above database anomaly detection method, the original data is obtained by querying the active session list; the active session list is used to record the load data of the database; the original data is serialized and preprocessed to obtain a preprocessed sequence; the preprocessed sequence is time-series predicted according to the pre-trained time series prediction model to obtain the time series point deviation value; the time series point deviation value is detected according to the pre-trained target anomaly detection model to obtain the anomaly detection result of the database. Therefore, by querying the active session list to obtain the original data, the monitoring item value path is optimized, the number of queries to the business database is greatly reduced, and after the original data is serialized and preprocessed to obtain the preprocessed sequence, the preprocessed sequence is time-series predicted according to the pre-trained time series prediction model, the time series deviation value can be predicted in real time based on the preprocessed sequence, and the time series point deviation value can be detected according to the pre-trained target anomaly detection model. That is, the time series deviation value and anomaly detection are obtained through dynamic time series prediction, which can optimize the alarm threshold and alarm dimension, thereby reducing the subjective dependence on operation and maintenance personnel and achieving a high level of alarm coverage and accuracy.
[0085] In an exemplary embodiment, Figure 3 As shown, the database anomaly detection method also includes a time series prediction model training step, which includes steps S302 to S306. Among them:
[0086] Step S302, obtaining training load data, and preprocessing the training load data to obtain training preprocessing data.
[0087] Among them, the training load data is the load data about the database, and the training preprocessing data is the data.
[0088] In some embodiments, the load data of the database may be obtained in real time, or the load data of the database may be obtained more efficiently by querying a list of active training sessions responsible for monitoring the database, but the present invention is not limited thereto.
[0089] In some embodiments, the training load data is cleaned, and the cleaned data is serialized to obtain training preprocessed data.
[0090] Step S304, performing time series prediction on the training preprocessed data according to the original time series prediction model to obtain a time series point deviation value.
[0091] Among them, the original time series prediction model is a predefined original model structure, such as a predefined number of layers, number of units, activation function, etc.
[0092] In some embodiments, time series prediction is performed on the training preprocessed data according to a predefined original time series prediction model to obtain a time point deviation value for evaluating whether the original time series prediction model can reasonably predict the future data trend of the training preprocessed data.
[0093] Step S306, adjusting the model parameters of the original time series prediction model according to the verification data and the time series point deviation value corresponding to the training load data to obtain the time series prediction model.
[0094] Among them, the adjustment of model parameters is mainly aimed at data such as batch size, number of iterations, and learning rate.
[0095] In some embodiments, the timing point deviation value is verified and evaluated through verification data corresponding to the training load data to obtain specific evaluation data, and the model parameters of the original timing prediction model are adjusted based on the evaluation data to obtain the timing prediction model.
[0096] In this embodiment, training load data is obtained and prediction processing is performed on the training load data to obtain training preprocessing data; time series prediction is performed on the training preprocessing data according to the original time series prediction model to obtain time series point deviation values; model parameters of the original time series prediction model are adjusted according to the verification data and the time series point deviation values corresponding to the training load data to obtain a time series prediction model, which can ensure that the time series prediction model can accurately predict the future trend of the time series data, thereby improving the anomaly detection efficiency for database anomalies in the application stage.
[0097] In order to understand the solution of this application more clearly, Figure 4 and Figure 5 The details are as follows:
[0098] in, Figure 4 The complete process diagram of this scheme in practical application is shown in Figure 5 It is a flowchart of the training process of the time series prediction model, where Sequential is the original time series prediction model and the input X-preprocessed refers to the training preprocessed data.
[0099] First, in Figure 4 In the process, the database active session collection module is used to connect to the DB database to query the active session list and obtain the original data. The active session list is used to record the load data of the database. After the acquired active session information is stored, the stored original data is serialized and preprocessed, such as Figure 4 The data clearing and data conversion operations shown in the figure are performed, and then synchronously in the process of time series model training, the original time series prediction model originally defined is trained and parameter adjustment is performed according to the preprocessing sequence obtained in real time. The trained time series prediction model is used to perform time series prediction on the preprocessing sequence to obtain the deviation value of each time series point, and then the time series point deviation value is input into the anomaly detection model for training and parameter adjustment. Finally, outlier processing is performed in combination with the business context, which can better detect and process database load anomalies in combination with actual application scenarios.
[0100] Secondly, in Figure 5 In the example, the original time series prediction model is first defined as a linear stacked layer model, the LSTM input layer and the Dense output layer are added, and the Adm optimizer is used to compile it. Then, the dimension of the input data X-preprocessed is adjusted to ensure that the LSTM input matches the input data in dimension. Then, the fit function is used to pass real-time numerical values to ensure a more real-time model training and updating process, thereby ensuring the accuracy of time series prediction. Finally, the predicted value obtained by the original time series prediction model is subtracted from the validation set data to obtain the specific time series point deviation value.
[0101] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0102] Based on the same inventive concept, the embodiment of the present application also provides a database anomaly detection device for implementing the database anomaly detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more database anomaly detection device embodiments provided below can refer to the limitations of the database anomaly detection method above, and will not be repeated here.
[0103] In an exemplary embodiment, Figure 6 As shown, a database anomaly detection device 600 is provided, comprising: a query module 601, a preprocessing module 602, a time series prediction module 603 and an anomaly detection module 604, wherein:
[0104] The query module 601 is used to query the active session list and obtain the original data; the active session list is used to record the load data of the database;
[0105] The preprocessing module 602 is used to perform serialization preprocessing on the original data to obtain a preprocessing sequence;
[0106] The timing prediction module 603 is used to perform timing prediction on the preprocessed sequence according to the pre-trained timing prediction model to obtain the timing point deviation value;
[0107] The anomaly detection module 604 is used to perform anomaly detection on the time point deviation value according to the pre-trained target anomaly detection model to obtain anomaly detection results about the database.
[0108] In some embodiments, the preprocessing module 602 is further used to perform data cleaning on the original data to obtain candidate data; and perform data conversion on the candidate data to obtain a preprocessing sequence.
[0109] In some embodiments, the timing prediction module 603 is further used to perform database load analysis on the preprocessing sequence according to the timing prediction model to obtain a database load baseline; and perform residual calculation on the preprocessing sequence according to the database load baseline to obtain a timing point deviation value.
[0110] In some embodiments, the anomaly detection module 604 is further used to perform path length prediction on the time series data corresponding to the time series point deviation value according to the target anomaly detection model to obtain the path length of the time series data;
[0111] Anomaly detection is performed on time series data according to the target anomaly detection model and path length to obtain anomaly detection results.
[0112] In some embodiments, the device also includes: a training module, used to obtain training load data, and preprocess the training load data to obtain training preprocessed data; perform timing prediction on the training preprocessed data according to the original timing prediction model to obtain a timing point deviation value; adjust the model parameters of the original timing prediction model according to the verification data and the timing point deviation value corresponding to the training load data to obtain a timing prediction model.
[0113] In some embodiments, after performing anomaly detection on the timing point deviation value according to the pre-trained target anomaly detection model to obtain the anomaly detection results about the database, the device also includes: an anomaly point analysis module, used to acquire business scenario data; performing anomaly point analysis on the anomaly detection results according to the business scenario data to obtain a target anomaly point set.
[0114] In the above-mentioned database anomaly detection device, the original data is obtained by querying the active session list; the active session list is used to record the load data of the database; the original data is serialized and preprocessed to obtain a preprocessed sequence; the preprocessed sequence is time-series predicted according to the pre-trained time series prediction model to obtain the time series point deviation value; the time series point deviation value is detected according to the pre-trained target anomaly detection model to obtain the anomaly detection result about the database. Therefore, by querying the active session list to obtain the original data, the monitoring item value path is optimized, the number of queries to the business database is greatly reduced, and after the original data is serialized and preprocessed to obtain the preprocessed sequence, the preprocessed sequence is time-series predicted according to the pre-trained time series prediction model, the time series deviation value can be predicted in real time based on the preprocessed sequence, and the time series point deviation value can be detected according to the pre-trained target anomaly detection model, that is, the time series deviation value and anomaly detection are obtained through dynamic time series prediction, which can optimize the alarm threshold and alarm dimension, thereby reducing the subjective dependence on operation and maintenance personnel and achieving a high level of alarm coverage and accuracy.
[0115] Each module in the above-mentioned database anomaly detection device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.
[0116] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 7As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store load data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a database anomaly detection method is implemented.
[0117] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0118] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above-mentioned database anomaly detection method when executing the computer program.
[0119] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned database anomaly detection method are implemented.
[0120] In one embodiment, a computer program product is provided, including a computer program, which implements the steps of the above-mentioned database anomaly detection method when executed by a processor.
[0121] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0122] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.
[0123] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0124] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A database anomaly detection method, characterized in that: The method comprises: Query the active session list to obtain original data; the active session list is used to record the load data of the database; Performing serialization preprocessing on the original data to obtain a preprocessing sequence; Performing time series prediction on the preprocessed sequence according to a pre-trained time series prediction model to obtain a time series point deviation value; Anomaly detection is performed on the time sequence point deviation value according to a pre-trained target anomaly detection model to obtain an anomaly detection result on the database.
2. The method according to claim 1, characterized in that The serialization preprocessing of the original data to obtain a preprocessing sequence includes: Performing data cleaning on the original data to obtain candidate data; Perform data conversion on the candidate data to obtain the preprocessing sequence.
3. The method according to claim 1, characterized in that The step of performing time series prediction on the preprocessed sequence according to the pre-trained time series prediction model to obtain a time series point deviation value includes: Performing database load analysis on the preprocessing sequence according to the time series prediction model to obtain a database load baseline; The residual calculation is performed on the preprocessing sequence according to the database load baseline to obtain the timing point deviation value.
4. The method according to claim 1, characterized in that: The performing anomaly detection on the time sequence point deviation value according to the pre-trained target anomaly detection model to obtain an anomaly detection result on the database includes: Performing path length prediction on the time series data corresponding to the time series point deviation value according to the target anomaly detection model to obtain the path length of the time series data; Anomaly detection is performed on the time series data according to the target anomaly detection model and the path length to obtain the anomaly detection result.
5. The method according to claim 1, characterized in that The method further comprises: Acquiring training load data, and preprocessing the training load data to obtain training preprocessing data; Perform time series prediction on the training preprocessed data according to the original time series prediction model to obtain a time series point deviation value; The model parameters of the original time series prediction model are adjusted according to the verification data corresponding to the training load data and the time series point deviation value to obtain the time series prediction model.
6. The method according to claim 1, characterized in that After performing anomaly detection on the time sequence point deviation value according to the pre-trained target anomaly detection model to obtain an anomaly detection result on the database, the method further includes: Obtain business scenario data; Anomaly point analysis is performed on the anomaly detection result according to the business scenario data to obtain a target anomaly point set.
7. A database anomaly detection device, characterized in that: The device comprises: A query module, used to query an active session list to obtain raw data; the active session list is used to record database load data; A preprocessing module, used for performing serialization preprocessing on the original data to obtain a preprocessing sequence; A timing prediction module, used to perform timing prediction on the preprocessed sequence according to a pre-trained timing prediction model to obtain a timing point deviation value; The anomaly detection module is used to perform anomaly detection on the time point deviation value according to a pre-trained target anomaly detection model to obtain anomaly detection results about the database.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.