Database transaction anomaly detection method and device and electronic equipment

Abstract

The invention relates to a database transaction anomaly detection method and device and electronic equipment. The method comprises the steps that execution information of a database transaction is obtained, and the execution information comprises SQL statements executed by the database transaction on each computing node; constructing a vectorization result for each SQL statement in the execution information, and sequentially arranging all the vectorization results to obtain a vector sequence of database transactions; inputting the vector sequence into a target neural network to enable the target neural network to perform anomaly detection on the database transaction based on the vector sequence, and outputting an anomaly probability of the database transaction; and under the condition that the abnormal probability is greater than a first threshold value, determining that the detection result of the database transaction is abnormal, and displaying the detection result on a monitoring platform. The technical problems that a traditional abnormal transaction detection method lacks adaptability, complex transactions cannot be processed, and manual intervention is needed are solved.

Description

Technical Field

[0001] The present application relates to the technical field of computer system applications, and in particular to a method, device and electronic device for detecting anomalies in database transactions. Background Art

[0002] A database transaction refers to a series of operations performed as a single logical unit of work. These operations are either all successful and make permanent changes to the database, or all fail, and the database will roll back to the state before the transaction started. This feature of transactions ensures the consistency, integrity, and reliability of the database. In modern database management systems, transaction anomaly detection is crucial, especially in high-risk industries such as finance and operators, where any data errors or security issues may lead to serious consequences, such as financial losses and loss of customer trust. Therefore, timely and accurate detection of abnormal behavior in database transactions is of great significance for ensuring system stability, security prevention, and computing resource optimization. Traditional database transaction anomaly detection methods mainly include the following:

[0003] (1) Rule-based approach: By defining specific rules or thresholds to determine whether a transaction is abnormal, this approach is simple and easy to implement, but has limited ability to identify complex abnormal behaviors.

[0004] (2) Statistical analysis: Through statistical analysis of historical data, a model of normal behavior is established and new transactions are compared with it. This method requires a large amount of historical data and is less effective when encountering new and unseen anomalies.

[0005] (3) Traditional machine learning methods: Traditional machine learning algorithms such as decision trees and support vector machines are used to classify and detect transactions. Although these methods can identify certain types of anomalies, they usually require manual parameter adjustment and feature engineering, and are not ideal when faced with complex transaction patterns.

[0006] (4) Expert knowledge base approach: Based on the experience and rules of experts, a knowledge base for transaction anomaly judgment is established. This approach relies on the subjective experience of experts and is difficult to adapt to the rapidly changing database environment. In addition, the maintenance and update costs of the knowledge base are high.

[0007] Although these traditional methods can detect abnormal transactions to a certain extent, they also have obvious limitations, such as lack of adaptability, inability to handle complex transactions, and the need for manual intervention. Summary of the invention

[0008] The present application provides a method, device and electronic device for detecting anomalies of database transactions, so as to solve the technical problems that traditional abnormal transaction detection methods lack adaptability, cannot handle complex transactions and require manual intervention.

[0009] In a first aspect, the present application provides a method for detecting anomalies in database transactions, comprising: obtaining execution information of a database transaction, wherein the execution information includes an SQL statement executed by the database transaction on each computing node; constructing a vectorization result for each SQL statement in the execution information, and arranging all the vectorization results in sequence to obtain a vector sequence of the database transaction; inputting the vector sequence into a target neural network, so that the target neural network performs anomaly detection on the database transaction based on the vector sequence, and outputs an anomaly probability of the database transaction; when the anomaly probability is greater than a first threshold, determining that the detection result of the database transaction is anomaly, and displaying the detection result on a monitoring platform.

[0010] In the second aspect, the present application provides an anomaly detection device for database transactions, including: a first acquisition module, used to obtain execution information of database transactions, wherein the above-mentioned execution information includes the SQL statement executed by the above-mentioned database transaction on each computing node; a construction module, used to construct a vectorization result for each SQL statement in the above-mentioned execution information, and arrange all the above-mentioned vectorization results in sequence to obtain a vector sequence of the above-mentioned database transaction; a detection module, used to input the above-mentioned vector sequence into a target neural network, so that the above-mentioned target neural network performs anomaly detection on the above-mentioned database transaction based on the above-mentioned vector sequence, and outputs the anomaly probability of the above-mentioned database transaction; a first determination module, used to determine that the detection result of the above-mentioned database transaction is anomaly when the above-mentioned anomaly probability is greater than a first threshold value, and display the above-mentioned detection result on a monitoring platform.

[0011] As an optional example, the above-mentioned construction module includes: a first processing unit, used to determine each of the above-mentioned SQL statements as a current SQL statement, and perform the following operations on the above-mentioned current SQL statement: decompose the above-mentioned current SQL statement into multiple tags, and remove useless tags; perform format standardization processing and data standardization processing on all the above-mentioned tags; assign a word vector to each of the above-mentioned tags to obtain a vocabulary, and according to the above-mentioned vocabulary, convert each of the above-mentioned tags into a corresponding word vector; arrange all the above-mentioned word vectors in sequence to obtain the vectorization result of the above-mentioned current SQL statement.

[0012] As an optional example, the target neural network includes a target convolutional neural network and a target recurrent neural network, and the detection module includes: a second processing unit, used to input the vector sequence into the target convolutional neural network, so that the target convolutional neural network performs feature extraction on the vector sequence and outputs a feature vector of the database transaction; a third processing unit, used to input the feature vector into the target recurrent neural network, so that the target recurrent neural network performs time series modeling on the feature vector and outputs the abnormal probability of the database transaction.

[0013] As an optional example, the second processing unit includes: a first processing subunit, used to perform a convolution operation on the vector sequence to obtain local features; a second processing subunit, used to perform a pooling operation on the local features to reduce the dimension of the local features to obtain a pooled feature map; a third processing subunit, used to flatten or perform a dimension conversion on the pooled feature map to obtain a feature vector of the database transaction.

[0014] As an optional example, the third processing unit includes: a fourth processing sub-unit, used to update the hidden state of the feature vector at each time step, and obtain the final hidden state after multiple time step iterations to capture the time dependency of the database transaction; a fifth processing sub-unit, used to output the abnormal probability of the database transaction according to the time dependency.

[0015] As an optional example, the above-mentioned device also includes: a second acquisition module, used to acquire a training data set before inputting the above-mentioned vector sequence into the target neural network; a labeling module, used to label the abnormal results of each database transaction in the above-mentioned training data set to obtain a verification data set; a training module, used to input the above-mentioned training data set into the initial neural network to obtain a result data set, wherein the above-mentioned initial neural network includes an initial convolutional neural network and an initial recurrent neural network; a calculation module, used to calculate the difference between the above-mentioned result data set and the above-mentioned verification data set to obtain a loss value; an adjustment module, used to adjust the parameters of the above-mentioned initial convolutional neural network and the above-mentioned initial recurrent neural network according to the above-mentioned loss value, and recalculate the above-mentioned loss value until the recalculated loss value is less than the second threshold value, so as to obtain the above-mentioned target convolutional neural network and the above-mentioned target recurrent neural network.

[0016] As an optional example, the above-mentioned device also includes: a second determination module, which is used to input the above-mentioned vector sequence into the target neural network so that the above-mentioned target neural network performs anomaly detection on the above-mentioned database transaction based on the above-mentioned vector sequence, and outputs the anomaly probability of the above-mentioned database transaction. When the above-mentioned anomaly probability is less than or equal to the above-mentioned first threshold, it is determined that the detection result of the above-mentioned database transaction is that there is no anomaly, and the above-mentioned detection result is displayed on the above-mentioned monitoring platform.

[0017] In a third aspect, the present application provides a storage medium storing a computer program, wherein the computer program executes the above-mentioned database transaction anomaly detection method when executed by a processor.

[0018] In a fourth aspect, the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-mentioned database transaction anomaly detection method through the computer program.

[0019] The above technical solution provided by the embodiment of the present application has the following advantages compared with the prior art:

[0020] The present application adopts a method of obtaining execution information of database transactions, wherein the execution information includes the SQL statements executed by the database transactions on each computing node; constructing a vectorized result for each SQL statement in the execution information, and arranging all the vectorized results in sequence to obtain a vector sequence of the database transactions; inputting the vector sequence into a target neural network, so that the target neural network performs anomaly detection on the database transactions based on the vector sequence, and outputs an anomaly probability of the database transactions; when the anomaly probability is greater than a first threshold, determining that the detection result of the database transaction is anomaly, and displaying the detection result on a monitoring platform. Since in the above method , by collecting SQL statements and related information executed by database transactions from each computing node, and then constructing vectorized results for each SQL statement, arranging the vectorized results of all SQL statements in execution order to form a vector sequence of database transactions, and then inputting the vector sequence into the target neural network for anomaly detection to obtain the anomaly probability, and judging whether the transaction is abnormal based on the anomaly probability, and finally displaying the anomaly detection results in real time on the monitoring platform, thereby achieving the purpose of reducing manual intervention, improving detection efficiency, more accurately and timely identifying complex abnormal patterns, and improving the reliability of the database system, thereby solving the technical problems that traditional abnormal transaction detection methods lack adaptability, cannot handle complex transactions and require manual intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0023] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0024] Figure 1 is a flowchart of an optional database transaction anomaly detection method according to an embodiment of the present application;

[0025] Figure 2 is an architectural diagram of an optional database transaction anomaly detection method according to an embodiment of the present application;

[0026] Figure 3 is a schematic structural diagram of an optional database transaction anomaly detection device according to an embodiment of the present application;

[0027] Figure 4 is a schematic diagram of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0029] The disclosure below provides many different embodiments or examples to realize the different structures of the present application. In order to simplify the disclosure of the present application, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the present application can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0030] According to a first aspect of an embodiment of the present application, a method for detecting anomalies in a database transaction is provided. Optionally, as follows: Figure 1 As shown, the above method includes:

[0031] S102, obtaining execution information of the database transaction, wherein the execution information includes an SQL statement executed by the database transaction on each computing node;

[0032] S104, constructing a vectorization result for each SQL statement in the execution information, and arranging all the vectorization results in sequence to obtain a vector sequence of the database transaction;

[0033] S106, inputting the vector sequence into the target neural network, so that the target neural network performs anomaly detection on the database transaction based on the vector sequence, and outputs anomaly probability of the database transaction;

[0034] S108: When the abnormality probability is greater than the first threshold, determine that the detection result of the database transaction is abnormal, and display the detection result on the monitoring platform.

[0035] Optionally, in this embodiment, if Figure 2 As shown in the architecture diagram, this method is implemented by combining OMM (Operations and Maintenance Management) management nodes and computing node components. The OMM management node includes a metadata repository, a metadata manager, a data preprocessing module, a target neural network (deep learning neural network) and a monitoring platform. A management agent is installed on each computing node to report the basic information of the database transaction executed on the computing node in real time, that is, the execution information (such as execution time, SQL statement, execution status, etc.) to the management node. The reported data is stored in the metadata repository to form a centralized basic transaction information library. The metadata manager integrates and processes the execution information in the metadata repository according to the preset configuration, and converts the execution information into fine-grained diagnostic data of the transaction. The data preprocessing module cleans and normalizes the integrated diagnostic data, removes useless information, ensures the consistency and comparability of the data, and converts the processed data into a vector form suitable for deep learning neural networks, that is, a vector sequence. The deep learning network receives and preprocesses the vector sequence, outputs the abnormal probability of the database transaction, and determines whether the database transaction is abnormal based on the preset first threshold. The deep learning network is trained with a large amount of labeled historical database transaction data and has learned the normal and abnormal behavior characteristics in database transactions. Finally, the monitoring platform intuitively presents the detection results of the database transaction, which is convenient for users to understand and respond quickly.

[0036] Optionally, in this embodiment, the specific details of the database transaction when it is executed on each computing node are obtained by the management agent on the computing node, including but not limited to SQL statements, execution time, return results, number of records affected, and other information, and stored in the metadata repository. Since SQL statements are usually in text format, deep learning neural networks cannot directly process text data. Therefore, each SQL statement needs to be vectorized by a data preprocessing module first. The vectorization process includes but is not limited to word segmentation, feature extraction, encoding, etc. The vectorization result of each SQL statement is a vector of fixed dimension, which represents the characteristics of the SQL statement. Then the vectorization results of each SQL statement are arranged in order to form a vector sequence. The vector sequence represents the operation process of the entire database transaction and retains the order and context information between SQL statements. The target neural network receives this vector sequence as input. By processing the input vector sequence, it can learn the normal and abnormal patterns in the transaction and generate an output value, namely the abnormal probability, indicating whether the transaction is normal or abnormal. It is usually a value between 0 and 1. The closer to 1, the greater the possibility of abnormality, and the closer to 0, the normal transaction. In the output abnormal probability value, a threshold is set. For example, if the abnormal probability is greater than 0.8, the transaction is considered abnormal. Once the abnormal probability of a transaction is detected to exceed the set threshold, the transaction is considered abnormal and marked. The result will be displayed on the monitoring platform. System administrators or operation and maintenance personnel can take timely measures based on the abnormal warnings in the monitoring platform to conduct detailed analysis or handle abnormal transactions.

[0037] Optionally, in this embodiment, through the automated detection of deep learning neural networks, anomalies in database transactions can be automatically identified without human intervention, reducing the risk of human error and improving detection efficiency, thereby saving a lot of manual error correction and maintenance costs. Deep learning neural networks are particularly good at learning patterns from complex data and can adapt to new transaction types and abnormal behaviors. Compared with traditional rule-based methods, detection based on deep learning neural networks can more accurately identify unknown abnormal situations, especially for complex transaction patterns. It is able to process multiple database transactions in real time and output abnormal probabilities in a timely manner, so that potential anomalies can be quickly discovered during the execution of transactions, rather than after-the-fact analysis, reducing response time and taking repair measures quickly.

[0038] As an optional example, building a vectorized result for each SQL statement in the execution information includes:

[0039] Determine each SQL statement as the current SQL statement and perform the following operations on the current SQL statement:

[0040] Break the current SQL statement into multiple tags and remove useless tags;

[0041] Perform format standardization and data normalization on all tags;

[0042] Assign a word vector to each token to get the vocabulary, and convert each token into the corresponding word vector according to the vocabulary;

[0043] Arrange all word vectors in order to obtain the vectorization result of the current SQL statement.

[0044] Optionally, in this embodiment, the SQL statement is usually composed of multiple components, such as keywords, table names, column names, operators, constant values, conditions, etc. In order to convert the SQL statement into a format that can be processed by the machine, the SQL statement needs to be first disassembled, that is, word segmentation, to obtain multiple tags (tokens). The SQL statement may contain some parts that are irrelevant to feature extraction, such as comments, redundant spaces, etc. In order to improve the detection efficiency and accuracy of the target neural network, useless tags need to be removed. In order to eliminate the differences caused by different styles in the SQL statement, the SQL statement needs to be uniformly formatted. For example, the keywords are uniformly converted to uppercase, the table names and column names are uniformly capitalized, and the style of each SQL statement is guaranteed to be consistent, which can avoid feature recognition errors caused by format differences. For numerical data (such as constant numbers) in SQL statements, standardization may be required. For example, digital values ​​such as 30 and 1000 may need to be converted to a unified scale so that the target neural network can better learn their relationship. Each tag in the SQL statement (such as keywords, table names, column names, etc.) needs to be mapped to a word vector, which is a multidimensional real number vector that can capture the semantic information of the tag. Build a vocabulary that contains all the tokens that appear in the SQL statement and assign a unique vector to each token. Use the previously built vocabulary to map the token of each SQL statement to the corresponding word vector. Arrange the word vectors corresponding to all tokens in the order in the SQL statement to form a vectorized result. This vectorized result represents the semantic information of the entire SQL statement and retains the structure and execution order of the SQL statement.

[0045] Optionally, in this embodiment, by standardization and removal of useless tags, the format of SQL statements becomes unified, and redundant information is removed, thereby making the input data clearer and more meaningful, which helps to improve the detection efficiency and accuracy of the neural network. By representing SQL tags through word vectors, the originally complex symbolic data can be converted into vectors of fixed dimensions, which can effectively capture the semantic relationship between tags.

[0046] As an optional example, the target neural network includes a target convolutional neural network and a target recurrent neural network. The vector sequence is input into the target neural network so that the target neural network performs anomaly detection on the database transaction based on the vector sequence. The output anomaly probability of the database transaction includes:

[0047] Inputting the vector sequence into the target convolutional neural network so that the target convolutional neural network extracts features from the vector sequence and outputs feature vectors of database transactions;

[0048] The feature vector is input into the target recurrent neural network so that the target recurrent neural network performs time series modeling on the feature vector and outputs the abnormal probability of the database transaction.

[0049] Optionally, in this embodiment, the vector sequence of the database transaction is used as the input of the target neural network, and the input sequence contains the grammatical information and structural features of the SQL statement, and forms a vector for neural network analysis after data preprocessing. The vector sequence is first input into the target convolutional neural network (CNN), which is good at automatically extracting local features from the input data. When processing the vector sequence, the convolutional neural network scans the input sequence through the convolution operation and captures the local rules and patterns. The local features after convolution are reduced in dimension through the pooling layer to reduce the computational complexity while retaining the most important features. Finally, the output feature vector represents the various features of the SQL statement in the local range, providing basic information for subsequent time series modeling. The feature vector extracted by the target convolutional neural network is input into the target recurrent neural network (RNN), which is particularly good at processing data with time series properties and can capture the long-term dependencies between elements in the input sequence. The LSTM (Long Short-Term Memory) unit in the target recurrent neural network can effectively handle dependencies within a long span, avoiding the gradient vanishing problem encountered by the traditional target recurrent neural network in a long sequence. The LSTM unit determines how to update and memorize the state through its internal gating mechanism (such as input gate, forget gate and output gate). In this embodiment, the LSTM unit will capture the temporal dependencies of SQL statements in the transaction execution process by processing the feature vector, such as continuous database operations, the time sequence of transaction execution, and possible abnormal modes in the transaction execution process. After the target recurrent neural network models the time series of the feature vector, the final network output is an abnormal probability value, which represents the possibility of whether the current database transaction is abnormal. If the abnormal probability is greater than the set threshold, the database transaction is marked as "abnormal", otherwise, the database transaction is considered "normal". The abnormal detection results will be further passed to the monitoring platform for subsequent processing or alarm.

[0050] Optionally, in this embodiment, by combining convolutional neural networks and recurrent neural networks, the neural network can automatically extract features from transaction data and identify potential abnormal patterns in sequence data. Compared with traditional rule-based or statistical methods, deep learning neural networks can more flexibly and efficiently identify complex and unknown abnormal patterns. Convolutional neural networks can effectively extract local and spatial features from database transaction data, such as certain fixed patterns or local correlations in database transactions. Recurrent neural networks can capture the order and long-term dependencies between database transactions, making up for the shortcomings of convolutional neural networks in processing sequence information. This combination can more accurately identify abnormal patterns because it can simultaneously consider the local features and time series features of the data, so that the network can capture the complex structure of SQL statements and the time dynamics of transaction execution, thereby providing comprehensive information for anomaly detection. The local feature extraction of convolutional neural networks and the time series modeling of recurrent neural networks can work together to improve the model's ability to recognize various transaction abnormal patterns. Compared with traditional methods, deep learning neural networks can better handle noise, nonlinear relationships and high-dimensional information in data.

[0051] As an optional example, the vector sequence is input into the target convolutional neural network so that the target convolutional neural network extracts features from the vector sequence, and the feature vector of the output database transaction includes:

[0052] Perform convolution operation on the vector sequence to obtain local features;

[0053] Perform a pooling operation on the local features to reduce the dimension of the local features and obtain a pooled feature map;

[0054] Flatten or transform the dimension of the pooled feature map to obtain the feature vector of the database transaction.

[0055] Optionally, in this embodiment, the vector sequence is input into the target convolutional neural network, and the target convolutional neural network will extract features of database transactions based on the vector sequence. Specifically, the vector sequence is used as input and passes through the convolution operation of the convolution layer. The convolution operation is achieved by sliding the convolution kernel on the input sequence. Each convolution kernel extracts local and important features in the sequence, usually in the adjacent part of the sequence. The function of the convolution layer is to "scan" the input sequence by sliding the convolution kernel to find valuable patterns and information in the data, thereby extracting local features. The local features extracted by the convolution layer are usually high-dimensional and contain a lot of redundant information. Therefore, it is necessary to reduce the dimension through the pooling operation. The pooling layer (maximum pooling or average pooling) reduces the local features output by the convolution layer. The goal of the pooling operation is to reduce the spatial size of the local features, retain the most significant features, and avoid information overload. The pooling layer not only helps to reduce the amount of calculation, but also can effectively reduce overfitting and improve the generalization ability of the model. After the pooling operation, the obtained feature map is still two-dimensional. However, the subsequent layers in the neural network usually require one-dimensional input, so the pooled feature map needs to be flattened or dimensionally converted. The flattening operation expands the pooled feature map into a one-dimensional vector, and each element represents an information point in the feature map. In this way, the final feature vector of the database transaction is obtained. This feature vector contains all the important features of the SQL statement. After convolution and pooling operations, it has been compressed and useful and important information has been extracted. Finally, the feature vector after convolution, pooling and flattening will be used as the feature representation of the database transaction.

[0056] Optionally, in this embodiment, the convolutional neural network can effectively extract the most meaningful features from the vector sequence of SQL statements, and these features will be used for subsequent database transaction anomaly detection, thereby improving the efficiency and accuracy of detection and the generalization ability of the model.

[0057] As an optional example, the feature vector is input into the target recurrent neural network so that the target recurrent neural network performs time series modeling on the feature vector, and the abnormal probability of the output database transaction includes:

[0058] The hidden state of the feature vector is updated at each time step, and after multiple time step iterations, the final hidden state is obtained to capture the temporal dependency of database transactions;

[0059] Output the abnormal probability of database transactions based on time dependencies.

[0060] Optionally, in this embodiment, in the target neural network, a recurrent neural network is used to perform time series modeling on the feature vector to capture the potential time dependency in the database transaction, and finally output the abnormal probability of the database transaction. Specifically, the feature vector extracted by the convolutional neural network will be passed as input to the target recurrent neural network, and the recurrent neural network updates its hidden state at each time step by gradually processing each feature vector in the input sequence. The hidden state is an internal representation of the recurrent neural network, which is used to save the context information of the sequence at that moment. At each time step, the recurrent neural network uses the current input (i.e., the feature vector at the current moment) and the hidden state of the previous time step to calculate a new hidden state. As the time step advances, the recurrent neural network gradually updates the hidden state through iterations of multiple time steps. Each iteration combines the hidden state of the previous moment with the current feature vector to capture the dependencies and context information across time steps in the database transaction. This iterative update allows the recurrent neural network to capture long-term dependencies in the time series. Temporal dependency refers to the execution order and dependency of different SQL statements in a transaction. There may be multiple SQL statements in a transaction. These statements are executed sequentially in time, and subsequent operations may depend on previous operations. The recurrent neural network can capture these temporal relationships through its memory structure and time step iteration mechanism. The final output of the recurrent neural network is an abnormality probability value, indicating whether there is an abnormality in the database transaction. After multiple iterations of time steps, the recurrent neural network will synthesize the hidden states of each time step and output the abnormality probability of the transaction based on the learned time dependency and pattern.

[0061] Optionally, in this embodiment, the recurrent neural network can effectively capture the temporal dependencies in time series data. In database transactions, the execution order and dependencies of SQL statements are usually time-dependent, and the execution results of certain operations will affect subsequent operations. Modeling temporal dependencies through recurrent neural networks can help detect possible abnormal behaviors and time-related errors in transactions, thereby improving the ability to detect complex abnormal transactions.

[0062] As an optional example, before inputting the vector sequence into the target neural network, the method further includes:

[0063] Get the training dataset;

[0064] Mark out the abnormal results of each database transaction in the training data set to obtain the validation data set;

[0065] Inputting the training data set into the initial neural network to obtain a result data set, wherein the initial neural network includes an initial convolutional neural network and an initial recurrent neural network;

[0066] Calculate the difference between the result dataset and the validation dataset to get the loss value;

[0067] The parameters of the initial convolutional neural network and the initial recurrent neural network are adjusted according to the loss value, and the loss value is recalculated until the recalculated loss value is less than the second threshold, thereby obtaining the target convolutional neural network and the target recurrent neural network.

[0068] Optionally, in this embodiment, when training a neural network for database transaction anomaly detection, the key steps involved include data set acquisition, model training, loss calculation, and model optimization. Specifically, the training data set refers to a sample data set used to train the initial neural network. The training data set consists of historical transaction records. Each transaction record includes an executed SQL statement and its execution context information. In order to effectively train the initial neural network, these data need to contain samples of normal transactions and abnormal transactions, and each sample must be labeled to clearly identify whether it is an abnormal transaction. Through labeling, the training data set can be divided into two parts: a training data set and a validation data set. The training data set is used to train the neural network, and the validation data set is used to evaluate the performance of the neural network. The initial neural network includes an initial convolutional neural network and an initial recurrent neural network. Each transaction sample in the training data set will be converted into a vector form and input into the initial neural network. The initial neural network will output the abnormal probability of each database transaction as a preliminary prediction result. During the training process, it is necessary to evaluate the difference between the prediction result of the neural network and the actual label. This difference is usually quantified by a loss function. According to the labeling results in the validation data set, the difference between the predicted result and the actual label of each transaction is calculated to obtain a loss value. According to the calculated loss value, the optimization algorithm (such as gradient descent method) is used to adjust the parameters of the neural network to minimize the loss value. When the loss value is less than the set threshold, the training process stops and the neural network is considered to have converged. At this time, the obtained neural network is the target convolutional neural network and target recurrent neural network after the training is completed.

[0069] As an optional example, after inputting the vector sequence into the target neural network so that the target neural network performs anomaly detection on the database transaction based on the vector sequence and outputs the anomaly probability of the database transaction, the method further includes:

[0070] When the abnormality probability is less than or equal to the first threshold, the detection result of the database transaction is determined to be free of abnormality, and the detection result is displayed on the monitoring platform.

[0071] Optionally, in this embodiment, when the abnormal probability is less than or equal to the first threshold, it indicates that the database transaction conforms to the normal behavior pattern, so the detection result is "normal" and no abnormal alarm is triggered. The system will display this information on the monitoring platform, usually marked as "normal" or "no abnormality", and the results displayed on the monitoring platform can help database administrators or operators confirm the normality of the transaction without further intervention.

[0072] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0073] According to another aspect of the embodiment of the present application, a database transaction anomaly detection device is also provided. Figure 3 As shown, including:

[0074] A first acquisition module 302 is used to acquire execution information of a database transaction, wherein the execution information includes an SQL statement executed by the database transaction on each computing node;

[0075] A construction module 304 is used to construct a vectorization result for each SQL statement in the execution information, and arrange all the vectorization results in sequence to obtain a vector sequence of the database transaction;

[0076] A detection module 306, configured to input the vector sequence into a target neural network, so that the target neural network performs anomaly detection on the database transaction based on the vector sequence, and outputs anomaly probability of the database transaction;

[0077] The first determination module 308 is used to determine that the detection result of the database transaction is abnormal when the abnormal probability is greater than a first threshold, and display the detection result on the monitoring platform.

[0078] It should be noted that the first acquisition module 302 in this embodiment can be used to execute step S102 in the embodiment of the present application, the construction module 304 in this embodiment can be used to execute step S104 in the embodiment of the present application, the detection module 306 in this embodiment can be used to execute step S106 in the embodiment of the present application, and the first determination module 308 in this embodiment can be used to execute step S108 in the embodiment of the present application.

[0079] As an optional example, the building blocks include:

[0080] The first processing unit is used to determine each SQL statement as a current SQL statement, and perform the following operations on the current SQL statement:

[0081] Decompose the current SQL statement into multiple tags and remove useless tags; perform format standardization and data standardization on all tags;

[0082] Assign a word vector to each token to get the vocabulary, and convert each token into the corresponding word vector according to the vocabulary;

[0083] Arrange all word vectors in order to obtain the vectorization result of the current SQL statement.

[0084] As an optional example, the target neural network includes a target convolutional neural network and a target recurrent neural network, and the detection module includes:

[0085] a second processing unit, configured to input the vector sequence into a target convolutional neural network, so that the target convolutional neural network performs feature extraction on the vector sequence and outputs a feature vector of the database transaction;

[0086] The third processing unit is used to input the feature vector into the target recurrent neural network so that the target recurrent neural network performs time series modeling on the feature vector and outputs the abnormal probability of the database transaction.

[0087] As an optional example, the second processing unit includes:

[0088] A first processing subunit is used to perform a convolution operation on the vector sequence to obtain a local feature;

[0089] The second processing subunit is used to perform a pooling operation on the local features to reduce the dimension of the local features and obtain a pooled feature map;

[0090] The third processing subunit is used to flatten or perform dimension conversion on the pooled feature map to obtain a feature vector of the database transaction.

[0091] As an optional example, the third processing unit includes:

[0092] A fourth processing subunit is used to update the hidden state of the feature vector at each time step, and obtain the final hidden state after multiple time step iterations to capture the time dependency of the database transaction;

[0093] The fifth processing subunit is used to output the abnormal probability of the database transaction according to the time dependency.

[0094] As an optional example, the above device further includes:

[0095] A second acquisition module is used to acquire a training data set before inputting the vector sequence into the target neural network;

[0096] The annotation module is used to annotate the abnormal results of each database transaction in the training data set to obtain the verification data set;

[0097] A training module, used for inputting a training data set into an initial neural network to obtain a result data set, wherein the initial neural network includes an initial convolutional neural network and an initial recurrent neural network;

[0098] The calculation module is used to calculate the difference between the result data set and the verification data set to obtain the loss value;

[0099] The adjustment module is used to adjust the parameters of the initial convolutional neural network and the initial recurrent neural network according to the loss value, and recalculate the loss value until the recalculated loss value is less than the second threshold, so as to obtain the target convolutional neural network and the target recurrent neural network.

[0100] As an optional example, the above device further includes:

[0101] The second determination module is used to input the vector sequence into the target neural network so that the target neural network performs anomaly detection on the database transaction based on the vector sequence, and after outputting the anomaly probability of the database transaction, when the anomaly probability is less than or equal to the first threshold, determine that the detection result of the database transaction is that there is no anomaly, and display the detection result on the monitoring platform.

[0102] For other examples of this embodiment, please refer to the above examples and will not be repeated here.

[0103] Figure 4 is a schematic diagram of an optional electronic device according to an embodiment of the present application, such as Figure 4 As shown, it includes a processor 402, a communication interface 404, a memory 406 and a communication bus 408, wherein the processor 402, the communication interface 404 and the memory 406 communicate with each other through the communication bus 408, wherein,

[0104] Memory 406, used to store computer programs;

[0105] The processor 402 is used to implement the following steps when executing the computer program stored in the memory 406:

[0106] Obtaining execution information of the database transaction, wherein the execution information includes the SQL statement executed by the database transaction on each computing node;

[0107] Build a vectorized result for each SQL statement in the execution information, and arrange all vectorized results in sequence to obtain a vector sequence of database transactions;

[0108] Inputting the vector sequence into the target neural network, so that the target neural network performs anomaly detection on the database transaction based on the vector sequence and outputs the anomaly probability of the database transaction;

[0109] When the abnormality probability is greater than the first threshold, the detection result of the database transaction is determined to be abnormal, and the detection result is displayed on the monitoring platform.

[0110] Optionally, in this embodiment, the communication bus may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 The communication interface is used for communication between the above electronic device and other devices.

[0111] The memory may include RAM, or may include non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0112] As an example, the memory 406 may include, but is not limited to, the first acquisition module 302, the construction module 304, the detection module 306, and the first determination module 308 in the abnormality detection device for database transactions. In addition, other module units in the abnormality detection device for database transactions may also be included but are not limited to, which will not be repeated in this example.

[0113] The above-mentioned processor can be a general-purpose processor, which can include but not be limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; it can also be DSP (Digital Signal Processing), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0114] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0115] It can be understood by those skilled in the art that Figure 4 The structure shown is for illustration only. The device for implementing the above-mentioned database transaction anomaly detection method may be a terminal device, which may be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, and other terminal devices. Figure 4 The structure of the electronic device is not limited. For example, the electronic device may also include Figure 4 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 4 Different configurations shown.

[0116] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which can include: a flash drive, ROM, RAM, a magnetic disk or an optical disk, etc.

[0117] According to another aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program executes the steps in the above-mentioned method for detecting anomalies of database transactions when executed by a processor.

[0118] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0119] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0120] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0121] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0122] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0123] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0124] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0125] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for detecting anomalies in database transactions, characterized in that: include: Acquire execution information of a database transaction, wherein the execution information includes an SQL statement executed by the database transaction on each computing node; Constructing a vectorization result for each SQL statement in the execution information, and arranging all the vectorization results in sequence to obtain a vector sequence of the database transaction; Inputting the vector sequence into a target neural network, so that the target neural network performs anomaly detection on the database transaction based on the vector sequence and outputs anomaly probability of the database transaction; When the abnormality probability is greater than a first threshold, the detection result of the database transaction is determined to be abnormal, and the detection result is displayed on a monitoring platform.

2. The method according to claim 1, characterized in that The constructing a vectorized result for each SQL statement in the execution information includes: Determine each of the SQL statements as a current SQL statement, and perform the following operations on the current SQL statement: Decomposing the current SQL statement into multiple tags and removing useless tags; Performing format standardization and data standardization on all the tags; Assigning a word vector to each of the tokens to obtain a vocabulary, and converting each of the tokens into a corresponding word vector according to the vocabulary; All the word vectors are arranged in order to obtain the vectorization result of the current SQL statement.

3. The method according to claim 1, characterized in that The target neural network includes a target convolutional neural network and a target recurrent neural network. The step of inputting the vector sequence into the target neural network so that the target neural network performs anomaly detection on the database transaction based on the vector sequence and outputting anomaly probability of the database transaction includes: Inputting the vector sequence into the target convolutional neural network so that the target convolutional neural network performs feature extraction on the vector sequence and outputs a feature vector of the database transaction; The feature vector is input into the target recurrent neural network so that the target recurrent neural network performs time series modeling on the feature vector and outputs the abnormal probability of the database transaction.

4. The method according to claim 3, characterized in that Inputting the vector sequence into the target convolutional neural network so that the target convolutional neural network performs feature extraction on the vector sequence and outputting the feature vector of the database transaction includes: Performing a convolution operation on the vector sequence to obtain local features; Performing a pooling operation on the local features to reduce the dimension of the local features and obtain a pooled feature map; The pooled feature map is flattened or dimensionally transformed to obtain a feature vector of the database transaction.

5. The method according to claim 3, characterized in that: The step of inputting the feature vector into the target recurrent neural network so that the target recurrent neural network performs time series modeling on the feature vector and outputting the abnormal probability of the database transaction includes: updating the hidden state of the feature vector at each time step, and obtaining a final hidden state after multiple time step iterations to capture the time dependency of the database transaction; The abnormal probability of the database transaction is output according to the time dependency.

6. The method according to claim 3, characterized in that Before inputting the vector sequence into the target neural network, the method further includes: Get the training dataset; Annotate the abnormal results of each database transaction of the training data set to obtain a verification data set; Inputting the training data set into an initial neural network to obtain a result data set, wherein the initial neural network includes an initial convolutional neural network and an initial recurrent neural network; Calculate the difference between the result data set and the verification data set to obtain a loss value; The parameters of the initial convolutional neural network and the initial recurrent neural network are adjusted according to the loss value, and the loss value is recalculated until the recalculated loss value is less than a second threshold, thereby obtaining the target convolutional neural network and the target recurrent neural network.

7. The method according to claim 1, characterized in that After inputting the vector sequence into the target neural network so that the target neural network performs anomaly detection on the database transaction based on the vector sequence and outputs anomaly probability of the database transaction, the method further includes: When the abnormality probability is less than or equal to the first threshold, the detection result of the database transaction is determined to be free of abnormality, and the detection result is displayed on the monitoring platform.

8. A database transaction anomaly detection device, characterized in that: include: A first acquisition module, configured to acquire execution information of a database transaction, wherein the execution information includes an SQL statement executed by the database transaction on each computing node; A construction module, used to construct a vectorization result for each SQL statement in the execution information, and arrange all the vectorization results in sequence to obtain a vector sequence of the database transaction; A detection module, used for inputting the vector sequence into a target neural network, so that the target neural network performs anomaly detection on the database transaction based on the vector sequence, and outputs anomaly probability of the database transaction; The first determination module is used to determine that the detection result of the database transaction is abnormal when the abnormal probability is greater than a first threshold, and display the detection result on a monitoring platform.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images