Database monitoring and early warning method and system
Database state modeling is solved through recurrent neural network (RNN), and the problem that database state changes in the existing technology is not included, continuous monitoring and accurate prediction of database state are achieved, and early warning effect is improved.
Patent Information
- Application Number
- CN202510587983.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-08
- Publication Date
- 2025-08-26
AI Technical Summary
Existing database monitoring and early warning methods cannot effectively include various factors in the state of the database system, resulting in poor prediction and early warning effects.
Recurrent neural network (RNN) is used to model database states. By establishing a sequence model between the current state of the database and the next state, continuous monitoring and prediction of database states are realized, and early warning is made when the state changes exceed expectations.
It realizes continuous monitoring of database status and more accurate prediction, and can conduct timely early warnings, improving the effectiveness of database monitoring and early warning.
Smart Images
Figure CN120540931A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of AI applications and database management technology, and more particularly to a database monitoring and early warning method and system. Background Art
[0002] As foundational software, databases serve as data infrastructure in numerous fields. The importance of databases has made their monitoring and early warning capabilities increasingly crucial. Timely awareness of database status and prediction of system state changes based on current operations have been the subject of research in the database field for decades. Currently, database monitoring and early warning techniques primarily utilize execution plans for database operations, combined with information such as database or underlying operating system resource status, for quantitative analysis and prediction. However, due to workload constraints, these techniques often fail to encompass the full range of factors that influence database system state changes, resulting in suboptimal prediction and early warning results. Summary of the Invention
[0003] The technical task of the present invention is to address the above shortcomings and provide a database monitoring and early warning method and system, which can realize the modeling between various database operations and state transitions, thereby realizing continuous monitoring of the database state and more effective and accurate continuous prediction, and issuing early warnings when the database state changes beyond expectations.
[0004] The technical solution adopted by the present invention to solve its technical problem is:
[0005] A database monitoring and early warning method uses a recurrent neural network (RNN) to implement artificial intelligence modeling of the current state of the database and the transition from user request to the next state. Database monitoring, state prediction, and early warning are implemented based on the recurrent neural network (RNN); the method comprises:
[0006] Establish a recurrent neural network for database monitoring and early warning: specify the state h in the database for the time period t t Its previous time period state h t-1 Establish an input operation based on the time period x t The sequence model is used to reflect the impact of user input on the database state during the continuous operation of data;
[0007] The database runs a recurrent neural network prediction engine. When a user request is received, the prediction engine's reasoning process is triggered and the database executes the user input operation x in the time period. t Enter the actual time period state after the request t ,The state of this time period can be used to iteratively train the recurrent neural network prediction model;
[0008] Run the recurrent neural network prediction model: configure the trained recurrent neural network model to the inference engine, and perform continuous inference calculations based on the status of the database in real time. t-1 And user input x t Predict the next state h of the database t0 ; When the database prediction vector h t0 When a value in the parameter exceeds the preset threshold, an alarm will be activated.
[0009] Furthermore, the establishment of a recurrent neural network for database monitoring and early warning,
[0010] In database monitoring and early warning modeling, the time axis is first divided into segments of configurable size. The state of the database and related resources in the tth time period is h t , the sum of user inputs in the tth time period is x t , then the formal formula of the recurrent neural network is:
[0011] h t =f(W h h t-1 +W x x t +b);
[0012] Among them, W h is the weight matrix of the state at the previous moment;
[0013] W x is the weight matrix of the current input;
[0014] b is the bias term;
[0015] f is the activation function;
[0016] W h 、W x It is the parameter to be learned in the training of recurrent neural network (RNN).
[0017] Furthermore, the parameter W h 、W x , b, and f are all designed and initialized using existing recurrent neural network model theories and techniques.
[0018] Furthermore, the state of the database and related resources in a specified time period h t The state is represented by a vector containing database system state information, including disk IO times, disk IO data volume, number of read records, number of written records, buffer resource usage, memory usage, CPU usage, concurrent work coroutine / thread usage, logical lock / physical lock usage, etc.
[0019] Furthermore, the user input operation x within the specified time period t , the user's query, write, including DDL, system parameter change and other requests and request objects are defined as their own vectors through feature engineering to participate in the calculation, and the sum of user inputs in the time period is obtained.
[0020] Furthermore, the database runs a recurrent neural network prediction engine, which runs as a component in the database; when a user request arrives at the database, the database kernel responds to the user's request and passes a copy of the request to the prediction engine, triggering the prediction engine's reasoning process.
[0021] Furthermore, the running recurrent neural network prediction model,
[0022] On another data path, the database executes the user input x synchronously t , so that the database actually enters the next state h t , this new state h t It can be used for continued learning of the prediction model and advance the parameters of the prediction formula from the previous state to the next state.
[0023] The present invention also claims a database monitoring and early warning system, which implements database monitoring, status prediction and early warning based on a recurrent neural network (RNN), comprising:
[0024] Model building module, used to build recurrent neural networks for database monitoring and early warning;
[0025] The prediction engine trigger module runs the recurrent neural network prediction engine in the database. When a user request is received, the prediction engine's reasoning process is triggered.
[0026] The prediction model operation module is used to implement the operation of the recurrent neural network prediction model, perform continuous reasoning calculations based on the status of the database in real time, and trigger an alarm when a value in the predicted state vector exceeds a preset threshold;
[0027] The system realizes database monitoring and early warning through the above method.
[0028] The present invention also claims protection for a database monitoring and early warning implementation device, comprising: at least one memory and at least one processor;
[0029] The at least one memory is configured to store a machine-readable program;
[0030] The at least one processor is configured to call the machine-readable program to implement the above method.
[0031] The present invention also claims protection for a computer-readable medium having computer instructions stored thereon, which are capable of implementing the above method when executed by a processor.
[0032] Compared with the prior art, the database monitoring and early warning method and system of the present invention have the following beneficial effects:
[0033] The present invention applies the recurrent neural network model in the field of database monitoring and early warning, and can realize the fusion prediction of the current state of the database and the database operation in the field of database state prediction, thereby achieving better prediction effect, realizing effective monitoring and early warning of the database, and realizing continuous monitoring, continuous early warning and continuous model training during operation. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 Schematic diagram of a database running an RNN prediction engine provided by one embodiment of the present invention;
[0035] Figure 2 This is a logical diagram of a prediction engine running an RNN model provided by one embodiment of the present invention. DETAILED DESCRIPTION
[0036] The present invention will be further described below with reference to specific embodiments.
[0037] An embodiment of the present invention provides a database monitoring and early warning method, which adopts a recurrent neural network (RNN) to implement artificial intelligence modeling of the current state of the database and the user request to the next state, and implements database monitoring, state prediction and early warning based on the recurrent neural network (RNN).
[0038] The implementation of this method includes:
[0039] Establish a recurrent neural network for database monitoring and early warning: specify the state h in the database for the time period t t Its previous time period state h t-1 Establish an input operation based on the time period x t The sequence model is used to reflect the impact of user input on the database state during the continuous operation of data;
[0040] The database runs a recurrent neural network prediction engine. When a user request is received, the prediction engine's reasoning process is triggered and the database executes the user input operation x in the time period. t Enter the actual time period state after the request t ,The state of this time period can be used to iteratively train the recurrent neural network prediction model;
[0041] Run the recurrent neural network prediction model: configure the trained recurrent neural network model to the inference engine, and perform continuous inference calculations based on the status of the database in real time. t-1 And user input x t Predict the next state h of the database t0 ; When the database prediction vector h t0 When a value in the parameter exceeds the preset threshold, an alarm will be activated.
[0042] RNNs (Recurrent Neural Networks) are a type of neural network model with "memory" capabilities. They can process sequential data and retain information between time steps. Unlike traditional feedforward neural networks, RNNs have a feedback structure that enables them to pass information through a loop of hidden states, thereby capturing dependencies within time series. RNNs were originally designed to process time series data, such as in speech recognition, natural language processing (NLP), and video analysis. The data in these tasks often has a temporal nature, meaning that subsequent inputs depend on previous inputs. RNNs are able to capture these dependencies through their hidden states. RNNs are characterized by recurrent connections between neurons, which enables them to process input sequences of varying lengths. The output at each time step depends not only on the current input but also on the hidden state (i.e., memory) of the previous time step. RNNs use a hidden state to store relevant information between time steps. This hidden state changes with the input at each time step and is passed to the next step, helping the model remember previous information.
[0043] Combined with attachment Figure 1 and 2 , the specific implementation process of this method is as follows:
[0044] 1. Establish a recurrent neural network (RNN) for database monitoring and early warning.
[0045] The formal formula of RNN is:
[0046] h t =f(W h h t-1 +W x x t +b);
[0047] Among them, h t is the state at time t;
[0048] x t is the input at time t;
[0049] W h is the weight matrix of the state at the previous moment;
[0050] W x is the weight matrix of the current input;
[0051] b is the bias term;
[0052] f is the activation function.
[0053] In database monitoring and early warning modeling, the time axis is first divided into segments of configurable size. The state of the database and related resources in the tth time period is h t This state can be represented by a vector, which includes information about the database system status, such as the number of disk IO times, disk IO data volume, number of read records, number of written records, buffer resource usage, memory usage, CPU usage, concurrent work coroutine / thread usage, and logical / physical lock usage. t It is the sum of user inputs within t time periods. In this embodiment, user queries, writes, including DDL, system parameter changes and other requests and request objects are defined as their own vectors through feature engineering to participate in the calculation. Other parameters W h 、W x , b, and f are designed and initialized using existing RNN model theory and technology. h 、W x It is the parameter to be learned in RNN training.
[0054] In this way, the previous state h of the database can be t-1 and the next state h t Establish an input operation x t This model reflects the process of data continuous operation, when the user inputs x t Impact on the database status. And because x t The effect of the operation will be superimposed on the database status of the previous period h t-1 This theoretically ensures the effectiveness of the model.
[0055] 2. The database runs the prediction engine to achieve continuous prediction and early warning.
[0056] like Figure 1 As shown in Figure 1, the recurrent neural network prediction engine runs as a component in the database. When a user request reaches the database, the database kernel responds to the user's request and passes a copy of the request to the prediction engine, triggering the prediction engine's reasoning process. t Enter the actual state after the request t , this state can be used to iteratively train the prediction model.
[0057] 3. Run the prediction engine of the RNN model.
[0058] like Figure 2 As shown, after the RNN model is trained, it is configured to the inference engine to perform continuous inference calculations based on the status of the database in real time. t-1 And user input x t Predict the next state h of the database t0 When the database prediction vector h t0 When a value in exceeds the preset threshold, an alarm will be triggered. On another data path, the database synchronously executes the user input x t , so that the database actually enters the next state h t , this new state h t It can be used for continued learning of the prediction model and advance the parameters of the prediction formula from the previous state to the next state.
[0059] The embodiment of the present invention further provides a database monitoring and early warning system, which implements database monitoring, status prediction and early warning based on a recurrent neural network (RNN), including:
[0060] The model building module is used to build a recurrent neural network for database monitoring and early warning. t Its previous time period state h t-1 Establish an input operation based on the time period x t The sequence model is used to reflect the impact of user input on the database state during the continuous operation of data.
[0061] The prediction engine trigger module runs the recurrent neural network prediction engine in the database. When a user request is received, the inference process of the prediction engine is triggered and the database executes the user input operation x in the time period. t Enter the actual time period state after the request t ,The state of this time period can be used to iteratively train the recurrent neural network prediction model.
[0062] The prediction model operation module is used to implement the operation of the recurrent neural network prediction model. The trained recurrent neural network model is configured into the inference engine, and continuous inference calculation is performed in real time according to the status of the database. t-1 And user input x t Predict the next state h of the database t0 ; When the database prediction vector h t0 When a value in the parameter exceeds the preset threshold, an alarm will be activated.
[0063] The system implements database monitoring and early warning through the database monitoring and early warning method described in the above embodiment. The specific implementation is as follows:
[0064] 1. Establish a recurrent neural network (RNN) for database monitoring and early warning.
[0065] The formal formula of RNN is:
[0066] h t =f(W h h t-1 +W x x t +b);
[0067] Among them, h t is the state at time t;
[0068] x t is the input at time t;
[0069] W h is the weight matrix of the state at the previous moment;
[0070] W x is the weight matrix of the current input;
[0071] b is the bias term;
[0072] f is the activation function.
[0073] In database monitoring and early warning modeling, the time axis is first divided into segments of configurable size. The state of the database and related resources in the tth time period is h t This state can be represented by a vector, which includes information about the database system status, such as the number of disk IO times, disk IO data volume, number of read records, number of written records, buffer resource usage, memory usage, CPU usage, concurrent work coroutine / thread usage, and logical / physical lock usage. t It is the sum of user inputs within t time periods. In this embodiment, user queries, writes, including DDL, system parameter changes and other requests and request objects are defined as their own vectors through feature engineering to participate in the calculation. Other parameters W h 、W x , b, and f are designed and initialized using existing RNN model theory and technology. h 、W x It is the parameter to be learned in RNN training.
[0074] In this way, the previous state h of the database can be t-1 and the next state h t Establish an input operation x t This model reflects the process of data continuous operation, when the user inputs x t Impact on the database status. And because xt The effect of the operation will be superimposed on the database status of the previous period h t-1 This theoretically ensures the effectiveness of the model.
[0075] 2. The database runs the prediction engine to achieve continuous prediction and early warning.
[0076] The recurrent neural network prediction engine runs as a component in the database; when a user request reaches the database, the database kernel responds to the user's request and passes a copy of the request to the prediction engine, triggering the prediction engine's reasoning process. t Enter the actual state after the request t , this state can be used to iteratively train the prediction model.
[0077] 3. Run the prediction engine of the RNN model.
[0078] After the RNN model is trained, it is configured to the inference engine to perform continuous inference calculations based on the status of the database in real time. t-1 And user input x t Predict the next state h of the database t0 When the database prediction vector h t0 When a value in exceeds the preset threshold, an alarm will be triggered. On another data path, the database synchronously executes the user input x t , so that the database actually enters the next state h t , this new state h t It can be used for continued learning of the prediction model and advance the parameters of the prediction formula from the previous state to the next state.
[0079] An embodiment of the present invention further provides a database monitoring and early warning implementation device, comprising: at least one memory and at least one processor;
[0080] The at least one memory is configured to store a machine-readable program;
[0081] The at least one processor is used to call the machine-readable program to implement the database monitoring and early warning method described in the above embodiment.
[0082] Embodiments of the present invention further provide a computer-readable medium having computer instructions stored thereon. When executed by a processor, the computer instructions implement the database monitoring and early warning methods described in the above embodiments. Specifically, a system or device equipped with a storage medium can be provided. The storage medium stores software program code that implements the functions of any of the above embodiments, and the computer (or CPU or MPU) of the system or device can read and execute the program code stored in the storage medium.
[0083] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute part of the present invention.
[0084] Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code can be downloaded from a server computer via a communication network.
[0085] In addition, it should be clear that the functions of any of the above embodiments can be achieved not only by executing the program code read by the computer, but also by enabling the operating system operating on the computer to complete part or all of the actual operations based on the instructions of the program code.
[0086] In addition, it can be understood that the program code read from the storage medium is written into the memory provided in the expansion board inserted into the computer or into the memory provided in the expansion unit connected to the computer, and then based on the instructions of the program code, the CPU installed on the expansion board or expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above embodiments.
[0087] The present invention has been shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the code review methods in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the scope of protection of the present invention.
Claims
1. A database monitoring and early warning method, characterized in that: Using a recurrent neural network to implement artificial intelligence modeling of the current state of the database and the user's request to the next state, and based on the recurrent neural network to achieve database monitoring, state prediction and early warning; include: Establish a recurrent neural network for database monitoring and early warning: specify the state h in the database for the time period t t Its previous time period state h t-1 Establish an input operation based on the time period x t The sequence model is used to reflect the impact of user input on the database state during the continuous operation of data; The database runs the recurrent neural network prediction engine: when a user request is received, the prediction engine's reasoning process is triggered, and the database executes the user input operation x in the time period t Enter the actual time period state after the request t ,The state of this time period can be used to iteratively train the recurrent neural network prediction model; Run the recurrent neural network prediction model: configure the trained recurrent neural network prediction model to the inference engine, and perform continuous inference calculations based on the status of the database in real time. t-1 And user input x t Predict the next state h of the database t0 ; When the database prediction vector h t0 When a value in the parameter exceeds the preset threshold, an alarm will be activated.
2. A database monitoring and early warning method according to claim 1, characterized in that: The establishment of a recurrent neural network for database monitoring and early warning, In database monitoring and early warning modeling, the time axis is first divided into segments of configurable size. The state of the database and related resources in the tth time period is h t , the sum of user inputs in the tth time period is x t , then the formal formula of the recurrent neural network is: h t =f(W h h t-1 +W x x t +b); Among them, W h is the weight matrix of the state at the previous moment; W x is the weight matrix of the current input; b is the bias term; f is the activation function; W h 、W x are the parameters to be learned in the recurrent neural network training.
3. A database monitoring and early warning method according to claim 2, characterized in that: The parameter W h 、W x , b, and f are all designed and initialized using existing recurrent neural network model theories and techniques.
4. A database monitoring and early warning method according to claim 1 or 2, characterized in that: The state of the database and related resources in a specified time period h t The state is represented by a vector containing database system state information, including disk IO times, disk IO data volume, number of read records, number of written records, buffer resource usage, memory usage, CPU usage, concurrent work coroutine / thread usage, and logical lock / physical lock usage.
5. A database monitoring and early warning method according to claim 1 or 2, characterized in that: The user input operation x within the specified time period t ,The user's query, write, including DDL, system parameter change request and request object, are defined as their own vectors through feature engineering to participate in the calculation, and the sum of user input within the time period is obtained.
6. A database monitoring and early warning method according to claim 1, characterized in that: The database runs a recurrent neural network prediction engine, which runs as a component in the database; when a user request arrives at the database, the database kernel responds to the user's request and passes a copy of the request to the prediction engine, triggering the prediction engine's reasoning process.
7. A database monitoring and early warning method according to claim 1, characterized in that: The running recurrent neural network prediction model, On another data path, the database executes the user input x synchronously t , so that the database actually enters the next state h t , this new state h t It can be used for continued learning of the prediction model and advance the parameters of the prediction formula from the previous state to the next state.
8. A database monitoring and early warning system, characterized in that: Database monitoring, status prediction, and early warning are achieved based on recurrent neural networks. The system includes: Model building module, used to build recurrent neural networks for database monitoring and early warning; The prediction engine trigger module runs the recurrent neural network prediction engine in the database. When a user request is received, the prediction engine's reasoning process is triggered. The prediction model operation module is used to implement the operation of the recurrent neural network prediction model, perform continuous reasoning calculations based on the status of the database in real time, and trigger an alarm when a value in the predicted state vector exceeds a preset threshold; The system realizes database monitoring and early warning through the method described in any one of claims 1 to 7.
9. A database monitoring and early warning implementation device, characterized in that: include: at least one memory and at least one processor; The at least one memory is configured to store a machine-readable program; The at least one processor is configured to call the machine-readable program to implement the method according to any one of claims 1 to 7.
10. A computer-readable medium, characterized in that The computer readable medium stores computer instructions, which, when executed by a processor, can implement the method according to any one of claims 1 to 7.