Database built-in anomaly detection and anomaly visualization system and method

Abstract

The present invention discloses a database built-in anomaly detection and anomaly visualization system and method, which belongs to the field of database technology. The technical problem to be solved by the present invention is how to detect and respond to abnormal situations in a database in real time and accurately. The technical scheme adopted is: a data acquisition module, which is used to periodically collect key indicator data in database operation, and store the collected key indicator data in database operation in a database table; wherein the key indicator data in database operation include CPU usage, disk I / O operation, memory usage and network traffic; an anomaly detection module, which is used to perform real-time analysis on the collected key indicator data in database operation, detect possible abnormal situations, and then obtain anomaly detection analysis results; a visualization display module, which is used to display the anomaly detection analysis results to users in the form of charts, highlight abnormal points, and provide detailed database operation status and related SQL statements.

Description

Technical Field

[0001] The present invention relates to the technical field of databases, and in particular to a database built-in anomaly detection and anomaly visualization system and method. Background Art

[0002] As the scale and complexity of database systems continue to increase, traditional manually set thresholds and rules have become difficult to effectively deal with complex and diverse abnormal situations. This approach not only relies on the subjective judgment of the DBA (database administrator) on system behavior, but is also easily affected by human factors, resulting in inaccurate anomaly detection and even missing key abnormal signals. In addition, manually set thresholds are usually static and cannot dynamically adapt to changes in data distribution, resulting in low detection accuracy and robustness when facing complex and changing system loads.

[0003] With the increasingly complex operating environment and diverse load characteristics in database systems, how to detect and respond to abnormal situations in the database in real time and accurately is a technical problem that needs to be solved urgently. Summary of the invention

[0004] The technical task of the present invention is to provide a database built-in anomaly detection and anomaly visualization system and method to solve the problem of how to detect and respond to abnormal situations in the database in real time and accurately.

[0005] The technical task of the present invention is achieved in the following manner: a database built-in anomaly detection and anomaly visualization system, the system comprising:

[0006] The data collection module is used to periodically collect key indicator data in the operation of the database and store the collected key indicator data in the operation of the database in the database table; wherein the key indicator data in the operation of the database includes CPU usage, disk I / O operation, memory usage and network traffic;

[0007] The anomaly detection module is used to perform real-time analysis on key indicator data collected during database operation, detect possible anomalies, and obtain anomaly detection analysis results;

[0008] The visualization module is used to display the anomaly detection analysis results to the user in the form of charts, highlight the anomalies, and provide detailed database operation status and related SQL statements; at the same time, users can interact through the chart interface to view the database operation data and anomaly detection analysis results of different time periods.

[0009] Preferably, the system supports driving the execution of the anomaly detection module through the front-end interface; specifically: when the front-end is refreshed, the SQL function execution of the anomaly detection module is automatically triggered, and the user selects different time ranges, indicators and anomaly detection algorithms for analysis.

[0010] Preferably, the data acquisition module collects data once every 10 seconds, and stores the collected data into a corresponding table in the database for subsequent analysis and query;

[0011] The collected data include the following:

[0012] CPU collection, as follows:

[0013] Collect the current user's CPU usage percentage through sys.cpu.user.percent;

[0014] Collect the current system CPU usage percentage through sys.cpu.sys.percent;

[0015] Use sys.cpu.combined.percent-normalized to take the current user and system CPU usage percentages and normalize them by the number of cores;

[0016] IO collection, as follows:

[0017] The number of disk read operations is collected through sys.host.disk.read.count;

[0018] The number of bytes read from the disk is collected through sys.host.disk.read.bytes;

[0019] Use sys.host.disk.read.time to collect the disk read time;

[0020] The number of bytes written to disk is collected through sys.host.disk.write.bytes;

[0021] Use sys.host.disk.io.time to collect the time when the disk reads or writes operations;

[0022] Use sys.host.disk.weightedio.time to collect the weighted time of disk read or write operations;

[0023] The total number of bytes received by the network interface is collected through sys.host.net.recv.bytes;

[0024] The number of packets received by the network interface is collected through sys.host.net.recv.packets;

[0025] Collect the total number of bytes sent by the network interface through sys.host.net.send.bytes;

[0026] Memory collection, as follows:

[0027] Use sys.rss to collect the space occupied by the current process in physical memory.

[0028] Preferably, the anomaly detection module has multiple built-in anomaly detection algorithms, including 3Sigma algorithm, box plot algorithm, ESD algorithm and KNN algorithm;

[0029] The anomaly detection module also supports detection based on event windows to ensure the accuracy and timeliness of anomaly detection, and the anomaly detection module also supports direct calls through SQL.

[0030] Preferably, the front end uses ECharts for data visualization to ensure that the charts are clear and effectively reflect the operation status of the database system; the contents of the chart display include the following:

[0031] ① Time range selection: Users can view data within a specific time period by selecting a time range (e.g., "last 10 minutes, last hour, last 6 hours, last day");

[0032] ②Chart display: The chart shows the changes in CPU usage, with the ordinate representing percentage and the abscissa representing time. Each data point corresponds to the CPU usage at a timestamp, helping users understand the changing trend of CPU usage.

[0033] ③ Anomaly point marking: The anomaly points are specially marked in the chart (for example, red marks), so that users can quickly locate the abnormal moment; when the user hovers the mouse over the anomaly point, detailed information is displayed, including the specific CPU usage and time.

[0034] ④Data Overview: There is a data overview area below the chart, which allows users to quickly navigate to different time periods by sliding, allowing users to quickly browse the data within the entire time range;

[0035] ⑤Interactive functions, as follows:

[0036] Hover: Display detailed information (such as CPU usage and time);

[0037] Double-click the exception point: jump to the exception details page, which provides CPU usage, disk I / O, and occupancy detailed data, and displays the database operation status and all SQL statements executed in the past 10 seconds to help users quickly locate the root cause of the problem.

[0038] A method for detecting and visualizing anomalies in a database is provided. The method is as follows:

[0039] Data collection: When the database system is running, key indicator data of the database operation is automatically collected every 10 seconds, and the collected key indicator data of the database operation is stored in the database table; the key indicator data of the database operation includes CPU usage, disk I / O operation, memory usage and network traffic;

[0040] Front-end display drive: Anomaly detection is driven through the front-end interface. Whenever the user refreshes the front-end page, the SQL function for anomaly detection is automatically triggered. The user selects different time ranges, indicators, and anomaly detection algorithms through the front-end interface.

[0041] Anomaly detection algorithm execution: Driven by the front end, the built-in anomaly detection algorithm is called to analyze the data, and the specified algorithm is used to analyze the collected data in real time according to the specified time period to identify anomalies;

[0042] Anomaly display: Based on the execution results of the anomaly detection algorithm, the detected anomalies will be visualized on the front-end interface. Anomalies are usually highlighted with special marks (such as red marks), and users can hover to view detailed information or double-click to jump to a more detailed anomaly analysis page. The anomaly analysis page displays the specific value of the anomaly, the time of occurrence, and other related database operation status, helping users quickly locate and analyze problems.

[0043] Preferably, during the data collection process, the collected data includes the following:

[0044] CPU collection, as follows:

[0045] Collect the current user's CPU usage percentage through sys.cpu.user.percent;

[0046] Collect the current system CPU usage percentage through sys.cpu.sys.percent;

[0047] Use sys.cpu.combined.percent-normalized to take the current user and system CPU usage percentages and normalize them by the number of cores;

[0048] IO collection, as follows:

[0049] The number of disk read operations is collected through sys.host.disk.read.count;

[0050] The number of bytes read from the disk is collected through sys.host.disk.read.bytes;

[0051] Use sys.host.disk.read.time to collect the disk read time;

[0052] The number of bytes written to disk is collected through sys.host.disk.write.bytes;

[0053] Use sys.host.disk.io.time to collect the time when the disk reads or writes operations;

[0054] Use sys.host.disk.weightedio.time to collect the weighted time of disk read or write operations;

[0055] The total number of bytes received by the network interface is collected through sys.host.net.recv.bytes;

[0056] The number of packets received by the network interface is collected through sys.host.net.recv.packets;

[0057] Collect the total number of bytes sent by the network interface through sys.host.net.send.bytes;

[0058] Memory collection, as follows:

[0059] Use sys.rss to collect the space occupied by the current process in physical memory.

[0060] Preferably, the anomaly detection algorithms include 3Sigma algorithm, box plot algorithm, ESD algorithm and KNN algorithm;

[0061] Among them, the 3Sigma algorithm is a statistical method based on normal distribution. It identifies outliers by calculating the mean and standard deviation. Data that exceeds three times the standard deviation is considered abnormal.

[0062] The box plot algorithm uses the quartiles of data to detect anomalies. Data beyond a certain range of the upper and lower quartiles are considered abnormal.

[0063] The ESD (Extreme Studentized Deviate) algorithm is a statistical method specifically used to detect outliers in time series. It identifies more outliers by iteratively calculating outliers and removing them.

[0064] The KNN (K-Nearest Neighbors) algorithm is based on similarity measurement and identifies points in the data set that are far away from other data points as anomalies.

[0065] The 3Sigma algorithm, box plot algorithm, ESD algorithm, and KNN algorithm are all implemented based on window functions and perform anomaly detection based on the data characteristics of a specified time period.

[0066] An electronic device comprising: a memory and at least one processor;

[0067] Wherein, the memory stores a computer program;

[0068] The at least one processor executes the computer program stored in the memory, so that the at least one processor performs the database built-in anomaly detection and anomaly visualization method as described above.

[0069] A computer-readable storage medium stores a computer program, which can be executed by a processor to implement the database built-in anomaly detection and anomaly visualization method as described above.

[0070] The database built-in anomaly detection and anomaly visualization system and method of the present invention have the following advantages:

[0071] (1) The present invention regularly collects multiple key performance indicators of the database (such as CPU utilization, disk I / O, network traffic, memory usage, etc.), and combines multiple built-in anomaly detection algorithms (such as 3sigma, box plot, ESD algorithm, KNN algorithm) to analyze the data in real time. It can monitor the key performance indicators of the database in real time and accurately identify abnormal behaviors; at the same time, the front-end displays the abnormal points, and the user can double-click the abnormal point to jump to the detailed analysis page. It can automatically store the collected data, provide timely and intuitive abnormal monitoring and management, and improve the stability and performance of the database;

[0072] (ii) The present invention can dynamically adapt to data changes and automatically adjust the detection threshold, thereby improving the accuracy and robustness of detection; in addition, the present invention provides an intuitive anomaly visualization interface, where users can double-click an anomaly point to view detailed anomaly analysis and related data, which not only simplifies the workload of the DBA, but also significantly improves the operational stability and anomaly handling capabilities of the database system;

[0073] (iii) The present invention can monitor key performance indicators in the database in real time: through the built-in anomaly detection algorithm, it automatically collects various performance indicators running in the database system, such as CPU usage, memory occupancy, disk I / O, etc., dynamically analyzes these data, and identifies possible abnormal behaviors in real time to ensure the stable operation of the system;

[0074] (IV) The present invention realizes the visualization and analysis of anomaly detection results: The present invention provides an intuitive visualization interface to mark and display detected anomalies. Users can use the visualization interface to quickly locate anomalies and view detailed anomaly analysis, helping DBAs and developers to quickly understand the source and impact of anomalies and then formulate reasonable response strategies;

[0075] (V) The present invention improves the accuracy and robustness of anomaly detection: through the built-in multiple anomaly detection algorithms, the system can automatically adapt to changes in data distribution according to the characteristics of the collected data, reduce false positives and false negatives, improve the accuracy and robustness of anomaly detection, and thus improve the overall operating efficiency of the database;

[0076] (vi) The present invention has dynamic adaptability: the built-in anomaly detection algorithm can automatically adjust according to the dynamic changes of data to improve the accuracy of detection;

[0077] (VII) The present invention can monitor in real time: data collection is performed every 10 seconds to ensure that anomalies can be discovered in a timely manner, thereby ensuring the timeliness and accuracy of the data used by the anomaly detection module;

[0078] (VIII) The present invention is easy to use: through the visual display and interaction of the front end, users can easily view and analyze abnormal points and their detailed information;

[0079] (IX) The present invention is highly comprehensive: the collected data covers various aspects such as CPU, IO, memory, etc., and can fully reflect the operation status of the database;

[0080] (10) The present invention is capable of persistent storage: all collected data will be persistently stored in the database for subsequent query and analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] The present invention is further described below in conjunction with the accompanying drawings.

[0082] Attached Figure 1 This is a schematic diagram of the structure of the database built-in anomaly detection and anomaly visualization system. DETAILED DESCRIPTION

[0083] The database built-in anomaly detection and anomaly visualization system and method of the present invention are described in detail below with reference to the accompanying drawings and specific embodiments of the specification.

[0084] Embodiment 1:

[0085] As attached Figure 1 As shown, this embodiment provides a database built-in anomaly detection and anomaly visualization system, the system comprising:

[0086] The data collection module is used to periodically collect key indicator data in the operation of the database and store the collected key indicator data in the operation of the database in the database table; wherein the key indicator data in the operation of the database includes CPU usage, disk I / O operation, memory usage and network traffic;

[0087] The anomaly detection module is used to perform real-time analysis on key indicator data collected during database operation, detect possible anomalies, and obtain anomaly detection analysis results;

[0088] The visualization module is used to display the anomaly detection analysis results to the user in the form of charts, highlight the anomalies, and provide detailed database operation status and related SQL statements; at the same time, users can interact through the chart interface to view the database operation data and anomaly detection analysis results of different time periods.

[0089] In this embodiment, the system supports driving the execution of the anomaly detection module through the front-end interface; specifically: when the front-end is refreshed, the SQL function execution of the anomaly detection module is automatically triggered, and the user selects different time ranges, indicators and anomaly detection algorithms for analysis.

[0090] The data collection module in this embodiment collects data every 10 seconds and stores the collected data in the corresponding table in the database for subsequent analysis and query;

[0091] The collected data include the following:

[0092] CPU collection, as follows:

[0093] Collect the current user's CPU usage percentage through sys.cpu.user.percent;

[0094] Collect the current system CPU usage percentage through sys.cpu.sys.percent;

[0095] Use sys.cpu.combined.percent-normalized to take the current user and system CPU usage percentages and normalize them by the number of cores;

[0096] IO collection, as follows:

[0097] The number of disk read operations is collected through sys.host.disk.read.count;

[0098] The number of bytes read from the disk is collected through sys.host.disk.read.bytes;

[0099] Use sys.host.disk.read.time to collect the disk read time;

[0100] The number of bytes written to disk is collected through sys.host.disk.write.bytes;

[0101] Use sys.host.disk.io.time to collect the time when the disk reads or writes operations;

[0102] Use sys.host.disk.weightedio.time to collect the weighted time of disk read or write operations;

[0103] The total number of bytes received by the network interface is collected through sys.host.net.recv.bytes;

[0104] The number of packets received by the network interface is collected through sys.host.net.recv.packets;

[0105] Collect the total number of bytes sent by the network interface through sys.host.net.send.bytes;

[0106] Memory collection, as follows:

[0107] Use sys.rss to collect the space occupied by the current process in physical memory.

[0108] The anomaly detection module in this embodiment has multiple built-in anomaly detection algorithms, including 3Sigma algorithm, box plot algorithm, ESD algorithm and KNN algorithm;

[0109] Among them, 3Sigma is a statistical method based on normal distribution. It identifies outliers by calculating the mean and standard deviation. Data beyond three times the standard deviation is considered abnormal.

[0110] The box plot algorithm uses the quartiles of data to detect anomalies. Data beyond a certain range of the upper and lower quartiles are considered abnormal.

[0111] The ESD (Extreme Studentized Deviate) algorithm is a statistical method specifically used to detect outliers in time series. It identifies more outliers by iteratively calculating outliers and removing them.

[0112] The KNN (K-Nearest Neighbors) algorithm is based on similarity measurement and identifies points in the data set that are far away from other data points as anomalies;

[0113] The above algorithm is implemented based on the window function, and can perform anomaly detection based on the data characteristics of the specified time period; for example, obtain the data of the past 5 days for anomaly detection, and only consider the data of these 5 days during the detection.

[0114] The anomaly detection module also supports detection based on event windows to ensure the accuracy and timeliness of anomaly detection.

[0115] One of the major advantages of the anomaly detection module is that it can be directly called through SQL. For example, users can use the following SQL query to use the KNN algorithm to perform anomaly detection on the data in the cpu_percent column of the database table:

[0116] SELECT*,knn(cpu_percent,6)OVER()FROM detection WHERE time>=NOW()-INTERVAL'5day'AND time<=NOW();

[0117] This SQL query will call the KNN algorithm in the detection table to detect the data of the past 5 days, and mark the records identified as abnormal in the query results. Users can seamlessly integrate anomaly detection into daily database queries through simple SQL operations, greatly facilitating database monitoring and management.

[0118] The front end of this embodiment uses ECharts to visualize the data, ensuring that the chart is clear and effectively reflects the operation status of the database system; the contents of the chart display include the following:

[0119] ① Time range selection: Users can view data within a specific time period by selecting a time range (e.g., "last 10 minutes, last hour, last 6 hours, last day");

[0120] ②Chart display: The chart shows the changes in CPU usage, with the ordinate representing percentage and the abscissa representing time. Each data point corresponds to the CPU usage at a timestamp, helping users understand the changing trend of CPU usage.

[0121] ③ Anomaly point marking: The anomaly points are specially marked in the chart (for example, red marks), so that users can quickly locate the abnormal moment; when the user hovers the mouse over the anomaly point, detailed information is displayed, including the specific CPU usage and time.

[0122] ④Data Overview: There is a data overview area below the chart, which allows users to quickly navigate to different time periods by sliding, allowing users to quickly browse the data within the entire time range;

[0123] ⑤Interactive functions, as follows:

[0124] Hover: Display detailed information (such as CPU usage and time);

[0125] Double-click the exception point: jump to the exception details page, which provides CPU usage, disk I / O, and occupancy detailed data, and displays the database operation status and all SQL statements executed in the past 10 seconds to help users quickly locate the root cause of the problem.

[0126] Embodiment 2:

[0127] This embodiment provides a database built-in anomaly detection and anomaly visualization method, the method is as follows:

[0128] S1. Data collection: When the database system is running, key indicator data of the database operation is automatically collected every 10 seconds, and the collected key indicator data of the database operation is stored in the database table; the key indicator data of the database operation includes CPU usage, disk I / O operation, memory usage and network traffic;

[0129] S2, front-end display drive: drive anomaly detection through the front-end interface. Whenever the user refreshes the front-end page, the SQL function for anomaly detection is automatically triggered. The user selects different time ranges, indicators and anomaly detection algorithms through the front-end interface.

[0130] S3, anomaly detection algorithm execution: driven by the front end, call the built-in anomaly detection algorithm for data analysis, and use the specified algorithm to perform real-time analysis on the collected data according to the specified time period to identify anomalies;

[0131] S4. Anomaly display: Based on the execution results of the anomaly detection algorithm, the detected anomalies will be visually displayed on the front-end interface. Among them, the anomalies are usually highlighted with special marks (such as red marks), and users can hover to view detailed information or double-click to jump to a more detailed anomaly analysis page. The anomaly analysis page displays the specific values ​​of the anomalies, the time of occurrence, and other related database operation status, helping users quickly locate and analyze problems.

[0132] In the data collection process in step S1 of this embodiment, the collected data includes the following:

[0133] CPU collection, as follows:

[0134] Collect the current user's CPU usage percentage through sys.cpu.user.percent;

[0135] Collect the current system CPU usage percentage through sys.cpu.sys.percent;

[0136] Use sys.cpu.combined.percent-normalized to take the current user and system CPU usage percentages and normalize them by the number of cores;

[0137] IO collection, as follows:

[0138] The number of disk read operations is collected through sys.host.disk.read.count;

[0139] The number of bytes read from the disk is collected through sys.host.disk.read.bytes;

[0140] Use sys.host.disk.read.time to collect the disk read time;

[0141] The number of bytes written to disk is collected through sys.host.disk.write.bytes;

[0142] Use sys.host.disk.io.time to collect the time when the disk reads or writes operations;

[0143] Use sys.host.disk.weightedio.time to collect the weighted time of disk read or write operations;

[0144] The total number of bytes received by the network interface is collected through sys.host.net.recv.bytes;

[0145] The number of packets received by the network interface is collected through sys.host.net.recv.packets;

[0146] Collect the total number of bytes sent by the network interface through sys.host.net.send.bytes;

[0147] Memory collection, as follows:

[0148] Use sys.rss to collect the space occupied by the current process in physical memory.

[0149] The anomaly detection algorithms in this embodiment include 3Sigma algorithm, box plot algorithm, ESD algorithm and KNN algorithm;

[0150] Among them, the 3Sigma algorithm is a statistical method based on normal distribution. It identifies outliers by calculating the mean and standard deviation. Data that exceeds three times the standard deviation is considered abnormal.

[0151] The box plot algorithm uses the quartiles of data to detect anomalies. Data beyond a certain range of the upper and lower quartiles are considered abnormal.

[0152] The ESD (Extreme Studentized Deviate) algorithm is a statistical method specifically used to detect outliers in time series. It identifies more outliers by iteratively calculating outliers and removing them.

[0153] The KNN (K-Nearest Neighbors) algorithm is based on similarity measurement and identifies points in the data set that are far away from other data points as anomalies.

[0154] The 3Sigma algorithm, box plot algorithm, ESD algorithm, and KNN algorithm are all implemented based on window functions and perform anomaly detection based on the data characteristics of a specified time period.

[0155] Embodiment 3:

[0156] This embodiment also provides an electronic device, including: a memory and a processor;

[0157] Wherein, the memory stores computer-executable instructions;

[0158] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the database built-in anomaly detection and anomaly visualization method in any embodiment of the present invention.

[0159] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor may be a microprocessor or any conventional processor, etc.

[0160] The memory can be used to store computer programs and / or modules. The processor realizes various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory can also include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash memory card, at least one disk storage period, a flash memory device, or other volatile solid-state storage devices.

[0161] Embodiment 4:

[0162] This embodiment also provides a computer-readable storage medium, which stores a plurality of instructions, which are loaded by a processor, so that the processor executes the database built-in anomaly detection and anomaly visualization method in any embodiment of the present invention. Specifically, a system or device equipped with a storage medium can be provided, on which a software program code that implements the functions of any of the above embodiments is stored, and a computer (or CPU or MPU) of the system or device reads and executes the program code stored in the storage medium.

[0163] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0164] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RYM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0165] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0166] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or written to a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or the expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0167] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A database built-in anomaly detection and anomaly visualization system, characterized in that: The system includes: The data collection module is used to periodically collect key indicator data in the operation of the database and store the collected key indicator data in the operation of the database in the database table; wherein the key indicator data in the operation of the database includes CPU usage, disk I / O operation, memory usage and network traffic; The anomaly detection module is used to perform real-time analysis on key indicator data collected during database operation, detect possible anomalies, and obtain anomaly detection analysis results; The visualization module is used to display the anomaly detection analysis results to the user in the form of charts, highlight the anomalies, and provide detailed database operation status and related SQL statements; at the same time, users can interact through the chart interface to view the database operation data and anomaly detection analysis results of different time periods.

2. The database built-in anomaly detection and anomaly visualization system according to claim 1, characterized in that: The system supports driving the execution of the anomaly detection module through the front-end interface; specifically: when the front-end is refreshed, the SQL function execution of the anomaly detection module is automatically triggered, and the user selects different time ranges, indicators and anomaly detection algorithms for analysis.

3. The database built-in anomaly detection and anomaly visualization system according to claim 1, characterized in that: The data collection module collects data every 10 seconds and stores the collected data in the corresponding table in the database for subsequent analysis and query; The collected data include the following: CPU collection, as follows: Collect the current user's CPU usage percentage through sys.cpu.user.percent; Collect the current system CPU usage percentage through sys.cpu.sys.percent; Use sys.cpu.combined.percent-normalized to take the current user and system CPU usage percentages and normalize them by the number of cores; IO collection, as follows: The number of disk read operations is collected through sys.host.disk.read.count; The number of bytes read from the disk is collected through sys.host.disk.read.bytes; Use sys.host.disk.read.time to collect the disk read time; The number of bytes written to disk is collected through sys.host.disk.write.bytes; Use sys.host.disk.io.time to collect the time when the disk reads or writes operations; Use sys.host.disk.weightedio.time to collect the weighted time of disk read or write operations; The total number of bytes received by the network interface is collected through sys.host.net.recv.bytes; The number of packets received by the network interface is collected through sys.host.net.recv.packets; Collect the total number of bytes sent by the network interface through sys.host.net.send.bytes; Memory collection, as follows: Use sys.rss to collect the space occupied by the current process in physical memory.

4. The database built-in anomaly detection and anomaly visualization system according to claim 1, characterized in that: The anomaly detection module has multiple built-in anomaly detection algorithms, including 3Sigma algorithm, box plot algorithm, ESD algorithm and KNN algorithm; The anomaly detection module also supports detection based on event windows to ensure the accuracy and timeliness of anomaly detection, and the anomaly detection module also supports direct calls through SQL.

5. The database built-in anomaly detection and anomaly visualization system according to any one of claims 1 to 4, characterized in that: The front end uses ECharts to visualize the data, ensuring that the charts are clear and effectively reflect the operation status of the database system; the contents of the charts include the following: ① Time range selection: Users can view data within a specific time period by selecting a time range; ②Chart display: The chart shows the changes in CPU usage, with the ordinate representing percentage and the abscissa representing time. Each data point corresponds to the CPU usage at a timestamp, helping users understand the changing trend of CPU usage. ③ Anomaly point marking: Anomaly points are specially marked in the chart, allowing users to quickly locate the abnormal moment; when the user hovers the mouse over the abnormal point, detailed information is displayed, including the specific CPU usage and time. ④Data Overview: There is a data overview area below the chart, which allows users to quickly navigate to different time periods by sliding, allowing users to quickly browse the data within the entire time range; ⑤Interactive functions, as follows: Hover: show detailed information; Double-click the exception point: jump to the exception details page, which provides CPU usage, disk I / O, and occupancy detailed data, and displays the database operation status and all SQL statements executed in the past 10 seconds to help users quickly locate the root cause of the problem.

6. A method for detecting and visualizing anomalies in a database, characterized in that: The method is as follows: Data collection: When the database system is running, key indicator data of the database operation is automatically collected every 10 seconds, and the collected key indicator data of the database operation is stored in the database table; the key indicator data of the database operation includes CPU usage, disk I / O operation, memory usage and network traffic; Front-end display drive: Anomaly detection is driven through the front-end interface. Whenever the user refreshes the front-end page, the SQL function for anomaly detection is automatically triggered. The user selects different time ranges, indicators, and anomaly detection algorithms through the front-end interface. Anomaly detection algorithm execution: Driven by the front end, the built-in anomaly detection algorithm is called to analyze the data, and the specified algorithm is used to analyze the collected data in real time according to the specified time period to identify anomalies; Anomaly point display: Based on the execution results of the anomaly detection algorithm, the detected anomalies will be visually displayed on the front-end interface; the anomalies are usually highlighted with special marks, and users can hover to view detailed information or double-click to jump to a more detailed anomaly analysis page; the anomaly analysis page displays the specific value of the anomaly, the time of occurrence, and other related database operation status.

7. The method for detecting and visualizing anomalies in a database according to claim 6, characterized in that: During the data collection process, the collected data include the following: CPU collection, as follows: Collect the current user's CPU usage percentage through sys.cpu.user.percent; Collect the current system CPU usage percentage through sys.cpu.sys.percent; Use sys.cpu.combined.percent-normalized to take the current user and system CPU usage percentages and normalize them by the number of cores; IO collection, as follows: The number of disk read operations is collected through sys.host.disk.read.count; The number of bytes read from the disk is collected through sys.host.disk.read.bytes; Use sys.host.disk.read.time to collect the disk read time; The number of bytes written to disk is collected through sys.host.disk.write.bytes; Use sys.host.disk.io.time to collect the time when the disk reads or writes operations; Use sys.host.disk.weightedio.time to collect the weighted time of disk read or write operations; The total number of bytes received by the network interface is collected through sys.host.net.recv.bytes; The number of packets received by the network interface is collected through sys.host.net.recv.packets; Collect the total number of bytes sent by the network interface through sys.host.net.send.bytes; Memory collection, as follows: Use sys.rss to collect the space occupied by the current process in physical memory.

8. The method for database built-in anomaly detection and anomaly visualization according to claim 6 or 7, characterized in that: Anomaly detection algorithms include 3Sigma algorithm, box plot algorithm, ESD algorithm and KNN algorithm; Among them, the 3Sigma algorithm is a statistical method based on normal distribution. It identifies outliers by calculating the mean and standard deviation. Data that exceeds three times the standard deviation is considered abnormal. The box-type algorithm uses the quartiles of data for anomaly detection, and data outside a certain range of the upper and lower quartiles is considered anomaly; The ESD algorithm is a statistical method specifically used to detect outliers in time series. It identifies more outliers by iteratively calculating outliers and removing them. The KNN algorithm is based on similarity measurement and identifies points in the data set that are far away from other data points as anomalies. The 3Sigma algorithm, box plot algorithm, ESD algorithm, and KNN algorithm are all implemented based on window functions and perform anomaly detection based on the data characteristics of a specified time period.

9. An electronic device, characterized in that: include: memory and at least one processor; Wherein, the memory stores a computer program; The at least one processor executes the computer program stored in the memory, so that the at least one processor executes the database built-in anomaly detection and anomaly visualization method as described in any one of claims 6 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which can be executed by a processor to implement the database built-in anomaly detection and anomaly visualization method as described in any one of claims 6 to 8.

Images