A database health assessment method and apparatus
By using an automated database health assessment method, which utilizes data acquisition plugins and machine learning models to filter periodic abnormal elements, classify and determine the categories of assessment elements, the problem of large errors in manual assessment is solved, and efficient and accurate database health assessment is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA UNIONPAY
- Filing Date
- 2022-08-12
- Publication Date
- 2026-07-14
AI Technical Summary
Existing technologies rely on human experience to assess database health, resulting in large errors, low efficiency, and high costs.
The system automatically collects data on multiple evaluation elements from the database using a data acquisition plugin. It then uses a machine learning model to filter out periodic abnormal elements, classifies and identifies the categories of evaluation elements, and finally determines the database health based on the comprehensive evaluation results, generating a health report.
It has achieved standardization, automation, periodicity, and self-service for database health assessment, improving the accuracy and efficiency of the assessment and reducing the cost of manual assessment.
Smart Images
Figure CN115438244B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of operation and maintenance technology, and in particular to a method and apparatus for assessing database health. Background Technology
[0002] With the rapid development of computer technology, databases have been widely used. In order for databases to provide stable services, in operation and maintenance scenarios, it is necessary to regularly perform health status checks or health assessments on the database. Database health status checks and health assessments have different focuses and meet different use cases.
[0003] In related technologies, database administrators manually collect relevant database metrics and then assess the database health based on their experience, ultimately generating a database health assessment report. However, this approach is limited by the database administrator's experience, making it prone to errors in the assessment results. Furthermore, manual assessment of database health is inefficient and costly. Summary of the Invention
[0004] This application provides a database health assessment method and apparatus to improve the accuracy and efficiency of database health assessment.
[0005] On one hand, embodiments of this application provide a database health assessment method, the method comprising:
[0006] Select the appropriate data acquisition plugin based on the type of database to be evaluated;
[0007] The data acquisition plugin is used to collect element data corresponding to multiple evaluation elements from the database to be evaluated.
[0008] Based on the obtained element data, the multiple evaluation elements are classified to obtain multiple evaluation element categories; and the evaluation elements in the multiple evaluation element categories are judged to obtain the evaluation results of the multiple evaluation element categories.
[0009] By combining the evaluation results of the multiple evaluation element categories, the target health level of the database to be evaluated is determined.
[0010] Optionally, the step of distinguishing the evaluation elements in the plurality of evaluation element categories to obtain the evaluation results of the plurality of evaluation element categories includes:
[0011] For each of the aforementioned evaluation element categories, the following steps shall be performed:
[0012] The feature data of each evaluation element in an evaluation element category is compared with the corresponding decision threshold to obtain the decision result corresponding to each evaluation element;
[0013] The evaluation result of the evaluation element category is determined based on the judgment results of multiple evaluation elements in the evaluation element category.
[0014] Optionally, determining the target health of the database to be evaluated by comprehensively considering the evaluation results of the multiple evaluation element categories includes:
[0015] Based on the evaluation results of the multiple evaluation element categories and their weights, the target health of the database to be evaluated is determined.
[0016] Optionally, before selecting the appropriate data acquisition plugin based on the type of database to be evaluated, the following steps are also included:
[0017] For at least one of the multiple evaluation element categories, perform the following steps respectively:
[0018] Based on historical element data of multiple original elements in an evaluation element category, periodic abnormal elements are screened out from the multiple original elements, and the periodic abnormal elements are removed from the multiple original elements to obtain the evaluation elements in the evaluation element category.
[0019] Optionally, the step of filtering out periodically anomalous elements from historical element data of multiple original elements in an evaluation element category includes:
[0020] Using machine learning models and soft voting classification methods, at least one anomalous element is selected from the historical element data of the multiple original elements.
[0021] For each of the at least one anomalous elements, perform the following steps:
[0022] If the month-on-month and year-on-year parameters corresponding to the historical data of an anomalous element meet the preset conditions, then the anomalous element is determined to be a periodic anomalous element.
[0023] Optionally, the at least one evaluation element category includes a monitoring indicator category and an alarm element category.
[0024] Optionally, after determining the target health of the database to be evaluated by comprehensively considering the evaluation results of the multiple evaluation element categories, the method further includes:
[0025] Based on the target health level and the historical health level of the database to be evaluated, a database health report is generated for the database to be evaluated.
[0026] Optionally, generating a database health report for the database to be evaluated based on the target health of the database to be evaluated and the historical health of the database to be evaluated includes:
[0027] Based on the target health level, the historical health level, and the database optimization scheme, a database health report for the database to be evaluated is generated.
[0028] Optionally, the multiple evaluation element categories include monitoring indicator categories, alarm element categories, database defect categories, standardization inspection categories, operational data statistics categories, and security element categories.
[0029] On one hand, embodiments of this application provide a database health assessment device, the device comprising:
[0030] The selection module allows you to choose the appropriate data acquisition plugin based on the type of database to be evaluated.
[0031] The data acquisition module is used to acquire element data corresponding to multiple evaluation elements from the database to be evaluated through the data acquisition plugin.
[0032] The processing module is used to classify the multiple evaluation elements based on the obtained element data to obtain multiple evaluation element categories; and to distinguish the evaluation elements in the multiple evaluation element categories to obtain the evaluation results of the multiple evaluation element categories.
[0033] The evaluation module is used to integrate the evaluation results of the multiple evaluation element categories to determine the target health of the database to be evaluated.
[0034] Optionally, the processing module is specifically used for:
[0035] For each of the aforementioned evaluation element categories, the following steps shall be performed:
[0036] The feature data of each evaluation element in an evaluation element category is compared with the corresponding decision threshold to obtain the decision result corresponding to each evaluation element;
[0037] The evaluation result of the evaluation element category is determined based on the judgment results of multiple evaluation elements in the evaluation element category.
[0038] Optionally, the evaluation module is specifically used for:
[0039] Based on the evaluation results of the multiple evaluation element categories and their weights, the target health of the database to be evaluated is determined.
[0040] Optionally, the processing module is further configured to:
[0041] Before selecting the appropriate data acquisition plugin based on the type of the database to be evaluated, the following steps are performed for at least one of the multiple evaluation element categories:
[0042] Based on historical element data of multiple original elements in an evaluation element category, periodic abnormal elements are screened out from the multiple original elements, and the periodic abnormal elements are removed from the multiple original elements to obtain the evaluation elements in the evaluation element category.
[0043] Optionally, the processing module is specifically used for:
[0044] Using machine learning models and soft voting classification methods, at least one anomalous element is selected from the historical element data of the multiple original elements.
[0045] For each of the at least one anomalous elements, perform the following steps:
[0046] If the month-on-month and year-on-year parameters corresponding to the historical data of an anomalous element meet the preset conditions, then the anomalous element is determined to be a periodic anomalous element.
[0047] Optionally, the at least one evaluation element category includes a monitoring indicator category and an alarm element category.
[0048] Optionally, the evaluation module is further configured to:
[0049] After determining the target health of the database to be evaluated by combining the evaluation results of the multiple evaluation element categories, a database health report is generated based on the target health and historical health of the database to be evaluated.
[0050] Optionally, the evaluation module is specifically used for:
[0051] Based on the target health level, the historical health level, and the database optimization scheme, a database health report for the database to be evaluated is generated.
[0052] Optionally, the multiple evaluation element categories include monitoring indicator categories, alarm element categories, database defect categories, standardization inspection categories, operational data statistics categories, and security element categories.
[0053] On one hand, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described image processing method.
[0054] On one hand, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the above-described image processing method.
[0055] In this embodiment, different data acquisition plugins are used to collect element data for different types of databases to meet the personalized data collection and processing needs of different types of databases in health assessment. It also supports the online addition of data collection for new types of databases later, making it widely applicable. Secondly, multiple assessment elements are classified to obtain multiple assessment element categories. Then, assessment elements in each category are judged separately to obtain corresponding assessment results. Finally, the assessment results of all assessment element categories are combined to obtain a database health score, achieving standardization, automation, periodicity, and self-service for database health assessment. Compared with manual assessment, this can effectively improve the accuracy and efficiency of database health assessment. Attached Figure Description
[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0057] Figure 1 A schematic diagram of a system architecture provided in an embodiment of this application;
[0058] Figure 2 This application provides a flowchart illustrating a database health assessment method.
[0059] Figure 3 This is a schematic diagram of the structure of an evaluation device provided in an embodiment of this application;
[0060] Figure 4 A flowchart illustrating an evaluation configuration adjustment method provided in this application embodiment;
[0061] Figure 5 A flowchart illustrating a data acquisition method provided in an embodiment of this application;
[0062] Figure 6 A flowchart illustrating a data processing method provided in an embodiment of this application;
[0063] Figure 7 A flowchart illustrating a method for calculating database health as provided in an embodiment of this application;
[0064] Figure 8 A flowchart illustrating a method for generating a database health report, provided in an embodiment of this application;
[0065] Figure 9 A schematic diagram of the structure of a database health assessment device provided in this application embodiment;
[0066] Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation
[0067] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0068] For ease of understanding, the terms used in the embodiments of this invention are explained below.
[0069] Database: A unified entity that provides database services to the outside world, consisting of several database instances, database agents, and other highly available components.
[0070] Soft Voting is a decision-making method in machine learning that uses the average probability of all models predicting a sample as a certain category as the standard, and the category corresponding to the highest probability is the final prediction result.
[0071] Database Administrator (DBA): The database administrator is responsible for the overall management and control of the database system, including basic tasks such as database installation, monitoring, backup, and recovery.
[0072] refer to Figure 1 This is a system architecture diagram applicable to the embodiments of this application. The system architecture includes at least an evaluation device 101 and a database 102. The number of evaluation devices 101 can be one or more, and the number of databases 102 can also be one or more. This application does not specifically limit the number of evaluation devices 101 and databases 102.
[0073] The evaluation device 101 can be a terminal device or a server. The terminal device can be a smartphone, tablet, laptop, desktop computer, smart home appliance, smart voice interaction device, smart vehicle device, etc., but is not limited to these.
[0074] A server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.
[0075] In this application, database 102 refers to a unified entity composed of several database instances, database agents, and other high-availability components that provide database services externally. In the context of databases, a database agent is a middle layer between the application and the database. Each database instance and database agent is an independent resource deployed on a server, with its own IP address, central processing unit (CPU), memory, and independent file system for tablespace and log space. Depending on the deployment, this independent resource can be a physical server, a virtual server, a container, etc.
[0076] When deploying a database, in order to prevent failures such as downtime or unexpected interruptions, high-availability architectures such as master-slave, master-slave, MHA (Master HA) clusters, and MGR (MySQL Group Replication) clusters are usually built. This is to restore the availability of the database as quickly as possible in the event of a failure, minimize downtime, and ensure that business operations are not interrupted due to database failures.
[0077] The evaluation device 101 and the database 102 can be connected directly or indirectly via wired or wireless communication, and this application does not impose any restrictions on this.
[0078] based on Figure 1 The system architecture diagram shown in this application illustrates the flowchart of a database health assessment method. Figure 2 As shown, the process of this method is executed by a computer device, which can be... Figure 1 The evaluation device 101 shown includes the following steps:
[0079] Step S201: Select the appropriate data acquisition plugin based on the type of database to be evaluated.
[0080] Specifically, database health assessment tasks can be triggered manually or by a configuration module. When a task is triggered manually, the initiator, initiation time, and assessment target are recorded in the assessment task file. Administrators define the initiation cycle through the configuration module, which then periodically triggers the assessment tasks, recording the initiator, initiation time, and assessment target in each task. The initiator is uniformly configured as an administrator.
[0081] In some embodiments, in order to meet the needs of adjusting the database evaluation cycle and scope in practical applications, the above configuration module is started during the evaluation device initialization phase. During the formal operation phase of the evaluation device, the configuration information of the configuration module can be modified according to actual needs. After modification, the evaluation device reloads the configuration information, which can specifically realize the adjustment of evaluation switch, evaluation cycle and evaluation scope.
[0082] In some embodiments, the types of databases to be evaluated include MySQL, Redis, DB2, and Oracle. The storage location and data format of the evaluation element data differ for different types of databases. Therefore, in this embodiment, corresponding data acquisition plugins are configured for different types of databases. Furthermore, it supports the later online addition of new data acquisition plugins for new types of databases, thus meeting the health assessment needs of different types of databases and demonstrating good applicability.
[0083] Step S202: Collect element data corresponding to multiple evaluation elements in the database to be evaluated through the data acquisition plugin.
[0084] Specifically, through a data acquisition plugin, monitoring data of the database is collected from the monitoring platform, historical alarm information of the database is collected from the alarm platform, database defect information is collected from the process platform, statistical information on the attributes and operational data of various database objects is collected from within the database, and security element information such as database user permissions and role permissions is collected from within the database. The collected data is then standardized to obtain element data in a unified format.
[0085] Step S203: Based on the obtained element data, classify multiple evaluation elements to obtain multiple evaluation element categories; and distinguish the evaluation elements in each of the multiple evaluation element categories to obtain the evaluation results of the multiple evaluation element categories.
[0086] Specifically, the evaluation element categories include monitoring indicator categories, alarm element categories, database defect categories, standardization inspection categories, operational data statistics categories, and security element categories.
[0087] The evaluation elements in the monitoring metric category include: monitoring data on computing, storage, and network resources used in database operation, such as CPU, memory, log space, tablespace, number of connections, network traffic, and network connectivity; as well as database monitoring data such as queries per second (QPS), number of open tables, standby database latency in seconds, current number of database connections, number of active database connections, maximum number of database connections, cache pool usage, buffer pool usage, running status, thread status, physical reads, and logical reads.
[0088] The evaluation elements in the alarm element category include: database log space utilization alarm, data space utilization alarm, CPU utilization spike alarm, database master-slave switchover alarm, memory utilization spike alarm, database connection pool utilization alarm, abnormal running status alarm, standby database latency alarm, and other alarms that affect the stable operation of the database.
[0089] Database defect categories refer to the severity level of defects present in the currently used database instance, database agent, and other database component versions. These can be categorized into crash, severe, moderate, and minor levels. The severity level of the defect has different impacts on the stable operation of the database, and the urgency of fixing it also varies.
[0090] Normative checks refer to the standardization checks performed on the attributes of database objects. Evaluation elements within the normative check category include: floating-point type checks for table fields, foreign key checks for table fields, non-null attribute checks for table fields, percentage of prepared statements executed, full table scan checks, uncommitted transaction checks, SQL full table scan checks, redundant index checks, primary key checks, number of indexes checks, number of indexed fields checks, average row length checks, index filtering factor checks, table capacity checks, large object field type checks, and number of table rows.
[0091] The collected runtime data differs depending on the database type; therefore, the objects of runtime data statistics also differ. Taking MySQL as an example, the evaluation elements in the runtime data statistics categories include: average SQL statement execution time statistics, maximum SQL execution time statistics, total SQL execution time statistics, total table wait time statistics, average table lock wait time statistics, SQL execution count statistics, event information statistics, SQL sorting count statistics, index read / write count statistics, prepared SQL statement execution count statistics, database deadlock information statistics, and database blocked session statistics, among other statistical results during database operation.
[0092] The evaluation elements for security elements include: user permission scope, role permission scope, password management, and authorization of critical system permissions, etc., which are security-related elements for the stable operation of the database.
[0093] In this embodiment, comprehensive data on elements that may affect the stable operation of the database are collected. This includes data on each evaluation element in the categories of monitoring indicators, alarm elements, and operational data statistics. At the same time, key data on each evaluation element in the categories of standardization checks, security elements, and database defects are also obtained from within the database. Compared with manual evaluation or other evaluation methods, this approach can uncover deep-seated hidden dangers in the database, thereby improving the accuracy of database health assessment.
[0094] Optionally, in operational scenarios, databases may exhibit periodic and regular abnormal indicators. These indicators exist due to actual business needs, not because the database itself is malfunctioning. Therefore, to prevent these abnormal indicators from affecting subsequent database health assessments when used as evaluation factors, periodic and regular abnormal indicators need to be filtered. Specifically, before selecting the appropriate data collection plugin based on the type of database to be evaluated, the following steps should be performed for at least one of the multiple evaluation factor categories:
[0095] Based on historical element data of multiple original elements in an evaluation element category, periodic abnormal elements are screened out from the multiple original elements, and then the periodic abnormal elements are removed from the multiple original elements to obtain the evaluation elements in that evaluation element category.
[0096] In practice, machine learning models and soft voting classification methods are used to filter out at least one anomalous element from the historical element data of multiple original elements. Then, for at least one anomalous element, the following steps are performed: if the month-on-month and year-on-year parameters corresponding to the historical element data of an anomalous element meet preset conditions, then the anomalous element is determined to be a periodic anomalous element.
[0097] At least one of the aforementioned assessment element categories includes monitoring indicator categories and alarm element categories. In practical applications, for multiple original monitoring indicators within a monitoring indicator category, abnormal monitoring indicators are selected from the multiple original monitoring indicators using machine learning models and soft voting classification methods, based on historical element data of these original monitoring indicators. For each abnormal monitoring indicator, corresponding month-on-month and year-on-year parameters are determined based on its historical element data. Based on the month-on-month and year-on-year parameters, it is determined whether the abnormal monitoring indicator is a periodic abnormal indicator; if so, it is filtered out. After filtering all periodic abnormal indicators, the remaining original monitoring indicators are used as assessment elements for database health evaluation.
[0098] For multiple original alarm elements within an alarm element category, a machine learning model and soft voting classification method are used to filter out abnormal alarm elements based on historical element data. For each abnormal alarm element, corresponding month-on-month and year-on-year parameters are determined based on its historical element data. Based on the month-on-month and year-on-year parameters, it is determined whether the abnormal alarm element is a periodic abnormal element. If so, the periodic abnormal element is filtered out. After filtering all periodic abnormal elements, the remaining original alarm elements are used as evaluation elements for database health assessment.
[0099] It should be noted that this application can also filter out periodic and regular abnormal indicators during the database health assessment process, and this application does not make specific limitations in this regard.
[0100] In this embodiment, by filtering out periodic and regular abnormal indicators, key indicators (key anomaly monitoring points and key alarms) that affect the stable operation of the database are obtained as evaluation elements to conduct database health assessment, thereby improving the effectiveness of database health assessment.
[0101] In some embodiments, after obtaining the element data corresponding to multiple evaluation elements, the element data is subjected to validity and completeness checks. After the checks pass, the multiple evaluation elements are sequentially classified based on the obtained element data to obtain multiple evaluation element categories, and the category of each evaluation element is identified.
[0102] In some embodiments, for multiple evaluation element categories, the following steps are performed respectively: The element data of each evaluation element in an evaluation element category is compared with the corresponding decision threshold to obtain the decision result corresponding to each evaluation element. Then, based on the decision results of multiple evaluation elements in the evaluation element category, the evaluation result of the evaluation element category is determined.
[0103] Specifically, based on database-related development, usage, and operation and maintenance specifications, as well as the experience of database experts, corresponding decision thresholds should be established for each evaluation element during the evaluation process. For example, in some database specifications, because large tables are not user-friendly for subsequent maintenance and modification, it is not recommended that the number of data rows recorded in the table exceed 1 million. Accordingly, for the evaluation element of checking the number of data rows in the table, the corresponding decision threshold can be set to 1 million.
[0104] When conducting database health assessment, the data of each assessment element is compared with the corresponding judgment threshold to obtain the judgment result for each assessment element. The judgment result includes a healthy state and an unhealthy state. For each assessment element category, the assessment score for that assessment element category is determined based on the number of assessment elements in the healthy state and the total number of assessment elements, specifically satisfying the following formula (1):
[0105] The evaluation score for each evaluation element category = (Number of evaluation elements in a healthy state / Total number of evaluation elements) * 100 ……………….(1)
[0106] For example, a standardization inspection category includes 29 evaluation elements. For each evaluation element, its data is compared with the corresponding judgment threshold. If the threshold is not exceeded, the judgment result for that evaluation element is set to 1, indicating that the evaluation element is in a healthy state. If the threshold is exceeded, the judgment result for that evaluation element is set to 0, indicating that the evaluation element is in an unhealthy state. After judging all evaluation elements, it is determined that 8 evaluation elements are in an unhealthy state and 21 evaluation elements are in a healthy state. Based on the above formula (1), the evaluation score for the standardization inspection category is 72.4 points.
[0107] Step S204: Combine the evaluation results of multiple evaluation element categories to determine the target health of the database to be evaluated.
[0108] In this embodiment, different data acquisition plugins are used to collect element data for different types of databases to meet the personalized data collection and processing needs of different types of databases in health assessment. It also supports the online addition of data collection for new types of databases later, making it widely applicable. Secondly, multiple assessment elements are classified to obtain multiple assessment element categories. Then, assessment elements in each category are judged separately to obtain corresponding assessment results. Finally, the assessment results of all assessment element categories are combined to obtain a database health score, achieving standardization, automation, periodicity, and self-service for database health assessment. Compared with manual assessment, this can effectively improve the accuracy and efficiency of database health assessment.
[0109] In some embodiments, the target health of the database to be evaluated is determined based on the evaluation results of multiple evaluation element categories and the weights of the multiple evaluation element categories.
[0110] Specifically, different weights are assigned to each evaluation element category in advance based on the importance of multiple evaluation element categories and their impact on database stability. Weight refers to the relative importance of a certain evaluation element category in the database health assessment; the higher the weight, the more important the evaluation element category. This application uses expert experience and a subjective weighting method to assign weights to each evaluation element category. It should be noted that the weights of the evaluation element categories can be adjusted according to actual operating conditions or needs. Furthermore, based on the evaluation results of multiple evaluation element categories and their weights, the target health of the database to be evaluated is determined to satisfy the following formula (2):
[0111]
[0112] Where T represents the target health of the database to be evaluated, X(i) represents the evaluation score of the evaluation feature category i, W(i) represents the weight corresponding to the evaluation feature category i, and N represents the total number of evaluation feature categories.
[0113] For example, the weight of the monitoring indicator category can be set to 0.2, the weight of the alarm element category to 0.2, the weight of the database defect category to 0.1, the weight of the standardization check category to 0.3, the weight of the operation data statistics category to 0.1, and the weight of the security element category to 0.1.
[0114] The evaluation scores for the monitoring indicator category were 92.4, the alarm element category were 80, the database defect category were 90, the standardization inspection category were 72.4, the operational data statistics category were 95.5, and the security element category were 98.
[0115] Using the above formula (2), it can be found that the target health score of the database to be evaluated is 84.55.
[0116] In this embodiment, different weights are assigned to different evaluation element categories, and then the evaluation scores of each evaluation element category are weighted and calculated based on the assigned weights to obtain the health score of the database. This quantifies the health evaluation results of the database, making it easier for database administrators and relevant personnel to continuously track the health status of the database, and also reduces the later tracking costs.
[0117] In some embodiments, after determining the target health of the database to be evaluated by integrating the evaluation results of multiple evaluation element categories, a database health report of the database to be evaluated is generated based on the target health and historical health of the database to be evaluated.
[0118] Specifically, a report generation template can be used to automatically generate a database health report for the database to be evaluated, based on the target health level and the historical health level of the database to be evaluated.
[0119] Furthermore, if potential vulnerabilities are identified during the database health assessment, expert experience is incorporated to provide modification and optimization solutions. For example, if a table lacks a primary key, it will be suggested to add one, along with potential issues arising from not doing so. Correspondingly, the database health report can be generated based on the target health level, historical health levels, and optimization plans. The database health report is presented in PDF format and includes a cover page, overview, analysis conclusions, basic database information, software version, resource usage, alarm analysis, and optimization recommendations.
[0120] After generating the database health report, a notification is triggered to the user to retrieve it promptly. For example, the system can retrieve the user's registered email address, phone number, or other information, and then notify the user via email or SMS that the database health report has been successfully generated, reminding them to retrieve it promptly.
[0121] In this embodiment, after comprehensively evaluating the results of multiple evaluation factor categories to determine the target health level of the database to be evaluated, a database health report is automatically generated, making the evaluation results visual and allowing users to understand the overall health status of the database, optimization suggestions, or problem remediation methods. Secondly, through the database health report, application developers, database designers, operations and maintenance personnel, and other relevant personnel can obtain first-hand information on potential problems in the database, reducing intermediate communication steps and lowering communication costs.
[0122] To better explain the embodiments of this application, the following describes a database health assessment method provided by the embodiments of this application in conjunction with a specific implementation scenario. The process of this method can be described by: Figure 1 The evaluation equipment shown performs the evaluation, wherein the structure of the evaluation equipment is as follows: Figure 3 As shown, it includes a configuration module 301, a task startup module 302, a data acquisition module 303, a data processing module 304, a scoring module 305, an information storage module 306, a scoring query module 307, a report generation module 308, and a user notification module 309.
[0123] The task startup module 302 is used to start the database health assessment task. The task startup module can be triggered in two ways: method 1 is triggered manually by the user, and method 2 is triggered by the configuration module 301 on a timed basis. In this case, the administrator predefines the initiation period in the configuration module 301.
[0124] The data acquisition module 303 is used to acquire element data corresponding to multiple evaluation elements from the database.
[0125] The data processing module 304 is used to classify multiple evaluation elements based on the collected element data, obtain multiple evaluation element categories, and then make a judgment on the evaluation elements in each evaluation element category to obtain the evaluation result of the corresponding evaluation element category.
[0126] The scoring module 305 is used to obtain the target health of the database based on the evaluation results of multiple evaluation element categories.
[0127] The information storage module 306 is used to store the target health status of the database and related information, including the database name, scoring time, and scoring initiator.
[0128] The rating query module 307 is used to receive query requests from other modules and return the target health score, historical health score, and related information from the database.
[0129] The report generation module 308 is used to generate a database health report based on the target health status, historical health status, optimization suggestions, monitoring information and alarm information of the database.
[0130] User notification module 309 is used to execute user notifications after the report is generated.
[0131] In practice, the configuration module 301 can adjust the evaluation switch, evaluation cycle, and evaluation range. The specific process is as follows: Figure 4 As shown, it includes the following steps:
[0132] Step 401, Begin.
[0133] Step 402: Respond to administrator actions and adjust the evaluation switch.
[0134] Step 403: Determine if the adjustment was successful. If yes, proceed to step 404; otherwise, proceed to step 408.
[0135] Step 404: Respond to administrator actions and adjust the evaluation cycle.
[0136] Step 405: Determine if the adjustment was successful. If yes, proceed to step 406; otherwise, proceed to step 408.
[0137] Step 406: Respond to administrator actions and adjust the evaluation scope.
[0138] Step 407: Determine if the adjustment was successful. If yes, proceed to step 409; otherwise, proceed to step 408.
[0139] Step 408, exception handling.
[0140] Step 409, End.
[0141] The process by which the data acquisition module 303 acquires element data corresponding to multiple evaluation elements from the database is as follows: Figure 5 As shown, it includes the following steps:
[0142] Step 501, Begin.
[0143] Step 502: Select the appropriate data acquisition plugin based on the database type.
[0144] Step 503: Collect monitoring data from the database of the monitoring platform using the data acquisition plugin.
[0145] Step 504: Determine if the data collection was successful. If yes, proceed to step 505; otherwise, proceed to step 517.
[0146] Step 505: Collect historical alarm information from the alarm platform using the data acquisition plugin.
[0147] Step 506: Determine if the data collection was successful. If yes, proceed to step 507; otherwise, proceed to step 517.
[0148] Step 507: Collect defect information from the database of the process platform through the data acquisition plugin.
[0149] Step 508: Determine if the data collection was successful. If yes, proceed to step 509; otherwise, proceed to step 517.
[0150] Step 509: Collect attribute information of each object from the database using the data acquisition plugin.
[0151] Step 510: Determine if the data collection was successful. If yes, proceed to step 511; otherwise, proceed to step 517.
[0152] Step 511: Collect runtime statistics from the database using the data acquisition plugin.
[0153] Step 512: Determine if the data collection was successful. If yes, proceed to step 513; otherwise, proceed to step 517.
[0154] Step 513: Collect security element information from the database using the data acquisition plugin.
[0155] Step 514: Determine if the data collection was successful. If yes, proceed to step 515; otherwise, proceed to step 517.
[0156] Step 515: Data normalization processing.
[0157] The collected data is standardized to obtain a uniform format, and then input into the data processing module 304.
[0158] Step 516: Determine whether normalization was successful. If yes, proceed to step 518; otherwise, proceed to step 517.
[0159] Step 517, exception handling.
[0160] Step 518, End.
[0161] The process by which the data processing module 304 obtains the evaluation results of the evaluation element categories is as follows: Figure 6 As shown, it includes the following steps:
[0162] Step 601, Begin.
[0163] Step 602, Data validity and integrity check.
[0164] Step 603: Determine whether all evaluation elements have been traversed. If yes, proceed to step 609; otherwise, proceed to step 604.
[0165] Step 604: Classify and label the current evaluation elements.
[0166] Specifically, based on the element data of the current evaluation element, the evaluation element category corresponding to the current evaluation element is determined, and the corresponding category information is identified.
[0167] Step 605: Determine if the flag is successful. If yes, proceed to step 606; otherwise, proceed to step 611.
[0168] Step 606: Make a judgment on the current evaluation elements.
[0169] The data of the current evaluation element is compared with the corresponding judgment threshold. If it is less than or equal to the judgment threshold, the judgment score of the current evaluation element is determined to be 1, indicating that the current evaluation element is in a healthy state. If it is greater than the judgment threshold, the judgment score of the current evaluation element is determined to be 0, indicating that the current judgment element is in an unhealthy state.
[0170] Step 607: Determine whether the judgment was successful. If yes, proceed to step 608; otherwise, proceed to step 611.
[0171] Step 608: Iterate through the next evaluation element and execute step 603.
[0172] Step 609: Evaluate each of the obtained evaluation element categories to obtain the corresponding evaluation score.
[0173] The evaluation score for each evaluation element category is determined based on the decision score of each evaluation element within the evaluation element category.
[0174] Step 610: Determine whether the evaluation was successful. If yes, proceed to step 612; otherwise, proceed to step 611.
[0175] Step 611, exception handling.
[0176] Step 612, End.
[0177] The scoring module 305 described above obtains the target health score of the database based on the evaluation results of multiple evaluation element categories, as follows: Figure 7 As shown, it includes the following steps:
[0178] Step 701, Begin.
[0179] Step 702, Data validity and integrity check.
[0180] Step 703: Determine whether the check has passed. If yes, proceed to step 704; otherwise, proceed to step 708.
[0181] Step 704: Combine the evaluation scores of multiple evaluation element categories to obtain the target health of the database.
[0182] Step 705: Determine whether the evaluation was successful. If yes, proceed to step 706; otherwise, proceed to step 708.
[0183] Step 706: Store the target health status and task-related information.
[0184] Step 707: Determine whether the storage was successful. If yes, proceed to step 709; otherwise, proceed to step 708.
[0185] Step 708, exception handling.
[0186] Step 709, End.
[0187] The process by which the above report generation module 308 generates the database health report is as follows: Figure 8 As shown, it includes the following steps:
[0188] Step 801, Begin.
[0189] Step 802, Data validity and integrity check.
[0190] Step 803: Call the rating query module 307 to obtain the historical health score.
[0191] Step 804: Call data processing module 304 to obtain optimization suggestions.
[0192] Step 805: Call the data acquisition module 303 to obtain monitoring information and alarm information.
[0193] Step 806: Generate a database health report.
[0194] Specifically, a database health report is generated based on the database's target health status, historical health status, optimization suggestions, monitoring information, and alarm information.
[0195] Step 807: Determine whether the generation was successful. If yes, proceed to step 808; otherwise, proceed to step 809.
[0196] Step 808: Output the database health report.
[0197] Step 809, exception handling.
[0198] Step 810, End.
[0199] In this embodiment, different data acquisition plugins are used to collect element data for different types of databases to meet the personalized data collection and processing needs of different types of databases in health assessment. It also supports the online addition of data collection for new types of databases later, making it widely applicable. Secondly, multiple assessment elements are classified to obtain multiple assessment element categories. Then, assessment elements in each category are judged separately to obtain corresponding assessment results. Finally, the assessment results of all assessment element categories are combined to obtain a database health score, achieving standardization, automation, periodicity, and self-service for database health assessment. Compared with manual assessment, this can effectively improve the accuracy and efficiency of database health assessment.
[0200] Based on the same technical concept, this application provides a schematic diagram of the structure of a database health assessment device, such as... Figure 9 As shown, the device 900 includes:
[0201] Select module 901 to select the appropriate data acquisition plugin based on the type of the database to be evaluated;
[0202] The acquisition module 902 is used to acquire element data corresponding to multiple evaluation elements of the database to be evaluated through the data acquisition plug-in;
[0203] The processing module 903 is used to classify the multiple evaluation elements based on the obtained element data to obtain multiple evaluation element categories; and to distinguish the evaluation elements in the multiple evaluation element categories to obtain the evaluation results of the multiple evaluation element categories.
[0204] The evaluation module 904 is used to integrate the evaluation results of the multiple evaluation element categories to determine the target health of the database to be evaluated.
[0205] Optionally, the processing module 903 is specifically used for:
[0206] For each of the aforementioned evaluation element categories, the following steps shall be performed:
[0207] The feature data of each evaluation element in an evaluation element category is compared with the corresponding decision threshold to obtain the decision result corresponding to each evaluation element;
[0208] The evaluation result of the evaluation element category is determined based on the judgment results of multiple evaluation elements in the evaluation element category.
[0209] Optionally, the evaluation module 904 is specifically used for:
[0210] Based on the evaluation results of the multiple evaluation element categories and their weights, the target health of the database to be evaluated is determined.
[0211] Optionally, the processing module 903 is further configured to:
[0212] Before selecting the appropriate data acquisition plugin based on the type of the database to be evaluated, the following steps are performed for at least one of the multiple evaluation element categories:
[0213] Based on historical element data of multiple original elements in an evaluation element category, periodic abnormal elements are screened out from the multiple original elements, and the periodic abnormal elements are removed from the multiple original elements to obtain the evaluation elements in the evaluation element category.
[0214] Optionally, the processing module 903 is specifically used for:
[0215] Using machine learning models and soft voting classification methods, at least one anomalous element is selected from the historical element data of the multiple original elements.
[0216] For each of the at least one anomalous elements, perform the following steps:
[0217] If the month-on-month and year-on-year parameters corresponding to the historical data of an abnormal element meet the preset conditions, then the abnormal element is determined to be a periodic abnormal element.
[0218] Optionally, the at least one evaluation element category includes a monitoring indicator category and an alarm element category.
[0219] Optionally, the evaluation module 904 is further configured to:
[0220] After determining the target health of the database to be evaluated by combining the evaluation results of the multiple evaluation element categories, a database health report is generated based on the target health and historical health of the database to be evaluated.
[0221] Optionally, the evaluation module 904 is specifically used for:
[0222] Based on the target health level, the historical health level, and the database optimization scheme, a database health report for the database to be evaluated is generated.
[0223] Optionally, the multiple evaluation element categories include monitoring indicator categories, alarm element categories, database defect categories, standardization inspection categories, operational data statistics categories, and security element categories.
[0224] In this embodiment, different data acquisition plugins are used to collect element data for different types of databases to meet the personalized data collection and processing needs of different types of databases in health assessment. It also supports the online addition of data collection for new types of databases later, making it widely applicable. Secondly, multiple assessment elements are classified to obtain multiple assessment element categories. Then, assessment elements in each category are judged separately to obtain corresponding assessment results. Finally, the assessment results of all assessment element categories are combined to obtain a database health score, achieving standardization, automation, periodicity, and self-service for database health assessment. Compared with manual assessment, this can effectively improve the accuracy and efficiency of database health assessment.
[0225] Based on the same technical concept, embodiments of this application provide a computer device, which can be... Figure 1 The evaluation equipment shown, such as Figure 10 As shown, it includes at least one processor 1001 and a memory 1002 connected to at least one processor. In this embodiment, the specific connection medium between the processor 1001 and the memory 1002 is not limited. Figure 10 Taking the connection between processor 1001 and memory 1002 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.
[0226] In this embodiment of the application, the memory 1002 stores instructions that can be executed by at least one processor 1001. By executing the instructions stored in the memory 1002, at least one processor 1001 can perform the steps of the above-described database health assessment method.
[0227] The processor 1001 is the control center of the computer device, capable of connecting various parts of the computer device via various interfaces and lines. It performs database health assessments by running or executing instructions stored in the memory 1002 and accessing data stored in the memory 1002. Optionally, the processor 1001 may include one or more processing units. The processor 1001 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1001. In some embodiments, the processor 1001 and the memory 1002 may be implemented on the same chip; in other embodiments, they may be implemented on separate chips.
[0228] The processor 1001 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.
[0229] Memory 1002, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 1002 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 1002 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer device, but is not limited thereto. In the embodiments of this application, memory 1002 may also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.
[0230] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the above-described database health assessment method.
[0231] Those skilled in the art will understand that embodiments of the present invention can be provided as methods or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0232] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer apparatus or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0233] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer device or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0234] These computer program instructions may also be loaded onto a computer device or other programmable data processing equipment to cause a series of operational steps to be performed on the computer device or other programmable equipment to produce a process implemented by the computer device, thereby providing instructions that execute on the computer device or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0235] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.
[0236] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.
Claims
1. A method for assessing the health of a database, characterized in that, include: Select the appropriate data acquisition plugin based on the type of database to be evaluated; The data acquisition plugin collects element data corresponding to multiple evaluation elements from the database to be evaluated. The storage location and data format of the evaluation element data are different for different types of databases. Based on the obtained element data, the multiple evaluation elements are classified to obtain multiple evaluation element categories; And to distinguish the evaluation elements in the multiple evaluation element categories respectively, and obtain the evaluation results of the multiple evaluation element categories; By combining the evaluation results of the multiple evaluation element categories, the target health of the database to be evaluated is determined; Before selecting the appropriate data acquisition plugin based on the type of database to be evaluated, the process also includes: For at least one of the multiple evaluation element categories, the following steps are performed respectively: based on the historical element data of multiple original elements in an evaluation element category, periodic abnormal elements are screened out from the multiple original elements, and the periodic abnormal elements are removed from the multiple original elements to obtain the evaluation elements in the evaluation element category. The periodic abnormal elements are periodic and regular abnormal indicators.
2. The method as described in claim 1, characterized in that, The step of distinguishing the evaluation elements in the multiple evaluation element categories to obtain the evaluation results for the multiple evaluation element categories includes: For each of the aforementioned evaluation element categories, the following steps shall be performed: The feature data of each evaluation element in an evaluation element category is compared with the corresponding decision threshold to obtain the decision result corresponding to each evaluation element; The evaluation result of the evaluation element category is determined based on the judgment results of multiple evaluation elements in the evaluation element category.
3. The method as described in claim 1, characterized in that, The determination of the target health of the database to be evaluated, based on the comprehensive evaluation results of the multiple evaluation element categories, includes: Based on the evaluation results of the multiple evaluation element categories and their weights, the target health of the database to be evaluated is determined.
4. The method as described in claim 1, characterized in that, The process of filtering out periodic anomalous elements from historical element data of multiple original elements within an evaluation element category includes: Using machine learning models and soft voting classification methods, at least one anomalous element is selected from the historical element data of the multiple original elements. For each of the at least one anomalous elements, perform the following steps: If the month-on-month and year-on-year parameters corresponding to the historical data of an anomalous element meet the preset conditions, then the anomalous element is determined to be a periodic anomalous element.
5. The method as described in claim 3, characterized in that, The at least one evaluation element category includes a monitoring indicator category and an alarm element category.
6. The method as described in claim 1, characterized in that, After determining the target health of the database to be evaluated by comprehensively considering the evaluation results of the multiple evaluation element categories, the method further includes: Based on the target health level and the historical health level of the database to be evaluated, a database health report is generated for the database to be evaluated.
7. The method as described in claim 6, characterized in that, The process of generating a database health report for the database to be evaluated based on the target health level and the historical health level of the database to be evaluated includes: Based on the target health level, the historical health level, and the database optimization scheme, a database health report for the database to be evaluated is generated.
8. The method according to any one of claims 1 to 7, characterized in that, The multiple evaluation element categories include monitoring indicator categories, alarm element categories, database defect categories, standardization inspection categories, operational data statistics categories, and security element categories.
9. A database health assessment device, characterized in that, include: The selection module allows you to choose the appropriate data acquisition plugin based on the type of database to be evaluated. The data acquisition module is used to acquire element data corresponding to multiple evaluation elements of the database to be evaluated through the data acquisition plugin. The storage location and data format of the evaluation element data are different for different types of databases. The processing module is used to classify the multiple evaluation elements based on the obtained element data to obtain multiple evaluation element categories; And to distinguish the evaluation elements in the multiple evaluation element categories respectively, and obtain the evaluation results of the multiple evaluation element categories; The evaluation module is used to integrate the evaluation results of the multiple evaluation element categories to determine the target health of the database to be evaluated; Before selecting the appropriate data acquisition plugin based on the type of database to be evaluated, the process also includes: For at least one of the multiple evaluation element categories, the following steps are performed respectively: based on the historical element data of multiple original elements in an evaluation element category, periodic abnormal elements are screened out from the multiple original elements, and the periodic abnormal elements are removed from the multiple original elements to obtain the evaluation elements in the evaluation element category. The periodic abnormal elements are periodic and regular abnormal indicators.
10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 8.
11. A computer-readable storage medium, characterized in that, It stores a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the method according to any one of claims 1 to 8.