Unified inspection task scheduling method and system for multiple databases

Abstract

The invention discloses a multi-database-oriented unified inspection task scheduling method and system, and relates to the technical field of database inspection, and the method comprises the steps: obtaining inspection element information, and triggering the execution of an inspection task according to an inspection period; after the execution of the inspection task is triggered, according to the database connection information and the inspection index, screening an inspection plug-in matched with the current database type, and after unified calling, executing a corresponding acquisition logic according to the inspection index; wherein the base class of the abstract inspection plug-in is defined to unify the interface specification of the inspection plug-in; acquiring an inspection result after the inspection task is executed, analyzing the inspection result, extracting a set key field, converting the set key field into a universal field, and packaging the inspection result into a standard structure body; and carrying out early warning on the standard structure body according to a set warning strategy. And an inspection platform with unified entrances, unified scheduling, unified standards and unified results is provided, and efficient operation and maintenance under various database environments are supported.

Description

Technical Field

[0001] This invention relates to the field of database inspection technology, and in particular to a unified inspection task scheduling method and system for multiple databases. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In current enterprise IT architectures, databases have become the core support for the stable operation of various systems. However, with the continuous enrichment of business systems, multiple types of database systems coexist within enterprises, such as Oracle, MySQL, OceanBase, and TiDB. This data diversity brings complexity to database operations and maintenance, especially during the inspection phase, where the challenges are particularly prominent.

[0004] Currently, existing database inspection methods have at least the following problems: Inspection logic is fragmented: different databases are maintained manually by different personnel using scripts, resulting in a lack of standardized management.

[0005] The scheduling methods are fragmented: it is impossible to set a unified task scheduling strategy, and it lacks the ability to retry on failure and track status.

[0006] The results are fragmented: the inspection results of each database are stored in different files or tables, making it difficult to form a complete view.

[0007] Lack of intelligent analysis capabilities: It only records inspection results and lacks historical comparison, trend analysis, or automatic anomaly identification functions. Especially when hidden risks emerge, such as a continuous increase in the number of connections or fluctuations in lock wait time, traditional methods often fail to provide timely warnings, affecting the efficiency of fault handling. Summary of the Invention

[0008] To address the aforementioned issues, this invention proposes a unified inspection task scheduling method and system for multiple databases. In response to the trend of intelligent and standardized operation and maintenance, it provides an inspection platform with "unified entry point, unified scheduling, unified standards, and unified results," supporting efficient operation and maintenance in various database environments.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a unified inspection task scheduling method for multiple databases, comprising: Obtain inspection metadata to trigger the execution of inspection tasks according to the inspection cycle; After triggering the inspection task, the system selects inspection plugins that match the current database type based on the database connection information and inspection indicators. After unified invocation, the corresponding collection logic is executed according to the inspection indicators. In this process, an abstract base class for inspection plugins is defined to unify the interface specifications of the inspection plugins. Obtain the inspection results after the inspection task is executed. By parsing the inspection results, extract the set key fields, convert them into general fields, and then encapsulate the inspection results into a standard structure. Warnings are issued to standard structures based on the set alarm strategy.

[0010] As an optional implementation method, the inspection metadata includes the business team to which the database belongs, the environment in which the database is located, the database connection information, the inspection cycle, the inspection indicators, and the alarm policy.

[0011] As an alternative implementation, by scanning the specified inspection plugin path, all implementation classes are automatically loaded and registered in the registry, so that each database plugin only needs to inherit the base class and implement the specific logic.

[0012] As an alternative implementation, the process of encapsulating the inspection results into a standard structure includes: The parser is configured to select a parsing scheme based on the inspection indicators, supporting row and column structures returned by SQL, single / multi-value key-value pairs, and log strings; Based on the configured fields, the template is extracted. The key fields extracted include value, unit, threshold, and object name. It supports JSON path location, regular expression extraction, and table row and column mapping. Based on the configured field mapping table, key fields are converted into general fields.

[0013] As an alternative implementation method, the process of issuing an early warning includes: A periodic-aware moving average is introduced, and based on a historical T-day time series window, the periodic pattern is extracted, and the current value is compared with the EMA mean of the same period point. A sliding window adaptive adjustment strategy is introduced to automatically adjust the window size according to the indicator volatility. A sudden change point detection algorithm is introduced to segment the trend, and the residuals within each trend segment are modeled and evaluated. After modeling, standardized anomaly scores are output, anomaly impact vectors are constructed, and global anomaly scores are calculated.

[0014] As an alternative implementation method, if an alarm is triggered, an anomaly is pushed and the results are archived according to the business team to which the database belongs; the alarm content contains a triple of "business team to which the database belongs, environment in which the database is located, and inspection indicators".

[0015] Secondly, the present invention provides a unified inspection task scheduling system for multiple databases, comprising: The triggering module is configured to acquire inspection metadata to trigger the execution of inspection tasks according to the inspection cycle. The loading module is configured to, after triggering the execution of the inspection task, filter the inspection plugins that match the current database type based on the database connection information and inspection indicators, and execute the corresponding collection logic according to the inspection indicators after unified invocation; wherein, an abstract base class for inspection plugins is defined to unify the interface specification of inspection plugins; The formatting module is configured to obtain the inspection results after the inspection task is executed. By parsing the inspection results, it extracts the set key fields, converts them into general fields, and then encapsulates the inspection results into a standard structure. The early warning module is configured to issue early warnings to standard structures based on a set alarm strategy.

[0016] Thirdly, the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0017] Fourthly, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in the first aspect.

[0018] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention proposes a unified inspection task scheduling method and system for multiple databases. After triggering inspection tasks according to the inspection cycle, inspection plugins matching the current database type are selected based on database connection information and inspection indicators. These plugins are then uniformly invoked and executed according to the inspection indicators, following the corresponding data collection logic. An abstract base class for inspection plugins is defined to unify the interface specifications of the plugins. Inspection results are obtained after task execution, and key fields are extracted and converted into general fields before being encapsulated into a standard structure. Warnings are then issued based on the set alarm policies for this standard structure. Facing the trend of intelligent and standardized operation and maintenance, this invention provides an inspection platform with "unified entry point, unified scheduling, unified standards, and unified results," supporting efficient operation and maintenance in various database environments.

[0020] This invention designs a standardized definition for multi-database tasks, unifies the definition of inspection tasks through a template mechanism to reduce the difficulty of accessing different databases; facilitates historical record comparison, trend prediction, and anomaly focusing through unified data format and structured storage; and supports historical retention of inspection data and display of trend curves through result dumping and trend visualization capabilities, providing data support for database performance evaluation.

[0021] In traditional methods, scripts are tightly bound to the scheduling engine, and adding a new database requires writing and injecting scripts. In contrast, this invention abstracts the inspection logic into a unified interface, supporting object-oriented polymorphic extensions. During loading, the corresponding module is automatically instantiated based on the database type, without requiring changes to the main program logic, thus achieving modular encapsulation of inspection logic and on-demand loading.

[0022] Traditional methods output inspection results as a unified log, lacking structured content; while the method of this invention completes the integration of multi-source structures through steps such as field extraction, rule transformation, and JSON encapsulation; it supports complex multi-level indicator structure output, which is beneficial for subsequent statistical analysis and realizes the structured and unified format output of multi-source results.

[0023] Traditional methods lack indicator fluctuation analysis capabilities. The method of this invention provides an intelligent feedback mechanism that generates time-series curves and detects outliers, realizing a closed-loop process of "data fluctuation → trend modeling → anomaly prediction → alarm linkage".

[0024] Traditional methods only record inspection results in log form; the method of this invention uniformly stores the inspection results in a structured database and builds a traceable query based on multi-dimensional fields; it also supports in-depth inspection analysis capabilities such as periodic task comparison and indicator fluctuation screening.

[0025] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0027] Figure 1 The flowchart is for a unified inspection task scheduling method for multiple databases provided in Embodiment 1 of the present invention. Detailed Implementation

[0028] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0029] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0030] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. Furthermore, it should be understood that the terms “comprising” and “including”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0032] Example 1 This embodiment provides a unified inspection task scheduling method for multiple databases, such as... Figure 1 As shown, it includes: Obtain inspection metadata to trigger the execution of inspection tasks according to the inspection cycle; After triggering the inspection task, the system selects inspection plugins that match the current database type based on the database connection information and inspection indicators. After unified invocation, the corresponding collection logic is executed according to the inspection indicators. In this process, an abstract base class for inspection plugins is defined to unify the interface specifications of the inspection plugins. Obtain the inspection results after the inspection task is executed. By parsing the inspection results, extract the set key fields, convert them into general fields, and then encapsulate the inspection results into a standard structure. Warnings are issued to standard structures based on the set alarm strategy.

[0033] The method of this embodiment will be described in detail below.

[0034] S1: Inspection Task Configuration: The inspection metadata is obtained from the task configuration center and uniformly encapsulated into a configuration object InspectionTaskMeta. The inspection metadata includes: the business team to which the database belongs, the environment in which the database is located, the database connection information, the inspection cycle, the inspection indicators, and the alarm policy.

[0035] Specifically: The database belongs to the business team (project_group), which is used for attribution tag classification and alarm policy matching when storing results; The database environment, including production, pre-release, and testing environments, is used for scheduling strategies (whether it is production-level), separation of metric threshold configurations, and dimensional division of result storage, which affect the determination of metric thresholds and alarm levels. Database connection information (db_connection_info), including database IP, port, username, authentication method, etc., is used to connect to the database when the inspection plugin is executed; The inspection cycle (inspection_cycle), including minute-level, hour-level, daily, etc., is passed to the scheduling center for periodic task registration and setting the trigger frequency; The inspection metrics (metric_set) specify the metrics that need to be collected for this task, such as the number of connections, the number of slow SQL queries, and lock waits. The alert policy defines the rules for judging abnormal indicators. For example, if an indicator exceeds the threshold three times in a row, an alert will be triggered. It is used to determine whether to trigger an alert and push it when performing trend judgment and anomaly detection.

[0036] S2: Based on the obtained inspection cycle, the inspection task is triggered and executed periodically.

[0037] Specifically, when registering a periodic task, the inspection cycle field is read to generate an APScheduler Job object; a cron expression or interval type is set to ensure that the task is executed periodically at the configured frequency.

[0038] In this embodiment, periodic task registration, execution, and failure retry are implemented based on a unified scheduling center (such as APScheduler); concurrent scheduling, task sharding, and parallel execution are supported to improve the overall system throughput.

[0039] S3: After triggering the inspection task, based on the database connection information and inspection indicators, filter the inspection plugins that match the current database type, and execute the corresponding collection logic according to the inspection indicators after unified invocation; in particular, an abstract base class for inspection plugins is defined to unify the interface specifications of inspection plugins.

[0040] Specifically, it includes: Based on the database type, select the corresponding inspection plugin, such as OracleInspectionTask or MySQLInspectionTask, and inherit and implement the base class interface; A unified abstract base class is adopted, and plugins that implement the abstract base class are loaded to encapsulate the core interfaces of all inspection tasks (such as connect(), collect_metrics(), output()). All modules are loaded on demand as plug-ins, decoupling the inspection logic from the core framework and facilitating the subsequent expansion of more databases or indicators.

[0041] Specifically, when loading the plugin, the connection string and authentication information are obtained from InspectionTaskMeta.db_connection_info (database connection information of the configuration object); The plugin instance selectively executes the collection logic of the listed metrics based on the inspection metrics (metric_set), avoiding redundant execution and improving efficiency; The plugin is based on a unified interface abstraction, such as the collect_metrics(self, metric_list) method.

[0042] Furthermore: Based on an interface-oriented execution engine, it achieves modular and decoupled loading of different database inspection logics, supports plug-in-based on-demand registration and invocation, including the following five key mechanisms: 1) The purpose of defining a unified interface using an abstract base class is to achieve logical decoupling; Define an abstract base class for inspection plugins, BaseInspectionPlugin, to standardize the interface of inspection plugins.

[0043] The pseudocode example is as follows: class BaseInspectionPlugin(ABC): def __init__(self, connection_info: dict, metric_config: dict): self.conn_info = connection_info self.metric_config = metric_config @abstractmethod def support(self) ->List[str]: """Returns the database types supported by this plugin (e.g., MySQL, Oracle)"" pass @abstractmethod def collect(self) ->List[Dict]: """Execute the actual metric collection logic and return structured results."" pass Each database plugin only needs to inherit from this base class and implement the specific logic, without having to worry about system details such as the scheduler, upstream data format, and result structuring, thus achieving high cohesion and low coupling.

[0044] 2) Plugin Module Registry Center (for Interface Adaptation) At startup, the system scans the specified plugin paths and automatically loads and registers all implementation classes into the PluginRegistry registry.

[0045] The pseudocode example is as follows: class PluginRegistry: registry: Dict[str, List[BaseInspectionPlugin]] = defaultdict(list) @classmethod def register(cls, plugin_class): for db_type in plugin_class().support(): cls.registry[db_type].append(plugin_class) # Automatic registration in plugin files @PluginRegistry.register class MySQLConnectionPlugin(BaseInspectionPlugin): def support(self): return ["mysql"] def collect(self): ... Compared to the existing technology of hard-coding the script path, this method is more maintainable and scalable, and supports hot-swapping and on-demand loading.

[0046] 3) The scheduler loads plugins on demand (not full script execution). After receiving a task scheduling request, the scheduler dynamically filters and matches plugins in the PluginRegistry based on the database type db_type (such as "mysql") and the inspection metric_set specified in the inspection metadata.

[0047] The pseudocode example is as follows: def load_plugins_for_task(task_meta): plugins = PluginRegistry.registry.get(task_meta.db_type) result = [] for plugin_cls in plugins: if plugin_cls.metric in task_meta.metric_set: result.append(plugin_cls(task_meta.conn_info, plugin_cls.metric_config)) return result Compared to existing technologies that execute "script-level granularity" such as oracle_check.sh and mysql_check.sh, the method in this embodiment supports "index-level granularity of plugin combination loading," which provides more granular and precise control.

[0048] 4) Unified scheduling and execution by the execution engine (logical isolation across databases) The execution engine calls the unified collect() method to all plugins without needing to worry about the underlying implementation logic; it supports concurrent execution and database isolation.

[0049] The pseudocode example is as follows: def execute_plugins(plugin_list): results = [] for plugin in plugin_list: try: res = plugin.collect() results.extend(res) except Exception as e: logger.warn(f"Plugin execution failed: {e}") return results All plugins follow the collect() interface.

[0050] Each plugin executes its logic independently and connects to an independent database instance, without interfering with each other.

[0051] 5) Enhanced scalability and version compatibility (supports evolution); support for PostgreSQL and MongoDB can be added as needed, requiring only the addition of plugin classes without modifying the scheduling layer and execution engine code, which conforms to the design principle of being open for extension and closed to modification.

[0052] In this embodiment, the inspection results obtained after the inspection task is executed (such as the number of connections, lock wait, tablespace utilization, etc.) can be linked with preset alarm rules to trigger real-time risk alarms. For example, if the number of connections exceeds 100 or the tablespace utilization rate exceeds 80%, the alarm rule will be triggered immediately and pushed to the operation and maintenance personnel, forming a closed loop of "collection-judgment-response", which is no longer just "drawing diagrams".

[0053] Historical trend data can be provided for subsequent trend modeling and health score calculation, volatility analysis, periodic detection, and participation in subsequent health score calculations, such as "the number of lock-up waiting times has increased abnormally in the last 7 days, and the stability score has decreased."

[0054] It can also be associated with meta-information such as tasks, environments, and tags in the platform for multi-dimensional analysis and root cause tracing; for example, the system can be trained to create "indicator profiles under different businesses" to identify potential risks in trends such as "surge in connection count" and "sudden increase in IO".

[0055] Here, the metric is a structured, defined Metric object, for example: { "metric_name": "active_connection_count", "db_type": "mysql", "sql": "SELECT COUNT(*) FROM information_schema.processlist WHERECOMMAND != 'Sleep';", "threshold": 100, "unit": "connections", "warn_level": "high" } Each metric configuration includes fields such as data collection method, database type, unit, threshold, and level.

[0056] This embodiment uses a unified Metric configuration structure to manage inspection metrics, and supports semantic mapping and collection adaptation mechanisms for metrics under different database types.

[0057] For example, for the "lock wait" metric, MySQL uses the performance_schema view, while Oracle uses a join query between v$session and v$lock.

[0058] The indicator collection logic is encapsulated as a plugin, which automatically matches the database type and performs adaptive collection. The collection results are stored in a structured manner for subsequent rule matching and trend analysis.

[0059] S4: Obtain the inspection results after the inspection task is executed. By parsing the inspection results, extract the set key fields, convert them into general fields, and then encapsulate the inspection results into a standard structure.

[0060] The inspection results returned by the inspection task are processed through a structured workflow, which includes the following five stages: Multi-format parsing: Supports parsing different formats such as SQL result sets, key-value pairs, log text, and nested JSON; Field extraction: Extract key field information using methods such as JSONPath, regular expressions, and column mapping; Field mapping: Standardize the fields returned by the plugin to a platform-wide semantic. Structure encapsulation: Encapsulated into a unified indicator structure, containing fields such as indicator code, value, unit, threshold, alarm level, and time; Storage to disk: Structured results are written to a standard indicator database for subsequent trend analysis, curve display, alarm triggering, etc. Compared to the simple "log standardization" or "unified format output" in existing technologies, this embodiment constructs a configurable, scalable, and adaptable structural modeling engine, which has stronger robustness and implementation capabilities in supporting multi-source heterogeneous plugins and cross-database collection tasks.

[0061] The structured JSON contains embedded fields for the database's environment and project group, for example: { "project_group": "Order System", "environment": "prod", "metric_name": "active_connections", "value": 103, "timestamp": "2025-05-29T10:20:00Z" } Used for subsequent multidimensional query analysis and horizontal comparison (e.g., test vs. production).

[0062] The specific steps are as follows: (1) Multi-format parsing; The goal is to support the conversion of various return formats (raw SQL output, tables, key-value pairs, logs, nested objects, etc.) into a standard intermediate structure.

[0063] The implementation methods include: using a strategy pattern + registry mechanism to automatically select a parsing scheme based on the format parser bound to the indicator configuration.

[0064] The following input types are supported: row and column structure returned by SQL (cursor.fetchall); single-value / multi-value key-value pairs (such as Redis, Mongo, HBase); raw log strings (such as "disk usage=82%, warn threshold=80%").

[0065] Technical challenges: The return structures of various data sources are heterogeneous, and the developers of the indicator collection plugins are not uniform, which can easily lead to different return formats; in order to maintain consistency, a unified "standard row and column structure" is designed as an intermediate conversion target.

[0066] (2) Field extraction; The goal is to extract key fields, such as values, units, thresholds, and object names, from unstructured or semi-structured data.

[0067] Implementation: Provides configurable field extraction templates, supports JSON path location (supports nested extraction), regular expression extraction (supports parsing of raw string logs), and table row and column mapping.

[0068] extractor: type: regex pattern: "disk usage=(\d+)%" target_field: "disk_usage" Technical challenges: Different plugins return content with different styles, making it impossible to hardcode the parsing logic; it is necessary to support plugin developers in customizing configurations to extract logic and dynamically inject it.

[0069] (3) Field mapping; The goal is to map the fields returned by the plugin to platform-wide fields (such as “usage” → “used_percent”, or “connections” → “conn_cnt”).

[0070] Implementation: Each plugin is configured with a field mapping table, and the mapping is handled uniformly in the backend to maintain the semantic consistency of metrics in subsequent modules.

[0071] { "connections": "conn_cnt", "used": "used_percent", "status": "check_status } Technical challenges: Significant differences in naming conventions among development teams across databases and plugins; without field mapping, subsequent alarm and analysis modules have high logical coupling.

[0072] (4) Unified encapsulation into a standard structure (structural modeling).

[0073] The goal is to encapsulate all structured data into a unified JSON Schema data structure for use in alarm matching, trend analysis, and result display.

[0074] Example struct: { "metric_code": "conn_cnt", "db_type": "postgresql", "value": 102, "unit": "connections", "threshold": 100, "warn_level": "high", "check_time": "2025-05-29T08:00:00Z", "env": "prod", "host": "192.168.1.10" } Technical challenges: The structure fields need to be dynamically generated, and the structure must meet the requirements of other modules (alarms, curves, scores); if the fields are not standardized or are missing, it will cause the entire chain of subsequent modules to be abnormal. The method in this embodiment has a high degree of coupling.

[0075] (5) Data modeling and storage on disk; The goal is to store structured data in a unified indicator data table (such as MongoDB / PostgreSQL / ClickHouse) for subsequent trend queries and aggregation analysis.

[0076] Implementation method: Use a standardized indicator table to automatically complete platform-level fields (task ID, environment, time, plugin version, etc.).

[0077] S5: Issue warnings for standard structures based on the set alarm strategy.

[0078] Specifically, it includes: Use an alert policy for time series modeling and anomaly detection; store time series metrics in the structured results, such as connection count, lock wait, and space usage, using a time series database (such as InfluxDB); periodically construct the fluctuation trend curves of the metrics and generate historical analysis views based on the time series.

[0079] By introducing exponential moving average (EMA) and outlier detection algorithms, the indicators are evaluated for trends and anomalies are identified. When performing EMA moving average and anomaly detection for each indicator, the corresponding alert_policy rule is called; for example: policy = get_alert_policy(project_group, environment, metric_name).

[0080] The following strategies are supported: threshold rule (value>90); number of consecutive anomalies rule (3 consecutive anomalies); fitting error rate rule (deviation from the moving average exceeds a set proportion).

[0081] This embodiment does not apply the algorithm to general scenarios, but rather optimizes and innovates its combination to suit the specific characteristics of database inspection indicators, which exhibit significant "periodicity + abrupt changes + unstable fluctuations." For example: Connection count and CPU utilization show a clear daily cycle, with higher values ​​during the day and lower values ​​at night. Deadlock count and slow SQL count often exhibit sudden jumps. Tablespace utilization: shows a linear upward trend; If the original EMA or isolated forest algorithm is used for modeling, the following problems will occur: The judgment of periodic fluctuations caused a misjudgment of normal peaks as outliers; The smooth window is fixed and cannot adapt to abrupt changes. The lack of unified modeling for multiple indicators makes it impossible to detect interconnected anomalies (such as a simultaneous increase in connection count, lock wait, and IO usage).

[0082] This embodiment incorporates three "micro-innovations + customization" optimizations to create a practical "lightweight modeling engine," which is no longer a simple call.

[0083] Improvement 1: Since the traditional EMA cannot distinguish between periodic fluctuations and truly abnormal fluctuations, a period-aware moving average (PS-EMA) is introduced.

[0084] Specific implementation method: Based on the historical T-day time series window, extract the periodic pattern (such as daily cycle); compare the current value with the "EMA mean of the same period point" to improve the accuracy of anomaly detection.

[0085] For example, the CPU utilization rate at 12:00 every day is around 85%, and it is currently 90%. The original EMA considers it an outlier, while the cycle-aware model considers it normal.

[0086] Improvement point 2: Since the original independent modeling of indicators cannot identify "inter-indicator synergy anomalies", an indicator-level anomaly factor fusion model (MIF, Metric Influence Fusion) is introduced.

[0087] Specific implementation method: After modeling each indicator, output the standardized anomaly score (Z-score or IQR residual); construct the "anomaly impact vector": [conn_score, lock_score, io_score]; calculate the "global anomaly score" through weighted average, linear combination or machine learning model fusion.

[0088] It can identify the following potential problems: simultaneous increase in connection count, lock wait, and write TPS; hidden deadlock; simultaneous deterioration of CPU, IO, and remaining space due to full business operations; and potential OOM.

[0089] Improvement point 3: Since a fixed window length can cause an imbalance in the weights of old and new data or fail to capture sudden changes, a multi-segment trend modeling strategy that supports "dynamic sliding window + threshold self-adjustment" is introduced.

[0090] Specific implementation methods: Introduce a sliding window adaptive adjustment strategy: the window size is automatically adjusted according to the volatility of the indicator (if the volatility is large, it will be reduced), introduce a mutation point detection algorithm (such as BoCPD) to segment the trend, model and evaluate the residuals within each trend segment, and improve the mutation capture capability.

[0091] In summary, the improvements are as follows: (1) Periodic perception to avoid false alarms that periodic high values ​​are abnormal; (2) Multi-indicator fusion to support indicator linkage anomaly detection; (3) Dynamic trend modeling to improve adaptability to mutation and sliding window modeling.

[0092] In this embodiment, the trend analysis does not simply call existing time-series algorithms. Instead, it optimizes the model based on the characteristics of database inspection indicators in actual business environments, such as "periodic fluctuations + abrupt trends + coordinated changes of multiple indicators": 1. Periodic-aware moving average modeling: Establish a unique periodic model for each type of indicator to eliminate the interference of normal periodic fluctuations on anomaly detection.

[0093] 2. Multi-indicator anomaly factor fusion analysis: Construct anomaly scoring vectors for indicators and output global anomaly probabilities through rules or learning models to identify collaborative risks.

[0094] 3. Dynamic sliding window and trend segmentation mechanism: The sliding window length is automatically adjusted based on the volatility of historical data, and multi-segment modeling is performed in combination with the trend change point detection algorithm to improve the adaptability to nonlinear trend changes.

[0095] The above strategies constitute a lightweight, deployable database metric trend analysis and modeling engine that can accurately identify potential capacity risks, performance bottlenecks, and system vulnerabilities.

[0096] S6: If an alarm is triggered, an anomaly will be pushed and the results archived according to the business team to which the database belongs; The alarm content contains a triplet of "the business team to which the database belongs, the environment in which the database is located, and the inspection indicators", which makes it easy to pinpoint and locate the problem.

[0097] Meanwhile, it builds multi-dimensional query capabilities based on tag indexes (database type, host, indicator name, etc.); provides analysis methods such as abnormal indicator backtracking, trend tracking, and horizontal comparison; and automatically pushes abnormal results through configuration-driven email, Webhook, and other methods.

[0098] It should be noted that all data acquisition is conducted in accordance with laws and regulations and with user consent, and the data is used legally.

[0099] Example 2 This embodiment provides a unified inspection task scheduling system for multiple databases, including: The triggering module is configured to acquire inspection metadata to trigger the execution of inspection tasks according to the inspection cycle. The loading module is configured to, after triggering the execution of the inspection task, filter the inspection plugins that match the current database type based on the database connection information and inspection indicators, and execute the corresponding collection logic according to the inspection indicators after unified invocation; wherein, an abstract base class for inspection plugins is defined to unify the interface specification of inspection plugins; The formatting module is configured to obtain the inspection results after the inspection task is executed. By parsing the inspection results, it extracts the set key fields, converts them into general fields, and then encapsulates the inspection results into a standard structure. The early warning module is configured to issue early warnings to standard structures based on a set alarm strategy.

[0100] It should be noted that the above modules correspond to the steps described in Embodiment 1, and the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.

[0101] In further embodiments, the following is also provided: An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in Embodiment 1. For brevity, further details are omitted here.

[0102] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0103] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0104] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in Embodiment 1.

[0105] The method in Example 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0106] A computer program product includes a computer program that, when executed by a processor, implements the method described in Embodiment 1.

[0107] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which execute in a device on a target real or virtual processor to perform the processes / methods described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided among program modules as needed. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside in both local and remote storage media.

[0108] The computer program code used to implement the methods of the present invention may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the computer or other programmable data processing device, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a computer, partially on a computer, as a stand-alone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0109] In the context of this invention, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0110] Those skilled in the art will recognize that the units and algorithm steps described in conjunction with the embodiments herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0111] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A unified inspection task scheduling method for multiple databases, characterized in that, include: Obtain inspection metadata to trigger the execution of inspection tasks according to the inspection cycle; After triggering the inspection task, the system selects inspection plugins that match the current database type based on the database connection information and inspection indicators. After unified invocation, the corresponding collection logic is executed according to the inspection indicators. In this process, an abstract base class for inspection plugins is defined to unify the interface specifications of the inspection plugins. Obtain the inspection results after the inspection task is executed. By parsing the inspection results, extract the set key fields, convert them into general fields, and then encapsulate the inspection results into a standard structure. Warnings are issued to standard structures based on the set alarm strategy.

2. The unified inspection task scheduling method for multiple databases as described in claim 1, characterized in that, The inspection metadata includes the business team to which the database belongs, the environment in which the database is located, the database connection information, the inspection cycle, the inspection indicators, and the alarm policies.

3. The unified inspection task scheduling method for multiple databases as described in claim 1, characterized in that, By scanning the specified inspection plugin path, all implementation classes are automatically loaded and registered in the registry, so that each database plugin only needs to inherit the base class and implement the specific logic.

4. The unified inspection task scheduling method for multiple databases as described in claim 1, characterized in that, The process of encapsulating inspection results into a standard structure includes: The parser is configured to select a parsing scheme based on the inspection indicators, supporting row and column structures returned by SQL, single / multi-value key-value pairs, and log strings; Based on the configured fields, the template is extracted. The key fields extracted include value, unit, threshold, and object name. It supports JSON path location, regular expression extraction, and table row and column mapping. Based on the configured field mapping table, key fields are converted into general fields.

5. The unified inspection task scheduling method for multiple databases as described in claim 1, characterized in that, The process of issuing an early warning includes: A periodic-aware moving average is introduced, and based on a historical T-day time series window, the periodic pattern is extracted, and the current value is compared with the EMA mean of the same period point. A sliding window adaptive adjustment strategy is introduced to automatically adjust the window size according to the indicator volatility. A sudden change point detection algorithm is introduced to segment the trend, and the residuals within each trend segment are modeled and evaluated. After modeling, standardized anomaly scores are output, anomaly impact vectors are constructed, and global anomaly scores are calculated.

6. The unified inspection task scheduling method for multiple databases as described in claim 5, characterized in that, If an alarm is triggered, an anomaly will be pushed and the results archived according to the business team to which the database belongs; the alarm content contains a triple of "business team to which the database belongs, environment in which the database is located, and inspection indicators".

7. A unified inspection task scheduling system for multiple databases, characterized in that, include: The triggering module is configured to acquire inspection metadata to trigger the execution of inspection tasks according to the inspection cycle. The loading module is configured to, after triggering the execution of the inspection task, filter the inspection plugins that match the current database type based on the database connection information and inspection indicators, and execute the corresponding collection logic according to the inspection indicators after unified invocation; wherein, an abstract base class for inspection plugins is defined to unify the interface specification of inspection plugins; The formatting module is configured to obtain the inspection results after the inspection task is executed. By parsing the inspection results, it extracts the set key fields, converts them into general fields, and then encapsulates the inspection results into a standard structure. The early warning module is configured to issue early warnings to standard structures based on a set alarm strategy.

8. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, perform the method described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the method described in any one of claims 1-6.

Images