Information mining methods, apparatus, equipment, readable storage media, and program products

Abstract

This application provides an information mining method, apparatus, device, readable storage medium, and program product. Embodiments of this application can be applied to scenarios such as cloud technology, artificial intelligence, smart transportation, and vehicle-mounted systems. The method includes: collecting data multiple times from monitoring indicators of a database to obtain multiple indicator collection data, and recording multiple collection times; pairing and organizing the multiple indicator collection data according to the multiple collection times to obtain a data pair to be analyzed; wherein the collection times corresponding to the indicator collection data in the data pair to be analyzed differ by one or more time periods; and mining trend indicators from the monitoring indicators based on the internal differences of the data pair to be analyzed, using these as the information mining results. This application can improve the intelligence level of trend indicator mining.

Description

Technical Field

[0001] This application relates to database technology in cloud technology, and more particularly to an information mining method, apparatus, device, readable storage medium, and program product. Background Technology

[0002] Databases are prone to trend-related issues, which, if left unresolved, can pose serious security risks. Trend indicators are monitoring metrics that show a certain trend over time. To avoid these issues, appropriate trend indicators are typically selected for the database, and these indicators are used to predict potential trend problems.

[0003] In related technologies, trend indicators are usually discovered manually, such as by directly specifying a data tablespace as a trend indicator. However, manually discovering trend indicators results in low efficiency and accuracy, ultimately leading to a low level of intelligence in trend indicator discovery. Summary of the Invention

[0004] This application provides an information mining method, apparatus, device, readable storage medium, and program product, which can improve the intelligence level of trend indicator mining.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides an information mining method, including:

[0007] Multiple data collections were performed on the database monitoring metrics to obtain multiple metric data collection times, and the data collection times were recorded.

[0008] Based on the multiple collection times, the data collected by the multiple indicators are paired and organized to obtain a data pair to be analyzed; wherein the collection times corresponding to the data collected by the indicators in the data pair to be analyzed differ by one or more time periods.

[0009] Based on the internal differences of the data pairs to be analyzed, trend indicators that show a trend in data changes are extracted from the monitoring indicators, and these are used as information mining results.

[0010] This application provides an information mining device, comprising:

[0011] The data acquisition module is used to collect data from the database monitoring indicators multiple times, obtain multiple indicator data, and record multiple data collection times.

[0012] The data processing module is used to pair and process the collected data of the multiple indicators according to the multiple collection times to obtain a data pair to be analyzed; wherein the collection times corresponding to the collected data of the indicators in the data pair to be analyzed differ by one or more time periods.

[0013] The indicator mining module is used to extract trend indicators from the monitoring indicators based on the internal differences of the data pairs to be analyzed, so as to serve as the information mining results.

[0014] In some embodiments of this application, the data processing module is further configured to arrange the multiple indicator collection data according to the chronological order of the multiple collection times to obtain a data sequence; divide the data sequence according to the sequence length corresponding to the time period to obtain multiple sub-sequences; and obtain the data pair to be analyzed by pairing the indicator collection data of adjacent sub-sequences in the multiple sub-sequences.

[0015] In some embodiments of this application, the plurality of subsequences includes: N subsequences, where N is a positive integer greater than 1; the data processing module is further configured to determine the next subsequence of each of the first N-1 subsequences in the N subsequences as adjacent subsequences; pair the index collection data in the first N-1 subsequences with the index collection data in the corresponding adjacent subsequences to obtain N-1 data pairs; and determine the N-1 data pairs as the data pairs to be analyzed.

[0016] In some embodiments of this application, the data processing module is further configured to collect data for each indicator in the i-th subsequence of the first N-1 subsequences, and filter out matching data from the corresponding adjacent subsequences whose collection time differs from that of each indicator by one time period; wherein i is greater than or equal to 1 and less than or equal to N-1; pair each indicator collection data with the matching data to generate a sub-data pair corresponding to each indicator collection data; integrate the data pair corresponding to the i-th subsequence using the sub-data pairs corresponding to each indicator collection data; and obtain the N-1 data pairs when corresponding data pairs are generated for all of the first N-1 subsequences.

[0017] In some embodiments of this application, the data pairs to be analyzed include: N-1 data pairs; the indicator mining module is further configured to determine the trend characteristics corresponding to the monitoring indicator based on the difference set of each data pair in the N-1 data pairs; the trend characteristics represent that the collected data of the indicator shows any one of an upward trend, a downward trend, and a no-change trend; from the monitoring indicators, the trend indicators that represent that the collected data of the trend characteristics show the upward trend or the downward trend are selected.

[0018] In some embodiments of this application, the indicator mining module is further configured to construct a corresponding difference set for each of the N-1 data pairs; to count the number of positive and negative differences in the difference set to obtain the number of positive and negative differences for each data pair; and to determine the trend feature corresponding to the monitoring indicator based on the number of positive and negative differences.

[0019] In some embodiments of this application, the indicator mining module is further configured to determine the type of change trend of each data pair based on the relationship between the number of positive numbers and the number of negative numbers, and obtain the trend type; from the trend types of each data pair, the trend type that appears most frequently is selected as the trend feature corresponding to the monitoring indicator.

[0020] In some embodiments of this application, the indicator mining module is further configured to determine the trend type of each data pair as rising when the number of positive numbers is greater than the sum of the number of negative numbers and the obtained gap threshold; determine the trend type of each data pair as falling when the sum of the number of positive numbers and the gap threshold is less than the number of negative numbers; and determine the trend type of each data pair as no trend when the difference between the number of positive numbers and the number of negative numbers is less than the gap threshold.

[0021] In some embodiments of this application, the information mining device further includes: a time estimation module; the time estimation module is used to calculate the rate of change of the data pair corresponding to the i-th subsequence by using the difference between the data collected by each indicator and the matching data after obtaining the N-1 data pairs when corresponding data pairs are generated for the first N-1 subsequences; to obtain N-1 rates of change when all data pairs of the first N-1 subsequences are generated; and to estimate the abnormal time period based on the maximum and minimum rates of change among the N-1 rates of change; the abnormal time period represents the estimated time period during which the database will experience an anomaly.

[0022] In some embodiments of this application, the information mining device further includes: an information processing module; the information processing module is used to estimate the abnormal time period based on the maximum and minimum change rates among the N-1 change rates, and then generate an alarm message to prompt the database to be expanded based on the abnormal time period; or, when the difference between the current time point and the starting time point of the abnormal time period is less than a preset time threshold, the database is expanded to obtain an expanded database.

[0023] In some embodiments of this application, the indicator mining module is further configured to: filter the trend type that appears most frequently from the trend types of each data pair as the trend feature corresponding to the monitoring indicator; assign a corresponding feature value to the trend type of each data pair; and calculate the confidence level of the trend feature based on the feature value of each data pair and the total number of data pairs.

[0024] In some embodiments of this application, the indicator mining module is further configured to filter out trend indicators from the monitoring indicators where the trend feature characterization indicator data shows the upward trend or the downward trend, and the confidence level exceeds the confidence level threshold.

[0025] This application provides an information mining device, including:

[0026] Memory, used to store executable instructions;

[0027] The processor, when executing executable instructions stored in the memory, implements the information mining method provided in the embodiments of this application.

[0028] This application provides a computer-readable storage medium storing executable instructions, which, when executed by a processor, implement the information mining method provided in this application.

[0029] This application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the information mining method provided in this application.

[0030] The embodiments of this application have the following beneficial effects: The information mining device first collects data on the monitoring indicators of the database at multiple collection times, obtaining multiple indicator collection data. Then, based on the multiple collection times, the multiple indicator collection data are paired and organized, that is, indicator collection data with collection times differing by one or more time periods are paired to obtain data pairs to be analyzed. Finally, based on the internal differences of the indicator pairs to be analyzed, the changes in the monitoring indicator data in different time periods are identified. Finally, indicators with trends in data changes are mined from the monitoring indicators, obtaining trend indicators. In this way, the information mining device can not only automatically mine trend indicators, improving the efficiency of trend indicator mining, but also fully consider the periodicity of the monitoring indicator data during the trend indicator mining process, improving the accuracy of trend indicator mining. In summary, the embodiments of this application ultimately improve the intelligence level of trend indicator mining. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the architecture of the information mining system provided in the embodiments of this application;

[0032] Figure 2 This is a schematic diagram of the structure of the information mining device provided in the embodiments of this application;

[0033] Figure 3 This is a flowchart illustrating the information mining method provided in the embodiments of this application. Figure 1 ;

[0034] Figure 4 This is a flowchart illustrating the information mining method provided in the embodiments of this application. Figure 2 ;

[0035] Figure 5 This is a schematic diagram illustrating the process of grouping a data sequence according to an embodiment of this application;

[0036] Figure 6 This is a flowchart illustrating the information mining method provided in the embodiments of this application. Figure 3 ;

[0037] Figure 7 This is a schematic diagram illustrating the process of determining the trend characteristics of monitoring indicators provided in the embodiments of this application;

[0038] Figure 8 This is a system architecture diagram for mining trend indicators in a database, provided in an embodiment of this application.

[0039] Figure 9 This is a schematic diagram of the process for mining trend indicators provided in the embodiments of this application. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0041] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0042] In the following description, the terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0043] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0044] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0045] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0046] 1) Cloud technology refers to a managed technology that unifies a series of resources such as hardware, software, and networks within a wide area network or local area network to realize the computing, storage, processing, and sharing of data.

[0047] Cloud technology is a general term encompassing network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend servers of technical network systems require substantial computing and storage resources, such as those for video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring data to be transmitted to backend systems for logical processing. Data at different levels will be processed separately, and data from various industries will require robust system support, which can only be achieved through cloud computing.

[0048] 2) Cloud computing is a computing model that distributes computing tasks across a resource pool composed of a large number of computers, enabling various application systems to obtain computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, the resources in the "cloud" are infinitely scalable, readily available, on-demand, expandable, and pay-as-you-go.

[0049] 3) A database can be viewed as an electronic filing cabinet, a place to store electronic files. Users can perform operations such as adding, querying, updating, and deleting data in the files. A "database" is a collection of data stored together in a certain way, which can be shared by multiple users, has the lowest possible redundancy, and is independent of the degree of application.

[0050] 4) Monitoring metrics are indicators that reflect the working status of the database, and may include database throughput, concurrency, disk space usage, CPU utilization, and other metrics.

[0051] 5) Trend indicators are monitoring indicators that show a certain trend over time. For example, data table space and disk capacity tend to decrease over time, which are obvious trend indicators.

[0052] Databases are prone to certain recurring problems. If these problems are not addressed, they can pose serious security risks. For example, insufficient disk space can prevent write requests from being fulfilled, or even corrupt data and compromise its integrity. Excessive CPU utilization can reduce database throughput, thereby affecting normal business operations.

[0053] Trend indicators are monitoring metrics that show a certain trend over time. To avoid trend problems in databases, appropriate trend indicators are usually selected for the database, and these selected trend indicators are used to predict potential trend problems in the database.

[0054] In related technologies, trend indicators are typically mined manually, such as by directly specifying a data tablespace as a trend indicator. However, databases have numerous monitoring indicators, some with obvious trends, while others are difficult for database administrators to perceive. Furthermore, in cloud environments, business processes are complex; some monitoring indicators may show trends in one business area but not in another, meaning trend indicators are also business-specific. For these reasons, related technologies often require a considerable amount of time to mine suitable trend indicators for the database, and the accuracy of the mined indicators may be low. Therefore, manually mining trend indicators in these technologies is both inefficient and inaccurate, ultimately resulting in low intelligence in trend indicator mining.

[0055] Furthermore, the low accuracy of trend indicator mining can lead to the inability to issue timely warnings for certain trend issues, resulting in lower database security.

[0056] This application provides an information mining method, apparatus, device, readable storage medium, and program product, which can improve the intelligence level of trend indicator mining. The following describes exemplary applications of the information mining device provided in this application. The information mining device provided in this application can be implemented as various types of terminals such as laptops, tablets, desktop computers, mobile devices, and vehicle terminals, or as a server, or as a device cluster composed of terminals and servers. The following will describe exemplary applications when the information mining device is implemented as a server.

[0057] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of the information mining system provided in this application embodiment. To support an information mining application, in the information mining system 100, the server 200 connects to the business party's database 400-1 and the database management terminal 400-2 via a network 300. The network 300 can be a wide area network or a local area network, or a combination of both.

[0058] Database 400-1 is used to store business data from business partners.

[0059] Server 200 is used to collect data multiple times for the monitoring indicators of database 400-1, obtaining multiple indicator collection data and recording multiple collection times; based on the multiple collection times, the multiple indicator collection data are paired and organized to obtain data pairs to be analyzed; wherein the collection times corresponding to the indicator collection data in the data pairs to be analyzed differ by one or more time periods; based on the internal differences of the data pairs to be analyzed, trend indicators with trends in data changes are extracted from the monitoring indicators as information mining results to achieve information mining.

[0060] Server 200 is also used to send trend indicators to the database management terminal 400-2, so that administrators can view the trend indicators at any time on the database management terminal 400-2.

[0061] In some embodiments, server 200 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Management terminal 400-2 may be a smartphone, tablet, laptop, vehicle terminal, desktop computer, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment of the invention.

[0062] See Figure 2 , Figure 2 This is a schematic diagram of the structure of the information mining device provided in the embodiments of this application. Figure 2 The information mining device 500 shown includes at least one processor 510, a memory 550, at least one network interface 520, and a user interface 530. The various components in the information mining device 500 are coupled together via a bus system 540. It is understood that the bus system 540 is used to implement communication between these components. In addition to a data bus, the bus system 540 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 540.

[0063] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0064] User interface 530 includes one or more output devices 531 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 530 also includes one or more input devices 532, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0065] The memory 550 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 550 may optionally include one or more storage devices physically located away from the processor 510.

[0066] The memory 550 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 550 described in this application embodiment is intended to include any suitable type of memory.

[0067] In some embodiments, memory 550 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0068] Operating system 551 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0069] The network communication module 552 is used to reach other computing devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 including: Bluetooth, Wi-Fi, and Universal Serial Bus (USB), etc.

[0070] Presentation module 553 is used to enable the presentation of information (e.g., user interface for operating peripheral devices and displaying content and information) via one or more output devices 531 (e.g., display screen, speaker, etc.) associated with user interface 530.

[0071] The input processing module 554 is used to detect and translate one or more user inputs or interactions from one or more input devices 532.

[0072] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 An information mining apparatus 555 stored in memory 550 is shown. This apparatus can be software in the form of programs and plug-ins, and includes the following software modules: a data acquisition module 5551, a data processing module 5552, an indicator mining module 5553, a time estimation module 5554, and an information processing module 5555. These modules are logically connected and can therefore be arbitrarily combined or further divided according to the functions they implement. The functions of each module will be described below.

[0073] In other embodiments, the information mining apparatus provided in this application can be implemented in hardware. As an example, the information mining apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the information mining method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0074] For example, an embodiment of this application provides an information mining device, including:

[0075] Memory, used to store executable instructions;

[0076] The processor, when executing executable instructions stored in the memory, implements the information mining method provided in the embodiments of this application.

[0077] In some embodiments, the information mining device can implement the information mining method provided in this application by running a computer program. For example, the computer program can be a native program or software module in an operating system; it can be a native application (APP), i.e., a program that needs to be installed in the operating system to run; it can also be a small program, i.e., a program that only needs to be downloaded to a browser environment to run; or it can be a small program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module, or plugin.

[0078] The information mining method provided in this application can be applied to various application scenarios such as cloud technology, artificial intelligence, smart transportation, and vehicle technology. Below, the information mining method provided in this application will be described in conjunction with exemplary applications and implementations of the information mining equipment provided in this application.

[0079] See Figure 3 , Figure 3 This is a flowchart illustrating the information mining method provided in the embodiments of this application. Figure 1 , will combine Figure 3 The steps shown are explained.

[0080] S101. Perform multiple data collections on the monitoring indicators of the database to obtain multiple indicator collection data, and record the multiple collection times of the data collection.

[0081] This application embodiment is implemented in a scenario of mining trend indicators in a database. For example, it mines indicators in a business database that show trend changes related to business scenarios, and it mines indicators in a general database that show trend changes that are not easily noticeable. In this application embodiment, monitoring indicators reflect the working status of the database and can reflect the database's situation in various aspects. When they show a trend over time, they are trend indicators. Therefore, to mine trend indicators, the specific values ​​of the monitoring indicators must first be collected before the patterns of the monitoring indicators over time can be determined. Based on this, in this application embodiment, at the beginning of the information mining process, the information mining device first performs multiple data collections on the monitoring indicators, collecting the specific values ​​of the monitoring indicators at each collection, i.e., obtaining multiple indicator collection data, and simultaneously recording the time of data collection, i.e., obtaining multiple collection times.

[0082] It should be noted that monitoring metrics can include metrics obtained from monitoring database resources, such as one or more of the following: disk space usage, memory usage, memory utilization, and disk utilization. Monitoring metrics can also include metrics obtained from monitoring the database engine, such as one or more of the following: maximum number of connections, current number of connections, and access volume. Of course, monitoring metrics can also include other database metrics, which are not limited here.

[0083] Furthermore, when the monitoring indicators include only one type of indicator, the information mining equipment performs information mining to determine whether that single indicator is a trend indicator. When the monitoring indicators include multiple different types of indicators, the information mining equipment performs information mining to filter out trend indicators from these multiple types of indicators.

[0084] It is understandable that the information mining device can collect data on the monitoring indicators of the database at preset time intervals to obtain multiple indicator data, or it can collect data on the monitoring indicators of the database at multiple different random times to obtain multiple indicator data. Therefore, in this embodiment, the multiple collection times can be multiple time points with preset time intervals, or multiple random time points.

[0085] It should be noted that the database in the embodiments of this application can be any one or more of relational databases, key-value databases, document databases, and columnar storage databases.

[0086] Relational databases (such as MySQL) are well-suited for Online Transaction Processing (OLTP) workloads. OLTP is the primary application of traditional relational databases, mainly handling basic, routine transactions and recording immediate add, delete, update, and query operations. A key performance indicator for OLTP is system performance, specifically real-time response time—the time required to respond to a user's request. OLTP workloads are characterized by high real-time requirements (aiming to ensure that transactional applications only write the necessary data to process individual transactions as quickly as possible), relatively small data volumes, generally deterministic transactions (i.e., accessing deterministic data, such as deposits and withdrawals with specific amounts), and support for a large number of concurrent transactions (a large number of concurrent users periodically adding and modifying data, requiring high concurrency and strict integrity and security of transactions).

[0087] Key-value databases (such as Redis) store the database as a collection of key-value pairs, where the values ​​store the actual data and the keys are unique identifiers for retrieving data. Both keys and values ​​can be anything from simple to complex objects. Key-value databases are highly distinguishable and allow for horizontal scaling at a scale unattainable by other database types. They are primarily used for content caching, handling large amounts of data under high access loads, and sometimes for logging systems. Key-value databases offer very fast lookup speeds.

[0088] Document-oriented databases (such as MongoDB) are non-relational databases primarily used to store, index, and manage document-oriented data or semi-structured data. In document-oriented databases, data is grouped into "collections," each with a unique name and capable of containing an unlimited number of documents. These "collections" are similar to "tables" in relational databases, the only difference being that "collections" do not have a defined structure. Relational databases store data as a collection of key-value pairs, where the keys are strings and the values ​​are combinations of any data type, including arrays and documents.

[0089] Column-oriented databases (such as HBase) are a type of distributed database. Technically, column-oriented databases resemble distributed storage more than a distributed database; they lack features of relational database management systems, such as column types, secondary indexes, triggers, and high-level query languages. Column-oriented databases are column-family databases, capable of handling high-concurrency scenarios.

[0090] S102. Based on multiple collection times, pair and organize the collected data of multiple indicators to obtain the data pairs to be analyzed.

[0091] The specific values ​​of database monitoring metrics are highly likely to exhibit periodicity in time series. For example, the specific value of real-time access volume may fluctuate within a day. Therefore, when mining trend indicators, it is necessary to consider the periodicity of the monitoring indicator values ​​and analyze the changes in the specific values ​​of the monitoring indicators after several time periods. To address this, after obtaining multiple indicator collection data, the information mining equipment first pairs different indicator collection data according to multiple collection times. This ensures that the collection times corresponding to the indicator collection data in the paired data are exactly a certain number of time periods apart. The resulting data pairs are the data pairs to be analyzed subsequently, denoted as the data pairs to be analyzed. In other words, in this embodiment of the application, the collection times corresponding to the indicator collection data in the data pairs to be analyzed differ by one or more time periods.

[0092] In some embodiments, the information mining device can first group multiple indicator data according to the relationship between time periods and multiple collection times, so that indicator data collected within the same time period are grouped into the same group, and then pair indicator data between different groups, so that indicator data collected with collection times differing by one or more time periods form a data pair, thereby obtaining the data pair to be analyzed.

[0093] In other embodiments, the information mining device can also calculate the collection time of each indicator data collection time and the time difference between the collection time of other indicator data collection times, and pair the indicator data collection time that is a positive integer multiple of the time period with the indicator data collection time to obtain a set of data pairs, thereby obtaining the data pairs to be analyzed.

[0094] It is understandable that the time period can be days, hours, months, quarters, etc. Of course, the time period can also be the length of time set according to the actual situation, such as 10 days, 20 days, etc., which is not limited here.

[0095] It should be noted that the data pairs to be analyzed obtained by the information mining equipment refer to the collective term for the obtained data pairs. That is, in the embodiments of this application, the data pairs to be analyzed may include one or more data pairs.

[0096] Furthermore, when the monitoring indicators include multiple different types of indicators, the information mining equipment can obtain the data pairs to be analyzed for each type of indicator, that is, obtain one or more data pairs.

[0097] S103. Based on the internal differences of the data pairs to be analyzed, trend indicators that show a trend in data changes are extracted from the monitoring indicators, and used as information mining results.

[0098] After obtaining the data pair to be analyzed, the information mining equipment analyzes the changes in the collected data over different time periods, that is, it analyzes the internal differences of the data pair to clarify the data changes of each indicator in the monitoring indicators after one or more time periods. This clarifies whether there is an overall trend in the data changes of the monitoring indicators, and identifies indicators from one or more of the included indicators that exhibit an overall trend in data changes; these are the trend indicators. In this way, the information mining equipment can automatically complete the trend indicator mining process without human intervention. The mining process is highly efficient, and because it fully considers the periodic changes of the monitoring indicator data, the obtained trend indicators have high accuracy.

[0099] Understandably, when the data pair to be analyzed includes only one data pair, the information mining device can calculate the difference between the indicator data collected in that data pair. That is, by using the difference between the indicator data collected in that data pair, it can determine whether the data of the monitoring indicator increases, decreases or remains unchanged after a certain number of time periods, thereby clarifying whether there is a trend in the data change of the monitoring indicator and obtaining trend indicators.

[0100] When the indicators to be analyzed include multiple data pairs, the information mining equipment can calculate the difference for each data pair, then determine the difference between each data pair based on the difference, and then perform statistical analysis on the differences of multiple data pairs to determine whether there is a trend in the data changes of one or more indicators among the monitoring indicators. Finally, the indicators among the monitoring indicators whose data changes show a trend are identified as trend indicators to complete the information mining process.

[0101] In this embodiment, the information mining device first collects data on the monitoring indicators of the database at multiple collection times, obtaining multiple indicator data. Then, based on the multiple collection times, the multiple indicator data are paired and organized, that is, indicator data collected at times differing by one or more time periods are paired to obtain data pairs to be analyzed. Finally, based on the internal differences of the indicator pairs to be analyzed, the changes in the monitoring indicator data across different time periods are identified. Finally, indicators showing trends in data change are extracted from the monitoring indicators, yielding trend indicators. In this way, the information mining device can not only automatically discover trend indicators, improving the efficiency of trend indicator mining, but also fully consider the periodicity of the monitoring indicator data during the trend indicator mining process, improving the accuracy of trend indicator mining. In summary, this embodiment ultimately improves the intelligence level of trend indicator mining.

[0102] See Figure 4 , Figure 4 This is a flowchart illustrating the information mining method provided in the embodiments of this application. Figure 2In some embodiments of this application, the data collected from multiple indicators are paired and organized according to multiple collection times to obtain the data pairs to be analyzed. The specific implementation process of S102 may include: S1021-S1023, as follows:

[0103] S1021. Arrange the collected data of multiple indicators according to the order of multiple collection times to obtain a data sequence.

[0104] When data mining equipment pairs and organizes data from multiple indicators, it first uses the chronological order of each collection time as the corresponding order of the data in the sequence to arrange the data from multiple indicators. The resulting sequence is the data sequence. In other words, the information mining equipment arranges the data from multiple indicators in time sequence to obtain the data sequence.

[0105] S1022. Divide the data sequence according to the sequence length corresponding to the time period to obtain multiple subsequences.

[0106] Time periods typically have a corresponding period length, which is determined by the number of times data is collected from the monitored indicators within a given time period. For example, if the time period is days, and assuming m data collections are needed for the database monitoring indicators per day, then the sequence length is m. Information mining equipment divides the data sequence into multiple subsequences at intervals corresponding to the sequence length of each time period.

[0107] It should be noted that in some cases, after the last segmentation of the data sequence, the number of remaining indicator data is insufficient to reach the cycle length. In this case, the information mining device will still form a subsequence with the remaining indicator data. Thus, in multiple subsequences, the sequence length (i.e. the number of indicator data) of the subsequences other than the last subsequence is the cycle length.

[0108] For example, Figure 5 This is a schematic diagram illustrating the process of grouping a data sequence according to an embodiment of this application. The index data collected in data sequence 5-1 are data 5-11 to data 5-15, and the corresponding collection times are 1:00 AM on August 16, 1:00 PM on August 16, 1:00 PM on August 17, 1:00 PM on August 17, and 1:00 AM on August 18. Assuming the time period is one day, the sequence length corresponding to the time period is 2 (i.e., data is collected once at 1:00 AM and once at 1:00 PM, and no data is collected on August 18 before 1:00 PM). At this time, the information mining device will divide data sequence 5-1 according to the period length of 2, resulting in 3 subsequences, namely subsequence 5-21, subsequence 5-22, and subsequence 5-23.

[0109] S1023. Based on the pairing of index collection data of adjacent subsequences in multiple subsequences, the data pairs to be analyzed are obtained.

[0110] Information mining equipment extracts adjacent subsequences from multiple subsequences, then pairs the corresponding indicator data from these adjacent subsequences, resulting in a number of data pairs equal to the length of the subsequences. These data pairs constitute the data pairs corresponding to an adjacent subsequence. When indicator data pairing has been performed on all adjacent subsequences, all data pairs corresponding to adjacent subsequences are identified as the data pairs to be analyzed.

[0111] It is understood that the adjacent subsequences here can refer to each subsequence and its previous subsequence, or to each subsequence and its next subsequence; this application does not limit this.

[0112] In this embodiment of the application, the information mining device first sorts the data collected from multiple indicators to obtain a data sequence, and then divides the data sequence into multiple subsequences according to the sequence length corresponding to the time period. Then, it pairs the data collected from the indicators for the subsequences of the vectors in the multiple subsequences to obtain the data pairs to be analyzed.

[0113] In some embodiments of this application, the multiple subsequences include N subsequences, where N is a positive integer greater than 1. In this case, based on the pairing of index collection data of adjacent subsequences in the multiple subsequences, the data pairs to be analyzed are obtained. The specific implementation process of S1023 may include: S1023a-S1023c, as follows:

[0114] S1023a, Determine the next subsequence of each of the first N-1 subsequences in the N subsequences as the adjacent subsequences.

[0115] In this embodiment of the application, the information mining device takes the next subsequence after each subsequence as the adjacent subsequence. Thus, for N subsequences, only the first N-1 subsequences have corresponding adjacent subsequences, while for the Nth subsequence, no corresponding adjacent subsequence can be determined.

[0116] S1023b: Take the index data collected from the first N-1 subsequences of the N subsequences and pair them with the index data collected from the corresponding adjacent subsequences to obtain N-1 data pairs.

[0117] The information mining equipment pairs the index data collected in each of the first N-1 subsequences with the index data collected in the adjacent subsequences, thus obtaining a data pair corresponding to each subsequence. When all data pairs have been generated for the first N-1 subsequences, N-1 data pairs will be obtained.

[0118] For example, when each subsequence's data pairs are represented as At that time, information mining equipment will obtain , … There are a total of N-1 data pairs.

[0119] S1023c, Identify N-1 data pairs as the data pairs to be analyzed.

[0120] Information mining equipment combines the obtained N-1 data pairs to obtain the data pair to be analyzed. For example, when each subsequence data pair is represented as... At that time, the data pairs to be analyzed obtained by the information mining equipment can be represented as { , … }

[0121] In this embodiment, the information mining device first takes the next subsequence of each subsequence as the adjacent subsequence of each subsequence, thereby determining the corresponding adjacent subsequences for the first N-1 subsequences. Then, it pairs the indicator data collected in the first N-1 subsequences with the indicator data collected in the corresponding adjacent subsequences, resulting in a total of N-1 data pairs. The N-1 data pairs are then used to integrate the data pairs to be analyzed, so as to facilitate the subsequent analysis of the data pairs to be analyzed.

[0122] In some embodiments of this application, the indicator data collected in the first N-1 subsequences are paired with the indicator data collected in the corresponding adjacent subsequences to obtain N-1 data pairs. The specific implementation process of S1023b may include: S201-S204, as follows:

[0123] S201. For each indicator in the i-th subsequence of the first N-1 subsequences, collect data and filter out matching data from the corresponding adjacent subsequences whose collection time differs from that of each indicator by one time period.

[0124] In this embodiment, the adjacent subsequence corresponding to the i-th subsequence is actually the (i+1)-th subsequence. The information mining device collects data for each indicator in the i-th subsequence, and selects indicator data from the (i+1)-th subsequence whose collection time differs from its collection time by a full time period. This selected indicator data is then identified as matching data. Here, i is greater than or equal to 1 and less than or equal to N-1, meaning the maximum value of i is N-1.

[0125] For example, if the data collection time for a certain indicator in the i-th subsequence is 13:00 on August 16, the information mining device will search for the indicator data that is exactly one day different from it in the (i+1)-th subsequence, that is, the indicator data collected at 13:00 on August 17, as the matching data.

[0126] Understandably, a time cycle refers to the intervals in which time repeats in the same order. For example, a year is a time cycle when days repeat in the same order, and a day is a time cycle when hours repeat in the same order.

[0127] S202. Pair the collected data and matching data for each indicator to generate a sub-data pair corresponding to the collected data for each indicator.

[0128] S203. Using the sub-data pairs corresponding to the data collected for each indicator, integrate the data pairs corresponding to the i-th subsequence.

[0129] The information mining device combines the collected data and matching data for each indicator into a sub-data pair, thus obtaining the sub-data pair corresponding to each indicator's collected data. Next, the information mining device aggregates the sub-data pairs corresponding to each indicator's collected data to obtain the data pair corresponding to the i-th subsequence.

[0130] For example, when the i-th subsequence has m index data points, the sub-data pair corresponding to each index data point can be represented as ( , ),( , )…( , Therefore, the data pair corresponding to the i-th subsequence It consists of these sub-data pairs.

[0131] S204. When corresponding data pairs are generated for the first N-1 subsequences, N-1 data pairs are obtained.

[0132] In this embodiment of the application, the information mining device collects data for numerous indicators in each subsequence. It selects matching data from the corresponding adjacent subsequences, forms sub-data pairs with the corresponding matching data, and finally integrates the sub-data pairs to obtain the data pairs to be analyzed later.

[0133] See Figure 6 , Figure 6 This is a flowchart illustrating the information mining method provided in the embodiments of this application. Figure 3In some embodiments of this application, the data pairs to be analyzed include N-1 data pairs. In this case, based on the internal differences of the data pairs to be analyzed, trend indicators that show a trend in data changes are extracted from the monitoring indicators. The specific implementation process of S103 may include: S1031-S1032, as follows:

[0134] S1031. Based on the set of differences between each of the N-1 data pairs, determine the trend characteristics corresponding to the monitoring indicators.

[0135] Each data pair contains multiple sub-data pairs, and the differences between the collected indicator data within each sub-data pair can be calculated to obtain a set of differences for each data pair. Based on this set of differences, the information mining equipment analyzes whether the collected indicator data exhibits an upward trend, a downward trend, or remains unchanged after one or more time periods. The analysis results are then used as trend characteristics. In other words, trend characteristics indicate that the collected indicator data exhibits any one of the following trends: upward trend, downward trend, or no change.

[0136] It should be noted that when the monitoring indicators include multiple different indicators, the information mining equipment constructs N-1 data pairs for each indicator. Thus, the trend feature determined based on the set of differences between each of the N-1 data pairs is the trend feature corresponding to each indicator in the monitoring indicators. In other words, the trend feature contains features that correspond one-to-one with multiple indicators in the monitoring indicators.

[0137] S1032. From the monitoring indicators, select trend indicators that show an upward or downward trend in the collected data.

[0138] Information mining equipment analyzes the trend characteristics of monitoring indicators to clarify the changing trend of the data collected for the corresponding monitoring indicators after a period of time. Indicators whose trend characteristics indicate that the collected data shows an upward or downward trend are identified as trend indicators.

[0139] Understandably, when a monitoring indicator contains only one indicator, the information mining device determines whether the indicator's collected data has a trend based on the indicator's trend characteristics, in order to determine whether to mine the indicator as a trend indicator. When a monitoring indicator contains multiple indicators, the information mining device mines the indicators whose trend characteristics indicate that the collected data has a trend as trend indicators.

[0140] In this embodiment, the information mining device uses the difference set of each data pair to determine the trend characteristics of the monitoring indicators, thereby clarifying whether the changes in the indicator data collected for the corresponding monitoring indicators show a trend. Then, indicators showing an upward or downward trend are mined as trend indicators. In this way, the information mining device completes the process of mining trend indicators.

[0141] In some embodiments of this application, the trend characteristics corresponding to the monitoring indicator are determined based on the set of differences between each of the N-1 data pairs. The specific implementation process of S1031 may include: S1031a-S1031c, as follows:

[0142] S1031a. For each of the N-1 data pairs, construct the corresponding difference set.

[0143] In some embodiments, the information mining device can subtract metrics from the sub-data pairs within each data pair. For example, when a sub-data pair contains CPU utilization rates at the same time point on two adjacent days, the information mining device subtracts the CPU utilization rate of the previous day from the CPU utilization rate of the next day, or vice versa, to obtain the difference for that sub-data pair. In this way, the information mining device obtains the differences for each sub-data pair, and then integrates these differences to obtain a set of differences for each data pair.

[0144] S1031b: For the differences in the difference set, count the number of positive and negative numbers to obtain the number of positive and negative numbers for each data pair.

[0145] The information mining equipment categorizes the differences in the set of differences for each data pair into positive and negative numbers. Then, it counts the number of positive differences to obtain the number of positive differences for each data pair, and counts the number of negative differences to obtain the number of negative differences for each data pair.

[0146] For example, when the set of differences for a data pair is {-2, 0.5, 3}, the information mining device determines the number of positive numbers for that data pair to be 2, and the number of negative numbers to be 1.

[0147] S1031c. Based on the number of positive and negative numbers, determine the trend characteristics corresponding to the monitoring indicators.

[0148] After obtaining the number of positive and negative numbers for each data pair, the information mining equipment can compare the number of positive and negative numbers for each data pair to determine whether the overall change in the indicator data collected in each data pair tends to rise, fall, or remain unchanged. This allows the determination of the trend type of each data pair. Based on the trend type of each data pair, it can then be determined whether the indicator data collected for the monitoring indicator tends to rise, fall, or remain unchanged after one or more time periods, thus obtaining the trend characteristics corresponding to the monitoring indicator.

[0149] In this embodiment of the application, the information mining device can first construct a difference set for each of the N-1 data pairs, then count the number of positive and negative numbers for each data pair based on the difference set, and finally determine the trend characteristics based on the number of positive and negative numbers for each data pair, so as to mine trend indicators based on the trend characteristics in the future.

[0150] In some embodiments of this application, the trend characteristics corresponding to the monitoring indicators are determined based on the number of positive and negative numbers, i.e., the specific implementation process of S1031c, which may include: S301-S302, as follows:

[0151] S301. Based on the relationship between the number of positive and negative numbers, determine the type of trend for each data pair to obtain the trend type.

[0152] Information mining equipment compares the number of positive numbers with the number of negative numbers to analyze whether the number of positive numbers is much greater than the number of negative numbers, much less than the number of negative numbers, or nearly the same as the number of negative numbers. The analysis result is the type of change trend for each data pair, i.e., the trend type.

[0153] S302. From the trend types of each data pair, select the trend type that appears most frequently as the trend feature corresponding to the monitoring indicator.

[0154] Information mining equipment identifies the most prominent trend type among the trend types of each data pair, which is the trend type that appears most frequently in each data pair, and determines the trend type as the trend feature of the monitoring indicator.

[0155] For example, Figure 7 This is a schematic diagram illustrating the process of determining the trend characteristics of monitoring indicators provided in an embodiment of this application. See also... Figure 7There are 5 data pairs, from data pair 7-1 to data pair 7-5. When the number of negative values ​​in a data pair is greater than the number of positive values ​​(7-61), it indicates that the data collected by the indicator has tended to decrease after one or more time periods, suggesting a downward trend for that data pair. Conversely, if the number of negative values ​​is less than the number of positive values ​​(7-62), it indicates that the data collected by the indicator has tended to increase after one or more time periods, suggesting an upward trend for that data pair. When the number of negative numbers is 7-63 (the difference between the number of positive and negative numbers is less than a preset difference), it indicates that the difference between the two indicator data collected in each sub-data pair of the data pair conforms to a binomial distribution. At this time, the indicator data collected tends to remain unchanged after one or more time periods. In this way, the information mining device can determine the trend type 7-7 for data pairs 7-1 to 7-5, namely, decreasing 7-71, unchanged 7-72, increasing 7-73, increasing 7-73, and increasing 7-73. Then, the mode of these trend types, namely increasing 7-73, is determined as the trend feature 7-8.

[0156] In this embodiment, the information mining device first determines the trend type of each data pair based on the number of positive and negative data points, and then filters out the mode among all trend types as the final trend feature. This allows for a comprehensive assessment of the changes in the collected data corresponding to the monitoring indicators, enabling the identification of changes in monitoring indicators that are not easily perceived by humans, thereby improving the accuracy of trend indicator mining.

[0157] In some embodiments of this application, the trend type of each data pair is determined based on the relationship between the number of positive and negative numbers, thus obtaining the trend type. The specific implementation process of S301 may include any one of steps S3011 to S3013, as follows:

[0158] S3011. When the number of positive numbers is greater than the sum of the number of negative numbers and the obtained gap threshold, determine the trend type of each data pair as upward.

[0159] S3012. When the sum of the number of positive numbers and the gap threshold is less than the number of negative numbers, determine the trend type of each data pair as declining.

[0160] S3013. When the difference between the number of positive numbers and the number of negative numbers is less than the difference threshold, determine the trend type of each data pair as no trend.

[0161] In this embodiment, whether the number of positive numbers is significantly greater than, significantly less than, or approximately the same as the number of negative numbers needs to be determined based on a gap threshold. When the number of positive numbers is greater than the sum of the number of negative numbers and the gap threshold, it indicates that the number of positive numbers is significantly greater than the number of negative numbers, and the information mining device will determine the trend type of each data pair as upward. When the sum of the number of positive numbers and the gap threshold is less than the number of negative numbers, it indicates that the number of positive numbers is significantly less than the number of negative numbers, and the information mining device will determine the trend type of each data pair as downward. When the difference between the number of positive numbers and the number of negative numbers is less than the gap threshold, it indicates that the number of positive numbers and the number of negative numbers are approximately the same, and the information mining device will determine the trend type of each data pair as no trend.

[0162] It is understood that the gap threshold can be manually set according to the actual situation, such as setting it to 50, setting it to 200, etc., and this application does not limit this. The gap threshold can also be automatically generated by the information mining equipment in combination with factors such as the information mining time and the threshold used in the previous round of information mining. For example, when the information mining time is noon, the gap threshold is automatically set to 200, or 1.5 times the threshold used in the previous round of information mining is used as the gap threshold, etc., and this application does not limit this.

[0163] In this embodiment of the application, the information mining device can determine the trend type of each data pair based on the relationship between the number of positive numbers, the number of negative numbers, and the preset gap, so as to determine the trend characteristics of the monitoring indicator based on the trend type of each data pair.

[0164] In some embodiments of this application, after generating corresponding data pairs for the first N-1 subsequences, i.e. after S204, the method may further include: S205-S207, as follows:

[0165] S205. Calculate the rate of change of the data pair corresponding to the i-th subsequence by using the difference between the collected data and the matched data for each indicator.

[0166] S206. When the rate of change is generated for each of the data pairs of the first N-1 subsequences, N-1 rates of change are obtained.

[0167] The information mining equipment first subtracts the collected data and matching data for each indicator to obtain the data difference value corresponding to each indicator's collected data. Then, by dividing the data difference by the collected data for each indicator, the sub-rate of change corresponding to each indicator's collected data can be obtained. Next, the information mining equipment can obtain the rate of change of the data pair corresponding to the i-th subsequence by averaging the sub-rates of change corresponding to the collected data for each indicator, or by taking the mode of the sub-rates of change corresponding to each indicator's collected data. The information mining equipment performs this process for the first N-1 subsequences, thereby obtaining the rate of change corresponding to each data pair of the first N-1 subsequences, i.e., obtaining N-1 rates of change.

[0168] It should be noted that when the information mining equipment calculates the difference between the collected data and the matching data for each indicator, it first selects the larger of the two data sets: the collected data for each indicator and its corresponding matching data. Then, it subtracts the smaller data from the larger data set to ensure that the resulting data difference is non-negative. This ensures that the rate of change of the collected data for each indicator is non-negative, which is helpful for subsequent calculations of the average or mode.

[0169] Furthermore, when the matched data is greater than the data collected for each indicator, the calculated sub-rate of change for each indicator is the growth rate; when the matched data is less than the data collected for each indicator, the calculated sub-rate of change for each indicator is the decline rate.

[0170] S207. Based on the maximum and minimum rates of change among N-1 rates of change, estimate the abnormal time period.

[0171] Information mining equipment compares N-1 rates of change to determine the maximum and minimum rates of change. The maximum rate of change is used to calculate the shortest time for the collected data of the database's monitoring indicators to reach a critical value, while the minimum rate of change is used to calculate the longest time for the collected data to reach the critical value. Therefore, based on the maximum and minimum rates of change, the information mining equipment can determine a time period during which the collected data reaches the critical value. Generally, when the collected data reaches the critical value, the database must be expanded or optimized; otherwise, the database is highly likely to crash or malfunction. Therefore, the time period during which the collected data reaches the critical value is the time period during which the database may experience anomalies, and this time period is denoted as the abnormal time period. Thus, the abnormal time period represents the estimated time period during which the database may experience anomalies.

[0172] It should be noted that the information mining equipment can take the logarithm of the critical value, then compare the logarithmic result with the maximum rate of change to obtain the shortest time, and compare the logarithmic result with the minimum rate of change to obtain the longest time.

[0173] Understandably, when taking the logarithm of a critical value, the information mining device can use the largest indicator data in the last subsequence as the base to take the logarithm of the critical value, obtaining the first logarithmic result. Then, it compares the first logarithmic result with the maximum rate of change to obtain the shortest time. Similarly, the information mining device can also use the smallest indicator data in the last subsequence as the base to take the logarithm of the critical value, obtaining the second logarithmic result. Then, it compares the second logarithmic result with the minimum rate of change to obtain the longest time. Of course, the information mining device can also use a natural number as the base to take the logarithm of the critical value; this application does not limit this.

[0174] For example, embodiments of this application provide formulas for calculating the shortest time and the longest time, as shown in equations (1) and (2) respectively:

[0175] = (1)

[0176] = (2)

[0177] in, For the shortest time, For the longest time, Indicates the maximum rate of change. Indicates the minimum rate of change. ={ , … } (N-1 growth rates), where L in equation (1) is the largest indicator data collected in the last subsequence, and L in equation (2) is the smallest indicator data collected in the last subsequence.

[0178] When the information mining equipment calculates the shortest time is The longest time is If so, then the abnormal time period is [ , ].

[0179] Furthermore, the threshold value can be the maximum storage capacity of the database, the maximum number of accesses, or other values ​​related to the business scenario of the database. This application does not limit these values.

[0180] In this embodiment of the application, the information mining device can also calculate the rate of change of the data pair corresponding to the i-th subsequence after obtaining the data pair corresponding to the i-th subsequence, thereby obtaining N-1 rates of change. Then, based on the maximum and minimum rates of change among the N-1 rates of change, it can predict the time period during which the database may experience anomalies, so as to facilitate subsequent alarm or automatic expansion processing and improve the security of the database.

[0181] Of course, in some embodiments, the information mining device can also use the mean change rate of each data pair, such as the mean growth rate or the mean decrease rate, to estimate the abnormal time period. It can also use a fixed change rate, such as a fixed growth rate or a fixed decrease rate, to estimate the abnormal time period. These can be selected according to the actual business scenario of the database.

[0182] In some embodiments of this application, after estimating the abnormal time period based on the maximum and minimum rate of change among N-1 rates of change, i.e. after S207, the method may further include: S208 or S209, as follows:

[0183] S208. Based on the abnormal time period, generate alarm information to prompt for database expansion.

[0184] After receiving an abnormal time period, the information alarm device can generate alarm information for that period and send it to the database administrator so that the administrator can manually expand the database based on the abnormal time period.

[0185] Understandably, information alarm devices can generate alarm information by packaging abnormal time periods into emails and SMS messages.

[0186] S209. When the difference between the current time point and the starting time point of the abnormal time period is less than a preset time threshold, the database is expanded to obtain the expanded database.

[0187] In addition to alerting database administrators based on abnormal time periods, the information mining equipment can also calculate the difference between the current time period and the starting time of the abnormal time period. When the calculated difference is less than a preset time threshold, it means that the current time is close enough to the abnormal time period. At this time, the information mining equipment will automatically expand the database to obtain the expanded database.

[0188] In this embodiment, the information mining device can generate alarm information based on abnormal time periods, prompting database administrators to manually expand the database, or automatically expand the database when the current time is about to reach the abnormal time period. This ensures that the database is expanded before reaching the abnormal time period, thereby guaranteeing database security.

[0189] In some embodiments of this application, the trend type that appears most frequently in each data pair is selected as the trend feature corresponding to the monitoring indicator. That is, after S302, the method may further include: S303-S304, as follows:

[0190] S303. Assign corresponding feature values ​​to each data pair based on its trend type.

[0191] After obtaining the trend type of each data pair, the information mining equipment assigns corresponding feature values ​​to the trend type of each data pair, thereby quantifying the trend type to facilitate subsequent confidence calculation.

[0192] It is understandable that the trend type includes three cases: rising, falling, and no trend. Therefore, when the information mining device assigns feature values, it assigns corresponding values ​​to the distribution of these three cases, such as 1 for rising, 0 for no trend, and -1 for falling.

[0193] S304. Calculate the confidence level of the trend feature based on the feature value of each data pair and the total number of data pairs.

[0194] Information mining equipment accumulates the feature values ​​of each data pair, and then divides the accumulated result by the total number of data pairs. The resulting ratio is the confidence level of the trend feature.

[0195] For example, equation (3) shows the formula for calculating the confidence level of a trend feature:

[0196] C= (3)

[0197] in, is the feature value of each data pair, with a value range of {1, 0, -1}, N-1 is the total number of data pairs, and C is the confidence level.

[0198] Furthermore, after the information mining equipment calculates the confidence level of the trend characteristics, trend indicators that show an upward or downward trend in the collected data are selected from the monitoring indicators. The specific implementation process of S1032 can include: S1032a, as follows:

[0199] S1032a. From the monitoring indicators, select trend characteristics that show an upward or downward trend in the collected data and whose confidence level exceeds the confidence level threshold.

[0200] In other words, in this embodiment of the application, the information mining device not only needs to consider the changing trend of the indicator data represented by the trend feature, but also needs to consider whether the trend feature is reliable in order to screen out the trend indicator.

[0201] In this embodiment of the application, the information mining device can first represent the trend type of each data pair with a feature value, and then calculate the confidence level of the trend feature based on the feature value of each data pair and the total number of data pairs, so as to consider the credibility of the trend feature when screening trend indicators based on the trend feature, thereby further improving the accuracy of trend indicator mining.

[0202] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0203] The embodiments of this application are implemented in a scenario where a server (information mining device) automatically mines trend indicators from databases in different application scenarios.

[0204] See Figure 8 , Figure 8 This is a system architecture diagram for mining trend indicators in a database, provided in an embodiment of this application. The entire architecture 8-1 is divided into four layers: data source 8-11, acquisition layer 8-12, calculation layer 8-13, and interaction layer 8-14.

[0205] Among them, data sources 8-11 are generally various commercial databases, such as relational database MySQL 8-111, key-value database Redis 8-112, document database MongoDB 8-113, and column-oriented storage database HBase 8-114.

[0206] The acquisition layer 8-12 mainly collects various monitoring metrics from data source 8-11, obtaining data for different metrics, such as data 8-121, data 8-122, data 8-123, etc. (collecting the monitoring metrics data). It collects the time-series information of various monitoring metrics for each database, that is, it collects various monitoring metrics of the database at regular intervals (multiple acquisition times) (obtaining multiple metric acquisition data). All the collected data are arranged in chronological order (based on the order of multiple acquisition times, the multiple metric acquisition data are arranged to obtain a data sequence) and stored in a time-series database, such as Elasticsearch, for later use.

[0207] The computational layer 8-13 performs trend testing 8-131 and critical time estimation 8-132 based on the improved algorithm TXTming proposed in this application on the Cox-Stuart test algorithm. When the computational layer 8-13 detects a trend in the indicator, it estimates a critical time range (abnormal time period) based on the threshold (critical value) and calculates the confidence score. Then, it feeds back the trend and the time range confidence score to the interaction layer 8-14.

[0208] Interaction layer 8-14 sends alerts to administrators through various means such as SMS 8-141 and email 8-142 (alarm information).

[0209] Understandably, based on the above system architecture, a single database may have multiple different metrics (monitoring metrics include multiple indicators). For each metric, the server uses the TXTiming algorithm to determine trends. If a metric meets the characteristics of a trend (a trend indicator selected from monitoring metrics that shows an upward or downward trend in the collected data), the server will make proximity judgments based on a threshold, estimating the fastest time to reach the threshold (shortest time) and the slowest time to reach the threshold (longest time), for administrators to make decisions. Finally, multiple alarm methods are implemented so that administrators can proactively expand capacity based on the two given times and their experience; of course, the server can also automatically expand capacity to ensure sufficient online resources.

[0210] Figure 9 This is a schematic diagram of the process for mining trend indicators provided in an embodiment of this application. See also... Figure 9 The process may include:

[0211] S401, Begin.

[0212] S402, Indicator Collection.

[0213] The metrics collected can include CPU utilization, number of transactions processed, slow logs, etc.

[0214] S403, Trend Calculation (Determining the trend characteristics corresponding to the monitoring indicators).

[0215] S404. Determine if a trend is met. If it is, continue with S405.

[0216] S405, Calculation of near-time threshold (estimation of abnormal time period).

[0217] S406. Determine whether the time threshold is about to be reached. If the threshold is met (when the difference between the current time point and the starting time point of the abnormal time period is less than the preset time threshold), continue executing S407.

[0218] S407, Multiple alarm methods (generate alarm information).

[0219] S408, Automated expansion (expanding the database to obtain the expanded database).

[0220] The TXTiming algorithm provided in the embodiments of this application will be described below.

[0221] First, let's briefly introduce the Cox-Stuart test algorithm. The principle of the Cox-Stuart test algorithm is that for a data sequence { , , … }, using the data at the middle position C as the boundary, divide the data sequence into two parts, and pair them up into ( , ),( , )…( , Then, we can determine the trend of the data by looking at the form of ().

[0222] However, the Cox-Stuart test algorithm simply divides the data sequence into two parts. For database monitoring metrics, many metrics exhibit significant periodicity. Simply dividing all the data into two equal parts is clearly inappropriate; the impact of periodicity must be considered, along with the confidence level.

[0223] The TXTiming algorithm works roughly as follows: For time-series data (data sequences), the sequence is divided into n parts according to the period n, with each data part (subsequence) having a length of m (the sequence length corresponding to the time period). If the length of the last remaining part is less than m, then this part is taken as the remainder, and the remainder is taken as the last data part. Next, the i-th data part (the i-th subsequence) and the (i+1)-th data part (the next subsequence of each of the first N-1 subsequences is determined as adjacent subsequences) are paired up to obtain ( , ),( , )…( , (A sub-data pair of data collected for each indicator), each such data set is denoted as (The data pair corresponding to the i-th subsequence), thus obtaining { , … For each set of data Subtract the previous number from the next number in its sub-items and record the sign. This represents the number of positive numbers obtained (number of positive numbers). This indicates the number of negative numbers obtained (count of negative numbers). If Far greater than (The number of positive numbers is greater than the sum of the number of negative numbers and the preset difference), it is considered that Store as an upward trend type (each data pair has an upward trend type), otherwise... If it is a descent type, and If the difference between the number of positive and negative numbers is very close (less than the preset gap), then there is no significant change (no trend). Thus, we obtain... The corresponding type (trend type). For { , … (For the data pairs to be analyzed), the mode is used as the final trend, that is, the trend with the most occurrences is the final type of the indicator (from the trend types of each data pair, the trend type that appears most frequently is selected as the trend feature corresponding to the monitoring indicator).

[0224] At the same time, targeting The data growth / decline rate (rate of change) was also calculated, and for The type corresponds to a numerical value (assigning a corresponding feature value to the trend type of each data pair), for example { , },in, express It is an ascending type. express It has no trend. express For descent type, thus targeting { , … }, collected { , … } numerical values ​​of trend types, and { , … There are} growth / decline rates (N-1 rates of change). Then from { , … The maximum and minimum growth rates are selected from the data to calculate the fastest and slowest possible times to reach the threshold. This can be done using equations (1) and (2) to obtain the time range. , (Abnormal time period) is provided to managers for reference. At the same time, the confidence level can be calculated by formula (3).

[0225] In summary, the TXTiming algorithm takes (time series data D, period length N) as input and obtains (type T, confidence level C, time range) as output. , If the confidence level is greater than 0.6 (confidence threshold) and it exhibits a trend (the trend characteristic indicates that the collected data shows an upward or downward trend), then the indicator is considered to have a trend characteristic, and is thus a trend indicator.

[0226] The above methods can accurately and automatically identify trend indicators in monitoring metrics, improve the intelligence of trend indicator mining, and provide the possible time when thresholds are reached for management personnel's reference, or automatically expand the database to improve database security.

[0227] The following description continues to illustrate the exemplary structure of the information mining device 555 provided in the embodiments of this application as a software module. In some embodiments, such as... Figure 2 As shown, the software modules stored in the information mining device 555 in the memory 550 may include:

[0228] The data acquisition module 5551 is used to collect data from the monitoring indicators of the database multiple times, obtain multiple indicator data, and record multiple data collection times.

[0229] The data processing module 5552 is used to pair and process the collected data of the multiple indicators according to the multiple collection times to obtain a data pair to be analyzed; wherein the collection times corresponding to the collected data of the indicators in the data pair to be analyzed differ by one or more time periods.

[0230] The indicator mining module 5553 is used to mine trend indicators that show a trend in data changes from the monitoring indicators based on the internal differences of the data pair to be analyzed, as the information mining result.

[0231] In some embodiments of this application, the data processing module 5552 is further configured to arrange the multiple indicator collection data according to the chronological order of the multiple collection times to obtain a data sequence; divide the data sequence according to the sequence length corresponding to the time period to obtain multiple sub-sequences; and obtain the data pair to be analyzed by pairing the indicator collection data of adjacent sub-sequences in the multiple sub-sequences.

[0232] In some embodiments of this application, the plurality of subsequences includes: N subsequences, where N is a positive integer greater than 1; the data processing module 5552 is further configured to determine the next subsequence of each of the first N-1 subsequences in the N subsequences as adjacent subsequences; pair the index collection data in the first N-1 subsequences with the index collection data in the corresponding adjacent subsequences to obtain N-1 data pairs; and determine the N-1 data pairs as the data pairs to be analyzed.

[0233] In some embodiments of this application, the data processing module 5552 is further configured to collect data for each indicator in the i-th subsequence of the first N-1 subsequences, and filter out matching data from the corresponding adjacent subsequences whose collection time differs from that of each indicator by one time period; wherein i is greater than or equal to 1 and less than or equal to N-1; pair each indicator collection data with the matching data to generate a sub-data pair corresponding to each indicator collection data; integrate the data pair corresponding to the i-th subsequence using the sub-data pairs corresponding to each indicator collection data; and obtain the N-1 data pairs when corresponding data pairs are generated for all of the first N-1 subsequences.

[0234] In some embodiments of this application, the data pairs to be analyzed include: N-1 data pairs; the indicator mining module 5553 is further configured to determine the trend characteristics corresponding to the monitoring indicator based on the difference set of each data pair in the N-1 data pairs; the trend characteristics represent that the collected data of the indicator shows any one of an upward trend, a downward trend, and a no-change trend; from the monitoring indicators, the trend indicators that represent that the collected data of the trend characteristics show the upward trend or the downward trend are selected.

[0235] In some embodiments of this application, the indicator mining module 5553 is further configured to construct a corresponding difference set for each of the N-1 data pairs; to count the number of positive and negative differences in the difference set to obtain the number of positive and negative differences for each data pair; and to determine the trend feature corresponding to the monitoring indicator based on the number of positive and negative differences.

[0236] In some embodiments of this application, the indicator mining module 5553 is further configured to determine the type of change trend of each data pair based on the relationship between the number of positive numbers and the number of negative numbers, and obtain the trend type; from the trend types of each data pair, the trend type that appears most frequently is selected as the trend feature corresponding to the monitoring indicator.

[0237] In some embodiments of this application, the indicator mining module 5553 is further configured to determine the trend type of each data pair as rising when the number of positive numbers is greater than the sum of the number of negative numbers and the obtained gap threshold; determine the trend type of each data pair as falling when the sum of the number of positive numbers and the gap threshold is less than the number of negative numbers; and determine the trend type of each data pair as no trend when the difference between the number of positive numbers and the number of negative numbers is less than the gap threshold.

[0238] In some embodiments of this application, the information mining device 555 further includes: a time estimation module 5554; the time estimation module 5554 is used to calculate the rate of change of the data pair corresponding to the i-th subsequence by using the difference between the data collected by each indicator and the matching data after obtaining the N-1 data pairs when corresponding data pairs are generated for the first N-1 subsequences; when a rate of change is generated for the data pairs of the first N-1 subsequences, N-1 rates of change are obtained; and an abnormal time period is estimated based on the maximum and minimum rates of change among the N-1 rates of change; the abnormal time period represents the estimated time period during which the database will experience an anomaly.

[0239] In some embodiments of this application, the information mining device 555 further includes: an information processing module 5555; the information processing module 5555 is used to estimate the abnormal time period based on the maximum and minimum change rates among the N-1 change rates, and then generate an alarm message to prompt the database to be expanded based on the abnormal time period; or, when the difference between the current time point and the starting time point of the abnormal time period is less than a preset time threshold, the database is expanded to obtain an expanded database.

[0240] In some embodiments of this application, the indicator mining module 5553 is further configured to: filter the trend type that appears most frequently from the trend types of each data pair as the trend feature corresponding to the monitoring indicator; assign a corresponding feature value to the trend type of each data pair; and calculate the confidence level of the trend feature based on the feature value of each data pair and the total number of data pairs.

[0241] In some embodiments of this application, the indicator mining module 5553 is further configured to filter out trend indicators from the monitoring indicators where the trend feature characterization indicator data shows the upward trend or the downward trend, and the confidence level exceeds the confidence level threshold.

[0242] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the information mining method described in this application.

[0243] This application provides a computer-readable storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to execute the information mining method provided in this application. For example, ... Figure 3 The method shown.

[0244] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0245] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0246] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0247] As an example, executable instructions can be deployed to execute on a single information mining device, or on multiple information mining devices located at one location, or on multiple information mining devices distributed across multiple locations and interconnected via a communication network.

[0248] In summary, the embodiments of this application not only automatically discover trend indicators, improving the efficiency of trend indicator discovery, but also fully consider the periodicity of the monitored indicator data during the trend indicator discovery process, thereby improving the accuracy of trend indicator discovery. Ultimately, the embodiments of this application improve the intelligence level of trend indicator discovery; they can also estimate abnormal time periods for alerts or automatic expansion, ensuring database security.

[0249] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. An information mining method characterized by comprising: The method comprises the following steps: a plurality of data collection is carried out on the monitoring index of the database, a plurality of index collection data is obtained, and a plurality of collection times of data collection are recorded; a plurality of index collection data is arranged according to the plurality of collection times, and a pair of data to be analyzed is obtained; wherein the pair of data to be analyzed comprises N-1 data pairs, and the collection time corresponding to the index collection data in the pair of data to be analyzed is different by one or more time periods; for each data pair in the N-1 data pairs, a corresponding difference set is constructed; the number of positive numbers and negative numbers in the difference set is counted, and the number of positive numbers and negative numbers of each data pair is obtained; according to the size relationship between the number of positive numbers and the number of negative numbers, the trend type of each data pair is determined, and the trend type is obtained; from the trend type of each data pair, the trend type with the most occurrences is selected as the trend feature corresponding to the monitoring index; from the monitoring index, the trend index is selected, which represents the upward trend or downward trend of the index collection data, as the information mining result.

2. The method of claim 1, wherein, the plurality of index collection data is arranged according to the plurality of collection times, and a pair of data to be analyzed is obtained; comprising: the plurality of index collection data is arranged according to the order of the plurality of collection times, and a data sequence is obtained; the data sequence is divided according to the sequence length corresponding to the time period, and a plurality of sub-sequences is obtained; based on the pairing of index collection data in adjacent sub-sequences in the plurality of sub-sequences, the pair of data to be analyzed is obtained.

3. The method of claim 2, wherein, the plurality of sub-sequences comprises N sub-sequences, and N is a positive integer greater than 1; based on the pairing of index collection data in adjacent sub-sequences in the plurality of sub-sequences, the pair of data to be analyzed is obtained, comprising: the next sub-sequence of each of the first N-1 sub-sequences in the N sub-sequences is determined as the adjacent sub-sequence; the index collection data in the first N-1 sub-sequences is paired with the index collection data in the corresponding adjacent sub-sequences to obtain N-1 data pairs; the N-1 data pairs are determined as the pair of data to be analyzed.

4. The method of claim 3, wherein, the index collection data in the first N-1 sub-sequences is paired with the index collection data in the corresponding adjacent sub-sequences to obtain N-1 data pairs, comprising: for each index collection data in the i-th sub-sequence in the first N-1 sub-sequences, the matching data which is different from the collection time of the index collection data by one time period is screened out from the corresponding adjacent sub-sequence; wherein i is greater than or equal to 1 and less than or equal to N-1; the each index collection data and the matching data are paired to generate the corresponding sub-data pair of the each index collection data; the data pair corresponding to the i-th sub-sequence is integrated by using the corresponding sub-data pair of the each index collection data; when the corresponding data pair of the first N-1 sub-sequences is generated, the N-1 data pairs are obtained.

5. The method according to any one of claims 1 to 4, characterized in that, The trend feature represents any one of the upward trend, the downward trend and the no-change trend of the index collection data.

6. The method of claim 1, wherein, The trend type of each data pair is determined according to the size relationship between the positive number and the negative number, and a trend type is obtained. When the positive number is greater than the sum of the negative number and the obtained gap threshold, it is determined that the trend type of each data pair is upward. When the sum of the positive number and the gap threshold is less than the negative number, it is determined that the trend type of each data pair is downward. When the difference between the positive number and the negative number is less than the gap threshold, it is determined that the trend type corresponding to each data pair is no trend.

7. The method of claim 4, wherein, After the N-1 data pairs are obtained by generating corresponding data pairs for the first N-1 sub-sequences, the method further comprises: The change rate of the i-th sub-sequence corresponding to the data pair is calculated by using the difference between the each index collection data and the matching data. When the change rate of the data pair of the first N-1 sub-sequences is generated, N-1 change rates are obtained. According to the maximum change rate and the minimum change rate in the N-1 change rates, an abnormal time period is estimated, which represents the estimated time period when the database is abnormal.

8. The method of claim 7, wherein, After the abnormal time period is estimated according to the maximum change rate and the minimum change rate in the N-1 change rates, the method further comprises: Based on the abnormal time period, an alarm information for prompting to expand the database is generated; or When the difference between the current time point and the start time point of the abnormal time period is less than a preset time threshold, the database is expanded to obtain an expanded database.

9. The method of claim 1, wherein, After the trend type with the most occurrences is filtered from the trend types of each data pair as the trend feature corresponding to the monitoring index, the method further comprises: A corresponding feature value is assigned to the trend type of each data pair; According to the feature value of each data pair and the total number of data pairs, the confidence of the trend feature is calculated.

10. The method of claim 9, wherein, The trend feature represents any one of the upward trend, the downward trend and the no-change trend of the index collection data. The trend feature represents any one of the upward trend, the downward trend and the no-change trend of the index collection data.

11. An information mining apparatus characterized by comprising: The data collection module is configured to collect data for monitoring indexes of a database multiple times to obtain multiple index collection data and record multiple collection times of data collection. The data arrangement module is configured to pair and arrange the multiple index collection data according to the multiple collection times to obtain data pairs to be analyzed; wherein the data pairs to be analyzed include N-1 data pairs, and the collection times corresponding to the index collection data in the data pairs to be analyzed differ by one or more time periods. ​ The index mining module is configured to, for each data pair in the N-1 data pairs, construct a corresponding difference set; The number of positive numbers and the number of negative numbers in the difference set are counted to obtain the number of positive numbers and the number of negative numbers of each data pair; the trend type of each data pair is determined according to the size relationship between the number of positive numbers and the number of negative numbers, to obtain a trend type; From the trend type of each data pair, the trend type with the highest occurrence frequency is selected as the trend feature corresponding to the monitoring index; from the monitoring index, a trend index is selected, which represents an upward trend or a downward trend, as an information mining result.

12. The apparatus of claim 11, wherein The data arrangement module is further configured to arrange the plurality of index collection data according to the order of the plurality of collection times to obtain a data sequence; The data sequence is divided into a plurality of subsequences according to the sequence length corresponding to the time period; The plurality of subsequences include N subsequences, N being a positive integer greater than 1; 13. The apparatus of claim 12, wherein, The data arrangement module is further configured to determine the next subsequence of each of the first N-1 subsequences as the adjacent subsequence; The index collection data in the first N-1 subsequences are paired with the index collection data in the corresponding adjacent subsequences to obtain N-1 data pairs; The N-1 data pairs are determined as the data pairs to be analyzed. comprises:

14. An information mining apparatus characterized by comprising: a memory configured to store executable instructions; a processor configured to execute the executable instructions stored in the memory to implement the information mining method of any one of claims 1 to 10. executable instructions stored in the memory, which are configured to be executed by the processor to implement the information mining method of any one of claims 1 to 10.

15. A computer-readable storage medium, characterized in that, The computer program or instructions are executed by the processor to implement the information mining method of any one of claims 1 to 10.

16. A computer program product comprising computer programs or instructions, characterized in that, ​

Images