Method for expanding analysis capability of time sequence database

Abstract

The invention provides a time sequence database analysis capability expansion method, and relates to the field of data analysis, and the method comprises the steps: constructing a time sequence data analysis system which operates independently from a time sequence database system and is used for providing a time sequence data analysis service, the time sequence data analysis system can be expanded, upgraded and maintained under the condition that the normal operation of the time sequence database system is not influenced; an adaptive interface is added in the time series data analysis system, and different time series data analysis models can be called; establishing an analysis operator following the operator rule of the time sequence database system for the time sequence data analysis service in the time sequence database system; in the SQL query execution process of the time sequence database system, the analysis operator calls the corresponding time sequence data analysis model through the adaptive interface to complete time sequence data analysis, interaction between the time sequence database system and the time sequence data analysis system which are isolated from each other in the processing process is achieved, and the complexity of the analysis processing process is shielded.

Description

Technical Field

[0001] This invention relates to the field of data analysis, and more particularly to a method for extending the analytical capabilities of time-series databases. Background Technology

[0002] Time-series database analysis capabilities include simple data filtering, extraction, and aggregation capabilities, as well as complex data prediction and anomaly analysis capabilities.

[0003] In the field of predictive analysis and anomaly detection of time series data, researchers have developed a number of advanced analysis algorithms with different technical characteristics and applicable to different application scenarios, such as time series data prediction, time series data anomaly detection, and time series data change point detection algorithms. These analysis algorithms are usually available as data analysis toolkits in high-level programming languages ​​such as Python and R.

[0004] Currently, data analysis toolkits embedded in time-series database systems can be used in a cross-language manner, and analysis algorithms embedded in time-series database systems can be invoked using SQL statements. However, this direct invocation method has the following problems: 1. Poor scalability and difficulty in upgrading analysis algorithms. The expansion, upgrading, and maintenance of analysis algorithms are independent of the time-series database system. Once the analysis algorithms are packaged into the time-series database system, they cannot be updated independently. Any expansion or upgrade of the analysis algorithms requires repackaging and compiling the entire time-series database software.

[0005] 2. The algorithm call flow intrudes into the query execution logic of the time-series database system. Specifically, the APIs for calling different analysis algorithms vary significantly, requiring separate design of call analysis and logic processing flows for each algorithm. This is especially true for analysis algorithms that require pre-training versus those that do not, necessitating differentiated treatment, and the complexity of the processing logic cannot be masked. Summary of the Invention

[0006] This invention provides a method for extending the analysis capabilities of time-series databases, solving the problems of poor scalability and difficulty in upgrading existing analysis algorithms, as well as the intrusion of the analysis algorithm's calling process into the query execution logic of the time-series database system.

[0007] This invention provides a method for extending the analysis capabilities of a time-series database. The method includes: constructing a time-series data analysis system that operates independently of a time-series database system to provide time-series data analysis services, so as to extend, upgrade, and maintain the time-series data analysis system without affecting the normal operation of the time-series database system; adding an adaptation interface to the time-series data analysis system that can call different time-series data analysis models; establishing analysis operators in the time-series database system that follow the operator rules of the time-series database system for the time-series data analysis service, so as to realize interaction between the time-series database system and the time-series data analysis system, whose processing flows are isolated from each other; and in the execution of an SQL query process by the time-series database system, the analysis operators call the corresponding time-series data analysis model through the adaptation interface in the time-series data analysis system to complete the time-series data analysis.

[0008] Preferably, the construction of a time-series data analysis system that operates independently of the time-series database system for providing time-series data analysis services includes: separating the time-series data analysis function from the existing time-series database system to form an independent time-series data analysis system, wherein the time-series data analysis function is implemented by a time-series data analysis model running in the corresponding operating environment.

[0009] Preferably, the expansion, upgrade, and maintenance of the time series analysis system includes: extending the time series data analysis system with a new time series data analysis model, specifically: constructing a class file for the new time series data analysis model, wherein the class file contains the analysis method of the new time series data analysis model; saving the class file of the new time series data analysis model to a designated directory of the time series data analysis system; and loading the new time series data analysis model into the time series data analysis system by loading the class file of the new time series data analysis model in the designated directory, thereby expanding the time series data analysis system without affecting the normal operation of the time series database system.

[0010] Preferably, adding an adaptation interface to the time series data analysis system that can call different time series data analysis models includes: constructing a wrapper class for each time series data analysis model in the time series data analysis system, wherein the wrapper classes of the same type of time series data analysis model inherit from the same base class, so as to obtain an adaptation interface that adapts to different time series data analysis models.

[0011] Preferably, the step of establishing analysis operators in the time-series database system that follow the operator rules of the time-series database system for the time-series data analysis service provided by the time-series database system to enable interaction between the time-series database system and the time-series data analysis system, whose processing flows are isolated from each other, includes: establishing an analysis operator in the time-series database system for each type of time-series data analysis service according to the operator rules of the time-series database system, and embedding each established analysis operator into the SQL query flow within the time-series database system; wherein, the function of the analysis operator includes: converting the time-series data and time-series data analysis model call information received from the upstream operator of the time-series database system into JSON format analysis requests and sending them to the time-series data analysis system; converting the JSON format analysis results received from the time-series data analysis system into analysis results that conform to the internal data format of the time-series database system and transmitting them directly or after aggregation processing to the downstream operator of the time-series database system.

[0012] Preferably, when the type of time series data analysis service is predictive analysis, the analysis operator is a predictive analysis operator. Correspondingly, in the execution of an SQL query process by the time series database system, the analysis operator calls the corresponding time series data analysis model through the adaptation interface in the time series data analysis system to complete the time series data analysis. This includes: when a user calls a time series data analysis model through an SQL statement to analyze time series data, the time series database system executes an SQL query process; in the execution of the SQL query process, the predictive analysis operator receives the time series data output by its upstream operator and verifies the call information of the time series data analysis model specified in the SQL statement. The predictive analysis operator converts the time-series data and the invocation information into a JSON-formatted analysis request and sends it to the time-series data analysis system. This allows the time-series data analysis system to receive and parse the JSON-formatted analysis request, invoke the time-series data analysis model indicated by the JSON-formatted analysis request through the adaptation interface, perform predictive analysis, and return the JSON-formatted analysis results of the time-series data analysis model to the time-series database system. The predictive analysis operator receives the JSON-formatted analysis results returned by the time-series data analysis system, converts the JSON-formatted analysis results into analysis results in the internal data format of the time-series database system, and passes them to the downstream operator.

[0013] Preferably, when the type of time-series data analysis service is anomaly detection, the analysis operator of the time-series data analysis system is anomaly detection operator. Correspondingly, in the execution of an SQL query process, the analysis operator calls the corresponding time-series data analysis model through the adaptation interface in the time-series data analysis system to complete the time-series data analysis, including: when a user calls a time-series data analysis model through an SQL statement to analyze time-series data, the time-series database system executes an SQL query process; in the execution of the SQL query process, the anomaly detection operator receives the time-series data output by its upstream operator and verifies the call information of the time-series data analysis model specified in the SQL statement; the anomaly detection operator will... The time-series data and the calling information are converted into a JSON format analysis request and sent to the time-series data analysis system. The time-series data analysis system receives and parses the JSON format analysis request, calls the time-series data analysis model indicated by the JSON format analysis request through the adaptation interface to perform anomaly detection, and returns the JSON format analysis result of the time-series data analysis model to the time-series database system. The anomaly detection operator converts the received JSON format analysis result into an analysis result in the internal data format of the time-series database system, and aggregates the analysis result in the internal data format of the time-series database system according to the aggregation function specified in the SQL statement, then passes the aggregation result to the downstream operator.

[0014] Preferably, in the SQL query execution process of the time series database system, before the analysis operator calls the corresponding time series data analysis model through the adaptation interface in the time series data analysis system to complete the time series data analysis, the method further includes: adding the address information of the time series data analysis system to the metadata node of the time series database system; the time series database system, according to the address information of the time series data analysis system already added to its metadata node, obtaining and saving the metadata information related to the time series data analysis model provided by the time series data analysis system.

[0015] Preferably, the time-series database system is deployed on a server equipped with a CPU, and the time-series data analysis system is deployed on a server equipped with a GPU.

[0016] Preferably, the relationship between the time-series database system and the time-series data analysis system is one of the following: one-to-one, one-to-many, many-to-one, or many-to-many.

[0017] The method for extending the time-series database analysis capabilities provided by this invention constructs a time-series data analysis system independent of the time-series database system. This ensures that the expansion, upgrade, and maintenance of the time-series data analysis system do not affect the normal operation of the time-series database system. The time-series data analysis models (i.e., analysis algorithms) in the time-series data analysis system have good scalability, high flexibility, and low upgrade difficulty. In addition, by setting an adaptation interface in the time-series data analysis system, it is possible to call different time-series data analysis models in the time-series data analysis system without developing separate calling logic and usage interfaces for each time-series analysis model. Furthermore, by establishing analysis operators in the time-series database system that follow the operator rules of the time-series database system for the time-series data analysis service, the SQL query process of the time-series database system is isolated from the complex processing logic in the time-series data analysis system. Attached Figure Description

[0018] Figure 1 This is a flowchart of the method for extending the time-series database analysis capabilities provided by the present invention; Figure 2 This is an architecture diagram of the time series data analysis system provided by the present invention; Figure 3 This is a schematic diagram of a JSON format analysis request for predictive analysis provided by the present invention; Figure 4 It is a response Figure 3 A diagram illustrating the analysis results of the request in JSON format; Figure 5 This is a diagram showing the relationship between the time-series database system and the time-series data analysis system provided by this invention. Figure 6 This is a flowchart of the analysis call process provided by the present invention. Detailed Implementation

[0019] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the embodiments described below are only for illustration and explanation of the present invention and are not intended to limit the present invention.

[0020] This invention provides a method for extending the analytical capabilities of time-series databases, which is used to expand the analytical capabilities of time-series database systems, including but not limited to complex time-series data analysis capabilities such as time-series data prediction analysis and time-series data anomaly detection.

[0021] See Figure 1 The present invention provides a method for extending the analysis capabilities of time-series databases, comprising the following steps: Step S101: Construct a time-series data analysis system that operates independently of the time-series database system to provide time-series data analysis services, so as to expand, upgrade and maintain the time-series data analysis system without affecting the normal operation of the time-series database system.

[0022] On the one hand, complex time-series data analysis functions, such as predictive analysis and anomaly detection functions, can be separated from existing time-series database systems to form independent time-series data analysis systems. These functions are implemented by time-series data analysis models running in corresponding operating environments. On the other hand, new time-series data analysis models (i.e., new analysis algorithms) providing time-series data analysis functions can be extended to this independent time-series data analysis system. Specifically, a class file with a calling name based on the new time-series data analysis model is constructed for the new model. This class file contains the analysis methods of the new time-series data analysis model. The class file is then saved to a designated directory of the time-series data analysis system. By loading the class file from the designated directory, the new time-series data analysis model is loaded into the time-series data analysis system, extending its analytical capabilities without affecting the normal operation of the time-series database system.

[0023] In this invention, since the time series data analysis system and the time series database system are independent of each other, the upgrade, maintenance and expansion of the time series data analysis model in the time series data analysis system are not limited by the time series database system. Therefore, the time series data analysis system has good scalability, high flexibility and is easier to upgrade and maintain.

[0024] Step S102: Add an adaptation interface to the time series data analysis system that can call different time series data analysis models.

[0025] Specifically, an encapsulation class is constructed for each time series data analysis model in the time series data analysis system. The encapsulation classes of the same type of time series data analysis model inherit from the same base class to obtain an adaptation interface that adapts to different time series data analysis models.

[0026] In addition, the time series data analysis system also includes a request processing and adaptation library. The functions of the request processing and adaptation library include: (1) receiving and parsing analysis requests from the time series database system; (2) according to the call information of the time series data analysis model configured in the analysis request, such as the model name and call parameters, calling the time series data analysis model indicated in the analysis request through the adaptation interface to analyze the time series data in the analysis request; (3) returning the analysis results obtained by the time series data analysis model from analyzing the time series data to the time series database system.

[0027] This invention integrates different time series data analysis models into a standardized analysis service through request processing, adaptation libraries, and adaptation interfaces, thereby improving the time series data analysis system's support for different time series data analysis models. Therefore, it is not necessary to develop separate calling logic and usage interfaces for each time series analysis model.

[0028] Step S103: Establish analysis operators in the time series database system that follow the operator rules of the time series database system for the time series data analysis service, so as to realize interaction between the time series database system and the time series data analysis system, whose processing processes are isolated from each other.

[0029] According to the time-series database system operator rules, an analysis operator is established for each type of time-series data analysis service in the time-series database system. Each established analysis operator is embedded into the SQL query flow within the database system, so that when a user specifies a time-series data analysis service in an SQL statement, the SQL plan generator of the time-series database system will add the corresponding analysis operator to the syntax tree. The operator rules utilize existing technology and will not be elaborated further here. The functions of the analysis operator include: converting time-series data and time-series data analysis model call information received from upstream operators of the time-series database system into JSON format analysis requests and sending them to the time-series data analysis system; converting received JSON format analysis results from the time-series data analysis system into analysis results conforming to the internal data format of the time-series database system and directly transmitting them or transmitting them to downstream operators of the time-series database system after aggregation processing. Clearly, integrating the analytical operators with the above functions into the SQL query process of the time series database system can not only shield the query processing flow of the time series database system from the complex analytical processing logic of the time series data analysis system, but also utilize the analytical capabilities of the time series data analysis system to allow the time series database system to obtain the processing results of different analytical tasks.

[0030] Step S104: In the SQL query execution process of the time series database system, the analysis operator calls the corresponding time series data analysis model through the adaptation interface in the time series data analysis system to complete the time series data analysis.

[0031] For example, when the type of time series data analysis service is predictive analysis, the analysis operator is a predictive analysis operator. Accordingly, step S104 includes: when a user calls a time series data analysis model to analyze time series data via an SQL statement, the time series database system executes an SQL query process; during the execution of the SQL query process, the predictive analysis operator receives the time series data output by its upstream operator and verifies the call information of the time series data analysis model specified in the SQL statement; the predictive analysis operator converts the time series data and the call information into a JSON format analysis request and sends it concurrently. The request is sent to the time series data analysis system, so that the time series data analysis system can use its request processing and adaptation library to receive and parse the JSON format analysis request, call the time series data analysis model indicated by the JSON format analysis request through the adaptation interface to perform predictive analysis, and return the JSON format analysis result of the time series data analysis model to the time series database system; the predictive analysis operator receives the JSON format analysis result returned by the time series data analysis system, converts the JSON format analysis result into the analysis result of the internal data format of the time series database system, and passes it to the downstream operator.

[0032] For example, when the type of time series data analysis service is anomaly detection, the analysis operator of the time series data analysis system is an anomaly detection operator. Accordingly, step S104 includes: when a user calls a time series data analysis model to analyze time series data via an SQL statement, the time series database system executes an SQL query process; during the execution of the SQL query process, the anomaly detection operator receives the time series data output by its upstream operator and verifies the call information of the time series data analysis model specified in the SQL statement; the anomaly detection operator converts the time series data and the call information into a JSON format analysis request and sends it to the time series data analysis system. The system is configured to receive and parse the JSON-formatted analysis request using its request processing and adaptation library, call the time-series data analysis model indicated by the JSON-formatted analysis request through the adaptation interface to perform anomaly detection, and return the JSON-formatted analysis result of the time-series data analysis model to the time-series database system. The anomaly detection operator converts the received JSON-formatted analysis result into the analysis result of the internal data format of the time-series database system, and performs aggregation processing on the analysis result of the internal data format of the time-series database system according to the aggregation function specified in the SQL statement, and passes the aggregation result to the downstream operator.

[0033] This invention is not limited to predictive analysis operators and anomaly detection operators. Depending on the actual analysis needs, it can also establish analysis operators for other types of time series data analysis services and call the corresponding time series data analysis models in the time series data analysis system to provide time series data analysis services.

[0034] Furthermore, in order for the time-series database system to be aware of the existence of the time-series data analysis system and to dynamically invoke the time-series data analysis model in the time-series data analysis system during the SQL query process, the independently deployed and running time-series data analysis system needs to be dynamically registered with the time-series database system. Specifically, this includes: before the analysis operator calls the corresponding time-series data analysis model through the adaptation interface in the time-series data analysis system to complete the time-series data analysis during the execution of the SQL query process in the time-series database system, the address information of the time-series data analysis system is added to the metadata node of the time-series database system. The time-series database system then retrieves and saves the metadata information related to the time-series data analysis model provided by the time-series data analysis system based on the address information of the time-series data analysis system already added to its metadata node.

[0035] Furthermore, in existing solutions that embed analytical algorithms into time-series database systems, the reasoning and analysis process of these algorithms is a computationally intensive task, requiring significant CPU power and even relying on GPU acceleration. Frequent invocation of these algorithms can place a heavy computational load on the time-series database system's server, thus affecting the normal operation of other services. Therefore, the time-series database system of this invention is deployed on a server equipped with a CPU to ensure the server handles routine database functions. The time-series data analysis system is deployed on a server equipped with a GPU to improve the performance of the time-series data analysis service.

[0036] Furthermore, the relationship between the time-series database system and the time-series data analysis system is one of the following: one-to-one, one-to-many, many-to-one, or many-to-many. A time-series data analysis system, through collaboration with one or more time-series database systems, can provide one or more types of time-series data analysis services to one or more time-series database systems. A time-series database system, by accepting registrations from one or more time-series data analysis systems, can invoke the time-series data analysis models of one or more time-series data analysis systems. In this invention, there is no information exchange between multiple time-series data analysis systems, and each time-series data analysis system provides stateless time-series data analysis services, thus enabling rapid horizontal scaling of the time-series data analysis system.

[0037] The following combination Figures 2 to 6 This invention will be described in detail from five aspects: the architecture of the time series data analysis system, the message exchange format between the time series database system and the time series data analysis system, the interaction between the time series database system and the time series data analysis system, the processing flow of analysis calls, and the way to add new time series data analysis models to the time series data analysis system.

[0038] I. Architecture of the Time Series Data Analysis System See Figure 2 A time-series data analysis system that can be deployed and run independently includes: 1. Time series data analysis model. Time series data analysis models can provide various types of analysis services, offering different time series data analysis services for different scenarios, including predictive analysis, anomaly detection, missing data completion, pattern classification, and other analysis service functions. They serve as the carrier of analysis services.

[0039] Time series data analysis models can be categorized by model type into machine learning models, basic time series models, and deep learning models. Among them, basic time series models are large-scale pre-trained models designed for time series data, which achieve multi-task generalization capabilities through a general architecture.

[0040] 2. Basic runtime libraries. The basic runtime library provides the foundational environment for running time series data analysis models. It typically consists of various types of Python runtime libraries used to drive the execution of time series data analysis models, including machine learning models, basic time series models, and deep learning models.

[0041] 3. Request processing and adaptation libraries. The request processing and adaptation library is responsible for receiving and parsing JSON-formatted request messages, and then converting the analysis results back to JSON format before returning them to the requester.

[0042] To shield the differences between various time-series data analysis models and achieve unified access, the time-series data analysis system of this invention utilizes Python's class encapsulation and inheritance mechanisms to achieve unified management and access to different analysis models.

[0043] For each time series data analysis model, a corresponding Python class should be created to encapsulate that model. The class name is usually based on the model name. For example, the encapsulation class for the predictive analytics model Holtwinters is _HoltWintersService, and the encapsulation class for the predictive analytics model ARIMA is _ArimaService.

[0044] Time series data analysis models of the same type should inherit from the corresponding base class. For example, predictive analysis classes such as Holtwinters and ARIMA inherit from the AbstractForecastService class, and anomaly detection classes inherit from the AbstractAnomalyDetectionService class. This ensures that models of the same type have a unified basic structure.

[0045] Based on the base class's requirements for subclasses, all wrapper classes for the model need to implement two key functions: 1. the `set_param()` function, which sets the parameters necessary for model execution; and 2. the `execute()` function, which executes the model analysis process. This ensures that a unified model object conforming to a consistent interface is obtained during model invocation.

[0046] The time series database system specifies the analysis model to be used through a JSON format request message. The request processing and adaptation library of the time series data analysis system parses the JSON format request message to obtain the name of the model to be called. For example, if Holtwinters is specified as the predictive analysis model, then the corresponding wrapper class _HoltWintersService will be instantiated to generate a _HoltWintersService object. Then, the set_param() function of the object is called to pass in the call parameters, and then the execute() function of the object is called to execute the analysis process in the runtime environment.

[0047] By encapsulating the analysis model, using class inheritance hierarchy, and adapting the interfaces set_param() and execute(), regardless of whether the analysis model is Holtwinters, ARIMA, or another model, external calls to the analysis model do not need to worry about model differences. This achieves the shielding of external time series database systems from the differences between different analysis models and provides unified management of different internal analysis models.

[0048] II. Message exchange format between time series database systems and time series data analysis systems. The time series data analysis system provides analysis functions such as time series data prediction and time series data anomaly detection. It provides a unified service request interface to call different types of time series data analysis models, such as traditional machine learning analysis models, pre-trained deep learning models, and cutting-edge time series basic models.

[0049] The JSON-formatted analysis request contains the time-series data to be analyzed and the calling information of the time-series data analysis model (such as model name, calling parameters, etc.). This analysis request is sent to the time-series data analysis system via HTTP. For an example of a predictive analysis request, see [link to example]. Figure 3 The "algo" field specifies the model used for predictive analysis. Obviously, by modifying the content of the "algo" field, different model names can be set, thus calling different time series data analysis models in the time series data analysis system. The "data" field contains the time series data for predictive analysis, and the other fields contain option information for the time series data analysis model. The analysis results are also returned to the requesting time series database system in JSON format. See [link to typical predictive analysis return results](link to JSON). Figure 4 .

[0050] JSON is a standard format widely used for information exchange between different systems. Therefore, this invention uses JSON for data exchange and information transmission between the time-series data database system and the time-series data analysis system. It should be noted that this format is not used internally within the system because its expression efficiency is too low, resulting in high overhead and impacting system performance.

[0051] III. Interaction between time-series database systems and time-series data analysis systems. 1. The time-series data analysis system registers with the time-series database system so that the time-series database system is aware of the time-series data analysis system's existence and can send analysis requests to the time-series data analysis system. The registration process is as follows: Step 1.1: The system administrator uses a command to add the IP:PORT information of the time series data analysis system service to the time series database system metadata node.

[0052] Step 1.2: After the addition is completed, the time series database system actively requests the time series data analysis system to obtain the metadata information related to the time series data analysis model it provides, such as basic model attributes (e.g., name, version, creation time), input and output data characteristics (e.g., time series fields, time granularity, data type), architecture parameters (e.g., number of network layers, number of neurons, activation function), training configuration (e.g., optimizer, loss function, number of iterations), evaluation metrics (e.g., MAE, RMSE, MAPE), etc.

[0053] Step 1.3: After obtaining the metadata information of the time series data analysis model, the time series database system caches it in the metadata node of the time series database system and synchronizes it to other nodes of the time series database system, such as compute nodes, storage nodes, and backup nodes, through heartbeat.

[0054] 2. The relationship between a time-series data analysis system and a time-series database system can be many-to-many. See [link / reference]. Figure 5 This means that a time-series database system can register multiple time-series data analysis systems as needed. A time-series data analysis system can also register with multiple time-series database systems.

[0055] The time-series database system is deployed on servers with high-performance CPUs. The analysis system is deployed on servers equipped with high-performance GPUs, using GPUs to accelerate the machine learning / deep learning analysis process. Both the CPUs and GPUs are currently available high-performance CPUs and GPUs. Registering the time-series data analysis system with multiple time-series database systems allows the high-performance GPU servers to be shared across these systems, fully leveraging the performance advantages of the hardware.

[0056] 3. For each type of time series data analysis model, corresponding operators need to be established within the time series database system and integrated into the SQL query process within the time series database system.

[0057] In database systems, an operator is the basic functional unit that manipulates data when executing an SQL query, typically manifested as a specific operation step in the query plan. Examples include the scan operator, which scans the entire table; the aggregate operator, which aggregates data to generate results; and the project operator, which extracts table data by column.

[0058] To support predictive analysis and anomaly detection of time series data, this invention adds a predictive analysis operator for processing predictive analysis logic and an anomaly detection operator for processing anomaly detection logic to the time series database system.

[0059] The main functions of predictive analytics operators are as follows: (1) Receive timing data read by upstream operators (usually scan operators, etc.); (2) Verify the calling parameter information (or calling information) of the time series data analysis model specified in the SQL statement; (3) Convert the timing data and call information into a request message, in the following format: Figure 3 The data, in JSON format, is sent to the time series data analysis system.

[0060] (4) After receiving the analysis results in JSON format, the forecast operator converts them into the internal format of the time series database system (used in the calculation process) and passes them to downstream operators to continue the query execution process.

[0061] Anomaly detection operators are aggregation operators that aggregate time-series data within an outlier (data) point window formed by consecutive outlier data points according to a user-specified aggregation function. Their main functions are as follows: (1) Receive timing data read by upstream operators (usually scan operators, etc.); (2) Verify the calling parameter information (or calling information) of the time series data analysis model provided in the SQL statement; (3) Convert the time series data and call information into an analysis request in JSON format and send the analysis request to the time series data analysis system; (4) After receiving the analysis results in JSON format, the anomaly detection operator converts them into the internal format of the time series database system and generates aggregation results using the aggregation function specified by the user; (5) Pass the generated results to downstream operators.

[0062] IV. Analysis of the call processing flow. 1. See Figure 6 The main execution flow of the predictive analysis operator is as follows: (1) Receive data input from downstream operators.

[0063] The data used in the analysis usually comes from upstream operators. The upstream operator of the predictive analysis operator is usually a scan operator, which is responsible for scanning the data in the table and passing it to the predictive analysis operator for time series data predictive analysis.

[0064] (2) The predictive analysis operator is responsible for receiving the time series data obtained by the scanning operator and performing a validity check on the received time series data and the call information used to invoke the time series data analysis model. After the check is passed, the predictive analysis operator converts the time series data and call information used for analysis into an analysis request in JSON format and sends it to the time series data analysis system. After that, the predictive analysis operator itself is in a blocked state to wait for the message returned by the time series data analysis system.

[0065] (3) After receiving the analysis request of time series data, the time series data analysis system parses the analysis request and adapts the time series data analysis model according to the call information to perform analysis and processing. After the analysis is completed, the analysis results in JSON format will be returned.

[0066] (4) The predictive analysis operator receives the analysis results in JSON format returned by the time series data analysis system and transforms them into the data structure inside the time series database system. It then unblocks itself and passes the results to the downstream operator of the predictive analysis operator, namely the project operator, to continue the query processing flow.

[0067] 2. Similar to the main execution flow of the predictive analytics operator, the main execution flow of the anomaly detection operator is as follows: (1) Receive data input from the upstream operator.

[0068] The data used in the analysis usually comes from the upstream operator. The upstream operator of the anomaly detection operator is usually the scan operator. The scan operator is responsible for passing the data in the table to the anomaly detection operator for time series data anomaly detection.

[0069] (2) The anomaly detection operator is responsible for receiving the time series data obtained by the scanning operator and performing a validity check on the received time series data and the call information used to invoke the time series data analysis model. After the check is passed, the time series data and call information used for analysis are converted into an analysis request in JSON format and sent to the time series data analysis system. After that, the anomaly detection operator itself is in a blocked state to wait for the message returned by the time series data analysis system.

[0070] (3) After receiving the analysis request of time series data, the time series data analysis system parses the analysis request and adapts the time series data analysis model according to the call information to perform analysis and processing. After the analysis is completed, the analysis results in JSON format will be returned.

[0071] (4) The anomaly detection operator receives the analysis results in JSON format returned by the time series data analysis system and transforms them into the data structure inside the time series database system. It then unblocks itself, aggregates the analysis results, and passes the aggregated results to the downstream operator of the anomaly detection operator, namely the project operator, to continue the query processing flow.

[0072] V. Methods for Adding New Time Series Data Analysis Models to a Time Series Data Analysis System The steps for extending the time series data analysis model are as follows: Step 1: Add a class to the time series data analysis system that begins with an underscore (_). This class needs to inherit from the AbstractForecastService class and implement the virtual methods defined therein.

[0073] Step 2: Set an appropriate name (the name is the calling name of the model) and description for the new class.

[0074] Step 3: Implement the main method (method) `execute` for the added class. The `execute` method is the main method of the time series data analysis model. The specific logic implementation of the time series data analysis model needs to be executed in this method. At the end of the execution of the `execute` method, the time series data prediction analysis results need to be returned in the format of `<timestamp column> <predicted data column>`.

[0075] The added time series data analysis model class files are saved in the directory specified by the time series data analysis system, for example: . / lib / taosanalytics / algo / fc / directory.

[0076] By using commands or restarting the time series data analysis system, the time series data analysis model in this directory can be loaded. At this time, the newly added time series data analysis model will be loaded into the time series data analysis system and can provide time series data analysis services to external parties.

[0077] Because the time-series data analysis system is dynamically registered with the time-series database system, the time-series database system can obtain metadata information related to newly added time-series data analysis models. When a user needs to call this newly added time-series data analysis model, they can do so by modifying the "algo" field in the JSON format analysis request to the newly added time-series data analysis model.

[0078] It should be noted that time series data analysis systems do not save information related to the query process, do not record user information, provide stateless time series data analysis services, and there is no information exchange between different analysis systems, so they can be quickly scaled horizontally.

[0079] The present invention has the following technical effects: 1. Time-series database systems, as the core of application systems, are typically not allowed to be restarted or shut down in production environments. However, time-series data analysis systems require retraining, deployment, and upgrading of time-series data analysis models according to application needs. This invention separates the time-series data analysis function from the time-series database system to form an independent time-series data analysis system. This ensures that the expansion, upgrading, maintenance, and operation of the time-series data analysis system do not affect the normal operation of the core system (time-series database system).

[0080] 2. This invention separates the complexity of time-series data analysis from the time-series database system. The predictive analytics operator and the anomaly detection operator are only responsible for information interaction with the time-series data analysis system and are not involved in the analysis and processing process. Furthermore, the predictive analytics operator and the anomaly detection operator follow the rules of time-series database operators, thus enabling them to be embedded into the SQL query execution plan of the time-series database system. By isolating the complexity of the analysis and processing process through the predictive analytics operator and the anomaly detection operator, the influence of adjustments to the time-series data analysis model on the query processing process of the time-series database system can be avoided.

[0081] 3. The time-series data analysis system of this invention can be deployed on a server equipped with a GPU, which can greatly improve the performance of the time-series data analysis service. The time-series database system is deployed on a CPU server, which can ensure high-performance processing of conventional database functions in addition to time-series data analysis functions.

[0082] 4. The time series data analysis system of the present invention can work in conjunction with one or more time series database systems to provide various types of time series data analysis services.

[0083] 5. The time series data analysis system of the present invention can receive and parse standardized JSON format request messages, and perform data analysis by calling the time series data analysis model indicated by the JSON format request message through the adaptation interface, without having to develop calling logic and use interfaces for each type of analysis model.

[0084] 6. After adding a new time series data analysis model, the user can directly call the new time series data analysis model through SQL statements in the time series data analysis system of the present invention, without the need to upgrade / shut down / redeploy the time series database system.

[0085] Although the present invention has been described in detail above, it is not limited thereto, and those skilled in the art can make various modifications based on the principles of the present invention. Therefore, all modifications made in accordance with the principles of the present invention should be understood to fall within the protection scope of the present invention.

Claims

1. A method for extending the analytical capabilities of a time-series database, characterized in that, The method includes: Construct a time-series data analysis system that operates independently of the time-series database system to provide time-series data analysis services, so as to expand, upgrade, and maintain the time-series data analysis system without affecting the normal operation of the time-series database system; Add an adaptation interface to the time series data analysis system that can call different time series data analysis models; In the time series database system, analysis operators that follow the operator rules of the time series database system are established for the time series data analysis service, so as to realize interaction between the time series database system and the time series data analysis system, whose processing processes are isolated from each other; In the SQL query execution process of the time-series database system, the analysis operator calls the corresponding time-series data analysis model through the adaptation interface in the time-series data analysis system to complete the time-series data analysis.

2. The method according to claim 1, characterized in that, The construction of a time-series data analysis system that operates independently of the time-series database system and is used to provide time-series data analysis services includes: The time series data analysis function is separated from the existing time series database system to form an independent time series data analysis system, wherein the time series data analysis function is implemented by a time series data analysis model running in the corresponding operating environment.

3. The method according to claim 1, characterized in that, The expansion, upgrade, and maintenance of the time series analysis system includes: expanding the time series data analysis system with a new time series data analysis model, specifically: A class file for constructing the new time series data analysis model is provided, wherein the class file of the new time series data analysis model contains the analysis method of the new time series data analysis model; Save the class file of the new time series data analysis model to the specified directory of the time series data analysis system; By loading the class file of the new time series data analysis model in the specified directory, the new time series data analysis model is loaded into the time series data analysis system, thereby expanding the time series data analysis system without affecting the normal operation of the time series database system.

4. The method according to claim 1, characterized in that, The addition of an adaptation interface to the time series data analysis system that can call different time series data analysis models includes: An encapsulation class is constructed for each time series data analysis model in the time series data analysis system. The encapsulation classes of the same type of time series data analysis model inherit from the same base class to obtain an adaptation interface that adapts to different time series data analysis models.

5. The method according to claim 4, characterized in that, The step of establishing analysis operators in the time-series database system that follow the operator rules of the time-series database system for the time-series data analysis service, so as to realize interaction between the time-series database system and the time-series data analysis system, whose processing flows are isolated from each other, includes: According to the time-series database system operator rules, an analysis operator is established for each type of time-series data analysis service in the time-series database system, and each established analysis operator is embedded in the SQL query process within the time-series database system. The functions of the analysis operator include: converting the time-series data and time-series data analysis model call information received from the upstream operator of the time-series database system into JSON format analysis requests and sending them to the time-series data analysis system; converting the JSON format analysis results received from the time-series data analysis system into analysis results that conform to the internal data format of the time-series database system and transmitting them directly or after aggregation processing to the downstream operator of the time-series database system.

6. The method according to claim 5, characterized in that, When the type of time series data analysis service is predictive analysis, the analysis operator is a predictive analysis operator. Correspondingly, during the execution of an SQL query process in the time series database system, the analysis operator calls the corresponding time series data analysis model through the adaptation interface in the time series data analysis system to complete the time series data analysis, including: When a user calls a time-series data analysis model through an SQL statement to analyze time-series data, the time-series database system executes an SQL query process. During the execution of the SQL query process, the predictive analysis operator receives the time series data output by its upstream operator and verifies the calling information of the time series data analysis model specified in the SQL statement; The predictive analysis operator converts the time series data and the calling information into a JSON format analysis request and sends it to the time series data analysis system, so that the time series data analysis system can receive and parse the JSON format analysis request, call the time series data analysis model indicated by the JSON format analysis request through the adaptation interface to perform predictive analysis, and return the JSON format analysis results of the time series data analysis model to the time series database system. The predictive analysis operator receives the JSON-formatted analysis results returned by the time series data analysis system, converts the JSON-formatted analysis results into the analysis results of the internal data format of the time series database system, and passes them to the downstream operator.

7. The method according to claim 5, characterized in that, When the type of the time series data analysis service is anomaly detection, the analysis operator of the time series data analysis system is anomaly detection operator. Correspondingly, during the execution of an SQL query process, the analysis operator in the time series database system calls the corresponding time series data analysis model through the adaptation interface in the time series data analysis system to complete the time series data analysis, including: When a user calls a time-series data analysis model through an SQL statement to analyze time-series data, the time-series database system executes an SQL query process. During the execution of the SQL query process, the anomaly detection operator receives the time-series data output by its upstream operator and verifies the call information of the time-series data analysis model specified in the SQL statement; The anomaly detection operator converts the time-series data and the call information into a JSON format analysis request and sends it to the time-series data analysis system, so that the time-series data analysis system can receive and parse the JSON format analysis request, call the time-series data analysis model indicated by the JSON format analysis request through the adaptation interface to perform anomaly detection, and return the JSON format analysis result of the time-series data analysis model to the time-series database system. The anomaly detection operator converts the received JSON-formatted analysis results into analysis results in the internal data format of the time-series database system, and aggregates the analysis results in the internal data format of the time-series database system according to the aggregation function specified in the SQL statement, and then passes the aggregation results to the downstream operator.

8. The method according to claim 1, characterized in that, In the SQL query execution process of the time-series database system, before the analysis operator calls the corresponding time-series data analysis model through the adaptation interface of the time-series data analysis system to complete the time-series data analysis, the method further includes: Add the address information of the time series data analysis system to the metadata node of the time series database system; The time-series database system retrieves and saves metadata information related to the time-series data analysis model provided by the time-series data analysis system from the time-series data analysis system according to the address information of the time-series data analysis system that has been added to its metadata node.

9. The method according to any one of claims 1-8, characterized in that, The time-series database system is deployed on a server equipped with a CPU, and the time-series data analysis system is deployed on a server equipped with a GPU.

10. The method according to any one of claims 1-8, characterized in that, The relationship between the time-series database system and the time-series data analysis system is one of the following: one-to-one, one-to-many, many-to-one, or many-to-many.

Images