Query optimization method and system based on general time series framework
By generating candidate execution plans and performing cost estimation and quality assessment, the problem of low resource utilization in the general time series analysis framework is solved, adaptive execution plan optimization is achieved, and the system's resource utilization efficiency and analysis quality are improved.
Patent Information
- Application Number
- CN202611345372.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-01
- Publication Date
- 2026-11-06
AI Technical Summary
Existing general-purpose time series analysis frameworks use a static pipeline approach during the execution phase, which leads to full encoder calls, redundant feature calculations, increased inference latency, and decreased resource utilization, and lacks the ability to adaptively select execution plans.
By receiving time series analysis requests, parsing data, task types, budget constraints, and resource status, generating candidate execution plans, performing cost estimation and quality assessment, selecting the target execution plan using Pareto pruning and constraint search, and optimizing resource utilization through a multi-stage progressive execution mechanism.
It achieves adaptive selection of encoder subsets and fusion operators, reduces redundant model inference and resource waste, improves the controllability and stability of the system in multi-task scenarios, and reduces optimizer overhead and end-to-end execution time.
Smart Images
Figure CN123048598A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of database query optimization technology, specifically, it relates to a query optimization method and system based on a general time series framework. Background Technology
[0002] In recent years, scenarios such as industrial monitoring, IoT sensing, server maintenance, financial transactions, intelligent transportation, and energy management have continuously generated a large amount of time-series data. Time-series data is typically characterized by its large scale, rapid generation, complex variable correlations, strong noise interference, and dynamic data distribution. Practical analysis tasks include not only classification, clustering, and prediction, but also anomaly detection, missing value imputation, and other types of tasks. Traditional analysis methods that rely on manual features and task-specific models often require separate training, deployment, and parameter tuning for different tasks, making it difficult to meet the needs of unified modeling, flexible deployment, and resource control.
[0003] The development of self-supervised representation learning has made general time series analysis frameworks possible. General time series analysis frameworks, represented by UniTS, typically employ pre-trained encoders to learn transferable time series representations, which are then combined with feature fusion modules and downstream task models to complete various time series analysis tasks. These frameworks can reuse pre-trained representations, reducing the cost of repetitive training to some extent and improving cross-task generalization ability.
[0004] However, existing general-purpose time series analysis frameworks typically employ a static pipeline approach during the execution phase. For example, upon receiving a task request, the system defaults to calling a fixed encoder combination, using a fixed feature fusion strategy, and then inputting the fusion result into the corresponding task model. Since different tasks, data features, and resource constraints have varying requirements for representation capabilities and execution overhead, a fixed pipeline can easily lead to full encoder calls, redundant feature computations, increased inference latency, and decreased resource utilization. Some encoders contribute little to the current task but still incur model inference and feature fusion overhead; some lightweight scenarios do not require complex execution paths but are forced to execute high-cost models.
[0005] Database query optimization techniques improve query execution efficiency through cost models, execution plan search, and plan selection mechanisms. However, traditional query optimizers primarily focus on operations such as SQL operators, index selection, join order, and data scanning, lacking direct modeling methods for the time series analysis execution flow comprised of deep learning representation encoders, feature fusion operators, and task models. How to abstract time series analysis tasks into composable, comparable, and searchable execution plans, and adaptively balance analysis quality and execution cost, is a crucial technical problem that general-purpose time series analysis systems need to address. Summary of the Invention
[0006] The purpose of this invention is to provide a query optimization method and system based on a general time series framework, which solves the problems of existing general time series analysis frameworks lacking adaptive execution plan selection capabilities, easily generating redundant model inference, and fixed fusion overhead.
[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0008] A query optimization method based on a general time series framework includes the following steps:
[0009] S1, receive a time series analysis request, and parse the time series data to be analyzed, task type, budget constraint, task quality lower limit and current system resource status from the time series analysis request;
[0010] S2, generate multiple candidate execution plans based on the pre-trained representation encoder set, feature fusion operator set, and task model set, wherein each candidate execution plan includes at least a selected subset of representation encoders, feature fusion operators, task models, and running parameters;
[0011] S3, perform cost estimation and quality assessment for each candidate execution plan to obtain the predicted execution cost and expected task quality of the candidate execution plan;
[0012] S4. Based on the predicted execution cost, expected task quality, budget constraints, and task quality lower limit, perform feasibility filtering, Pareto pruning, and constraint search on multiple candidate execution plans to determine the target execution plan.
[0013] S5. Execute the corresponding time series analysis task according to the target execution plan, and update the cost estimation parameters, quality assessment parameters, and execution strategy parameters based on the actual execution results.
[0014] Furthermore, in this invention, the time series analysis request is represented as Q=<X,T,B,δ,R> Where X represents the input time series data, T represents the task type, B represents the budget constraint, δ represents the task quality lower limit, and R represents the current system resource status; the task type includes at least one of time series classification, clustering, prediction, anomaly detection, and missing value imputation.
[0015] Furthermore, in this invention, the candidate execution plan is represented as P=<S,φ,gT,θ> , where S represents a subset of encoders selected from the pre-trained representation encoder set, φ represents a feature fusion operator, gT represents a task model corresponding to task type T, and θ represents at least one of the following operating parameters: batch size, threshold, model accuracy mode, or execution device.
[0016] Furthermore, in this invention, step S2 includes the following steps:
[0017] S21. Determine the set of available task models based on the task type, and determine the set of available representation encoders based on the number of samples, sequence length, variable dimension, missing proportion, and data window size of the input time series data.
[0018] S22, enumerate or sample to obtain multiple encoder subsets, and combine each encoder subset with at least one feature fusion operator among splicing fusion, projection fusion, weighted fusion or gated fusion;
[0019] S23, bind the combined results with the corresponding task model and running parameters to obtain a set of candidate execution plans;
[0020] S24, eliminate candidate execution plans that do not match the task type, exceed resource hard constraints, or lack necessary input dimensions.
[0021] Furthermore, in this invention, the cost estimation includes encoder inference cost, feature fusion cost, task model execution cost, data transmission cost, and optimizer overhead; wherein, encoder inference cost is jointly determined by encoder parameter size, floating-point computation, input sequence length, variable dimension, batch size, and running hardware status; feature fusion cost is jointly determined by selected encoder output dimension, fusion operator type, and sample batch; and task model execution cost is jointly determined by task model structure, output space size, and running parameters.
[0022] Furthermore, in this invention, the quality assessment includes representation quality assessment, fusion quality assessment, and task adaptation quality assessment; wherein representation quality assessment is used to measure the degree of adaptation of a selected subset of encoders to the current task and input data features, fusion quality assessment is used to measure the complementarity and redundancy between the output representations of different encoders, and task adaptation quality assessment is used to measure the expected accuracy, AUC, prediction error, anomaly detection effect, or imputation error of the task model under the current execution plan.
[0023] Furthermore, in this invention, the non-dominated pruning is Pareto pruning; when the predicted execution cost of the first candidate execution plan is not higher than that of the second candidate execution plan and the expected task quality is not lower than that of the second candidate execution plan, and at least one metric is significantly better than that of the second candidate execution plan, the second candidate execution plan is deleted.
[0024] Furthermore, in this invention, the constraint search includes: maximizing the expected task quality within the budget constraint B, or minimizing the predicted execution cost within the task quality lower limit δ; when multiple candidate execution plans simultaneously satisfy the constraints, selecting the target execution plan based on the comprehensive objective function composed of quality benefits and cost penalties.
[0025] Furthermore, in this invention, step S5 also includes multi-stage progressive execution: first, a low-cost execution plan is used to quickly analyze the sample to obtain preliminary results and confidence or anomaly scores; when the confidence reaches a threshold or the sample does not belong to a key sample, the first-stage results are directly output; when the confidence is lower than the threshold or the sample belongs to a key sample, a high-precision execution plan is invoked to perform a second-stage fine analysis, and the threshold is dynamically adjusted based on execution feedback.
[0026] This invention also provides a query optimization system based on a general time series framework to implement the above method, including a request parsing module, a candidate plan generation module, a cost estimation module, a quality assessment module, a plan search and pruning module, an execution scheduling module, and a feedback update module; wherein, the request parsing module is used to perform the above step S1, the candidate plan generation module is used to perform step S2, the cost estimation module and the quality assessment module are used to perform step S3, the plan search and pruning module is used to perform step S4, and the execution scheduling module and the feedback update module are used to perform step S5.
[0027] Compared with the prior art, the present invention has the following beneficial effects:
[0028] (1) This invention transforms the general time series analysis process from a fixed pipeline into a set of composable, comparable and searchable execution plans, enabling the optimizer to adaptively select encoder subsets, fusion operators and task models based on task type, input data characteristics and resource constraints.
[0029] (2) This invention evaluates candidate execution plans by combining cost model and quality model. Instead of simply pursuing the lowest execution cost or blindly calling all encoders, it balances task effectiveness and execution efficiency, thereby improving the controllability and stability of the system in multi-task scenarios.
[0030] (3) This invention compresses the candidate plan space through Pareto pruning and budget constraint search, and quickly locates the better execution path while avoiding excessive exhaustive search overhead, thereby reducing the optimizer's own overhead.
[0031] (4) The present invention uses a multi-stage progressive execution mechanism to adopt a lightweight path for low-risk or high-confidence samples and a high-precision path for critical or uncertain samples, which helps to reduce model inference overhead, P95 latency and end-to-end execution time. Attached Figure Description
[0032] Figure 1 This is a schematic diagram of the general time series analysis execution process after introducing a query optimizer in an embodiment of the present invention.
[0033] Figure 2This is a diagram illustrating the overall architecture of a general time series framework with a query optimizer added in this embodiment of the invention.
[0034] Figure 3 This is a schematic diagram of the multi-stage progressive execution strategy process in an embodiment of the present invention. Detailed Implementation
[0035] The present invention will be further described below with reference to the accompanying drawings and embodiments. The embodiments of the present invention include, but are not limited to, the following embodiments.
[0036] This invention discloses a query optimization method for a general time series analysis framework. This method can be deployed on a general time series analysis platform, a time series database analysis engine, a model service platform, or an edge-side intelligent analysis system. It is used to automatically select appropriate representation encoders, fusion methods, and task models in tasks such as classification, clustering, prediction, anomaly detection, and missing value imputation.
[0037] like Figure 1 As shown, the overall execution flow of this embodiment includes a time series analysis request, a candidate encoder set, a user task request, a query optimizer, candidate fusion strategies, an execution plan generation, and downstream task output. First, the time series data to be analyzed, task type, budget constraints, task quality lower limit, and current system resource status are parsed from the time series analysis request. The candidate encoder set may include Transformer-based encoders, convolutional or TCN-based encoders, and encoders based on self-supervised contrastive learning; the user task request may include task type, input data features, latency budget, and resource constraints; the query optimizer evaluates candidate execution plans based on cost and quality models, automatically selecting an encoder subset and fusion strategy; candidate fusion strategies may include splicing fusion, projection fusion, weighted fusion, and gated fusion; the final generated execution plan determines the encoder subset, fusion strategy, and downstream task model, and outputs downstream task results such as classification, prediction, or anomaly detection.
[0038] Specifically, a time series analysis request is represented as Q=<X,T,B,δ,R> Here, X represents the input time series data, T represents the task type, B represents the budget constraint, δ represents the task quality lower bound, and R represents the current system resource status. The task type includes at least one of time series classification, clustering, prediction, anomaly detection, and missing value imputation. The input time series data X can be represented as X∈R^{N×D×L}, where N represents the number of samples, D represents the variable dimension, and L represents the sequence length. The task type T can include classification, clustering, prediction, anomaly detection, and missing value imputation. The budget constraint B can include the maximum allowable inference latency, end-to-end execution time, memory usage, CPU / GPU load, or energy consumption limit. The task quality lower bound δ can be set according to the task type as accuracy retention rate, AUC lower bound, prediction error upper bound, anomaly detection recall rate, or imputation error upper bound. The system resource status R can include available memory, current load, batch size, execution device type, and model cache status.
[0039] In the candidate plan generation phase, let the set of pre-trained representation encoders be E={e1,e2,...,em}, the set of feature fusion operators be Φ={φ1,φ2,...,φn}, and the set of task models be G. T ={g1,g2,...,gk}. The optimizer selects from E, Φ, and G based on task type and input data characteristics. T The candidate execution plan P is generated by combining the middle part.<S,φ,gT,θ> Where S⊆E represents the selected subset of encoders; φ can be concatenation fusion, projection fusion, weighted fusion, or gated fusion; gT is the task model matching the task type; θ is the batch size, accuracy mode, threshold, or device selection, etc. This step first determines the set of available task models based on the task type, and then determines the set of available representation encoders based on the number of samples, sequence length, variable dimensions, missing proportion, and data window size of the input time series data; multiple encoder subsets are enumerated or sampled, and each encoder subset is combined with at least one feature fusion operator from concatenation fusion, projection fusion, weighted fusion, or gated fusion; the combination result is bound to the corresponding task model and operating parameters to obtain a set of candidate execution plans; candidate execution plans that do not match the task type, exceed resource hard constraints, or lack necessary input dimensions are eliminated.
[0040] like Figure 2As shown, this invention can be deployed in a general time series analysis system that includes user input, query optimization, execution control, UniTS encoder, task training and fine-tuning, and result output. The system receives the task type, dataset, candidate model, fusion method, budget constraints, and quality requirements submitted by the user. The query optimization module completes context construction, candidate plan generation, cost model and quality model evaluation, and plan selection. Then, the execution controller schedules the encoder module, feature fusion module, and task training / inference module to complete the downstream tasks.
[0041] During the cost estimation phase, the optimizer estimates the total cost C(P,Q) for each candidate execution plan P. One possible implementation is as follows:
[0042]
[0043] in This represents the encoder inference cost. Indicates the cost of feature fusion. This represents the execution cost of the task model. This represents the cost of moving data between memory, CPU, and GPU. This represents the optimizer's own overhead. The encoder inference cost can be estimated by the model parameter size, FLOPs, input sequence length, variable dimension, batch size, and hardware status; the feature fusion cost can be estimated by the number of encoder outputs, output dimension, and fusion operator complexity; and the task model execution cost can be estimated by the task output space, model structure, and running accuracy mode.
[0044] During the quality assessment phase, the optimizer estimates the expected quality of the candidate execution plan P. The quality assessment includes overall representation quality, fusion quality, and task adaptation quality. Overall representation quality can be determined based on the encoder's performance on historical tasks, the similarity between the current input data features and the distribution of pre-training data, the missing value ratio, and sequence length adaptation. Fusion quality can be determined based on the complementarity, correlation, and redundancy between encoder outputs. Task adaptation quality can be determined based on historical statistics and online feedback of indicators such as classification accuracy, AUC, prediction error, anomaly detection F1 score, or missing value imputation error.
[0045] During the plan search and pruning phase, candidate execution plans form a Pareto front in the cost-quality two-dimensional space. The optimizer first performs feasibility filtering through hard constraints, such as deleting candidate plans that exceed the memory limit, exceed the maximum latency, or are incompatible with the task type; then, Pareto pruning is performed. If a candidate plan Pa has a cost no higher than Pb and a quality no lower than Pb, and at least one metric is strictly better than Pb, then Pa is considered to dominate Pb, and Pb is deleted. This step compresses the candidate space into a set of non-dominated plans.
[0046] During the budget constraint search phase, the optimizer can adopt two objective forms: First, maximize the expected task quality within the budget constraint B, i.e., P*=argmax Qlt(P,Q), satisfying C(P,Q)≤B; Second, minimize the predicted execution cost within the task quality lower bound δ, i.e., P*=argmin C(P,Q), satisfying Qlt(P,Q)≥δ. When multiple plans simultaneously meet the conditions, they can also be ranked using the comprehensive objective Score(P)=Qlt(P,Q)-λ·C(P,Q), where λ is the trade-off parameter between quality and cost.
[0047] During the execution scheduling phase, the optimizer sends the target execution plan to the execution engine. The execution engine performs representation extraction according to the selected encoder subset, feature fusion according to the selected fusion operator, and calls the corresponding task model to output the analysis results. During actual execution, feedback information such as end-to-end time, encoder inference time, peak memory usage, task metrics, and sample confidence scores is recorded. This feedback information is used to update cost estimation parameters, quality assessment parameters, and threshold strategies, enabling the optimizer to gradually adapt to different datasets, loads, and task scenarios.
[0048] In a preferred embodiment, the present invention further employs a multi-stage progressive execution strategy, such as... Figure 3 As shown. The first stage employs a low-cost execution plan, such as fewer encoders, lightweight fusion operators, or small task models, to quickly analyze batches of samples and generate preliminary results and confidence scores. For classification tasks, the confidence score can be the maximum class probability or the class margin; for anomaly detection tasks, the confidence score can be determined by the distance between the anomaly score and a threshold; for prediction or imputation tasks, the confidence score can be determined by prediction uncertainty or reconstruction error.
[0049] When the results of the first stage meet the preset threshold τ, the system directly outputs the first stage results. When the results of the first stage do not meet the threshold τ, or when the sample is identified as a critical sample, uncertain sample, or high-risk sample, the system invokes the second-stage high-precision execution plan. The second stage can use more representation encoders, more complex fusion operators, or higher-precision task models to improve the analysis quality of critical samples. The threshold τ can be dynamically adjusted based on the current budget utilization, historical task quality, sample difficulty, and resource status.
[0050] In terms of system implementation, the query optimization system corresponding to this invention includes a request parsing module, a candidate plan generation module, a cost estimation module, a quality assessment module, a plan search and pruning module, an execution scheduling module, and a feedback update module. The request parsing module extracts input data features, task types, and constraints; the candidate plan generation module generates a set of execution plans; the cost estimation and quality assessment modules assign comparable quantitative indicators to the candidate execution plans; the plan search and pruning module outputs the target execution plan; the execution scheduling module invokes the encoder, fusion operator, and task model; and the feedback update module updates model parameters and thresholds based on actual execution results.
[0051] In practical applications, this invention can be applied to scenarios such as anomaly detection in industrial equipment condition monitoring, fault identification in server operation and maintenance logs, financial time series forecasting, missing value imputation in IoT sensor data, and traffic flow prediction. When the system is in an offline analysis scenario, the quality weight can be increased to select a higher quality execution plan; when the system is in an online service or edge deployment scenario, the cost weight can be increased to prioritize low-latency plans that meet the quality minimum.
[0052] In summary, this invention introduces database query optimization concepts into a general time series analysis framework. By jointly modeling and searching encoder selection, fusion strategies, and task model configuration before execution, it achieves a transformation from a static analysis pipeline to a dynamic adaptive execution plan. This method can reduce redundant inference and resource waste while ensuring analysis quality, thus improving the engineering practicality of general time series analysis systems.
[0053] The above embodiments are merely one of the preferred embodiments of the present invention and should not be used to limit the scope of protection of the present invention. Any modifications or refinements made to the main design concept and spirit of the present invention that are not of substantial significance, but solve the same technical problem as the present invention, should be included within the scope of protection of the present invention.
Claims
1. A query optimization method based on a general time series framework, characterized in that, Includes the following steps: S1, receive a time series analysis request, and parse the time series data to be analyzed, task type, budget constraint, task quality lower limit and current system resource status from the time series analysis request; S2, generate multiple candidate execution plans based on the pre-trained representation encoder set, feature fusion operator set, and task model set, wherein each candidate execution plan includes at least a selected subset of representation encoders, feature fusion operators, task models, and running parameters; S3, perform cost estimation and quality assessment for each candidate execution plan to obtain the predicted execution cost and expected task quality of the candidate execution plan; S4. Based on the predicted execution cost, expected task quality, budget constraints, and task quality lower limit, perform feasibility filtering, Pareto pruning, and constraint search on multiple candidate execution plans to determine the target execution plan. S5. Execute the corresponding time series analysis task according to the target execution plan, and update the cost estimation parameters, quality assessment parameters, and execution strategy parameters based on the actual execution results.
2. The query optimization method based on a general time series framework according to claim 1, characterized in that, The time series analysis request is represented as Q=<X,T,B,δ,R> Where X represents the input time series data, T represents the task type, B represents the budget constraint, δ represents the task quality lower limit, and R represents the current system resource status; the task type includes at least one of time series classification, clustering, prediction, anomaly detection, and missing value imputation.
3. The query optimization method based on a general time series framework according to claim 1, characterized in that, The candidate execution plan is represented as P=<S,φ,gT,θ> , where S represents a subset of encoders selected from the pre-trained representation encoder set, φ represents a feature fusion operator, gT represents a task model corresponding to task type T, and θ represents at least one of the following operating parameters: batch size, threshold, model accuracy mode, or execution device.
4. The query optimization method based on a general time series framework according to claim 1, characterized in that, Step S2 includes the following steps: S21. Determine the set of available task models based on the task type, and determine the set of available representation encoders based on the number of samples, sequence length, variable dimension, missing proportion, and data window size of the input time series data. S22, enumerate or sample to obtain multiple encoder subsets, and combine each encoder subset with at least one feature fusion operator among splicing fusion, projection fusion, weighted fusion or gated fusion; S23, bind the combined results with the corresponding task model and running parameters to obtain a set of candidate execution plans; S24, eliminate candidate execution plans that do not match the task type, exceed resource hard constraints, or lack necessary input dimensions.
5. The query optimization method based on a general time series framework according to claim 1, characterized in that, The cost estimation includes encoder inference cost, feature fusion cost, task model execution cost, data transmission cost, and optimizer overhead. The encoder inference cost is determined by encoder parameter size, floating-point computation, input sequence length, variable dimension, batch size, and operating hardware status. The feature fusion cost is determined by the selected encoder output dimension, fusion operator type, and sample batch size. The task model execution cost is determined by the task model structure, output space size, and operating parameters.
6. The query optimization method based on a general time series framework according to claim 1, characterized in that, The quality assessment includes representation quality assessment, fusion quality assessment, and task adaptation quality assessment. The quality assessment measures how well a selected subset of encoders fits the current task and input data features. The fusion quality assessment measures the complementarity and redundancy between the output representations of different encoders. The task fit quality assessment measures the expected accuracy, AUC, prediction error, anomaly detection performance, or imputation error of the task model under the current execution plan.
7. The query optimization method based on a general time series framework according to claim 1, characterized in that, The non-dominated pruning is Pareto pruning; when the predicted execution cost of the first candidate execution plan is not higher than that of the second candidate execution plan and the expected task quality is not lower than that of the second candidate execution plan, and at least one metric is significantly better than that of the second candidate execution plan, the second candidate execution plan is deleted.
8. The query optimization method based on a general time series framework according to claim 1, characterized in that, The constraint search includes: maximizing the expected task quality within the budget constraint B, or minimizing the predicted execution cost within the task quality lower limit δ; when multiple candidate execution plans simultaneously satisfy the constraints, the target execution plan is selected based on the comprehensive objective function composed of quality benefits and cost penalties.
9. The query optimization method based on a general time series framework according to claim 1, characterized in that, Step S5 further includes multi-stage progressive execution: first, a low-cost execution plan is used to quickly analyze the sample to obtain preliminary results and confidence or anomaly scores; when the confidence reaches a threshold or the sample does not belong to a key sample, the first-stage results are directly output; when the confidence is lower than the threshold or the sample belongs to a key sample, a high-precision execution plan is invoked to perform a second-stage fine analysis, and the threshold is dynamically adjusted based on execution feedback.
10. A query optimization system based on a general time series framework, characterized in that, It includes a request parsing module, a candidate plan generation module, a cost estimation module, a quality assessment module, a plan search and pruning module, an execution scheduling module, and a feedback update module; wherein, the request parsing module is used to execute step S1 in claim 1, the candidate plan generation module is used to execute step S2, the cost estimation module and the quality assessment module are used to execute step S3, the plan search and pruning module is used to execute step S4, and the execution scheduling module and the feedback update module are used to execute step S5.