Automatic time sequence feature generation system and method for large model prompt project

Abstract

The invention discloses a time sequence feature automatic generation system and method oriented to a large model prompt project, belongs to the technical field of crossing of a time sequence database and artificial intelligence, and aims to solve the technical problem of how to eliminate a traditional ETL link, remarkably reduce end-to-end reasoning time delay and guarantee time consistency of features and original data. According to the technical scheme, the prompt word template compiler is used for analyzing a declarative Prompt template into a continuous query execution plan; the continuous query execution engine is used for triggering a continuous query execution plan on a time sequence database write path and completing sliding window feature calculation; the feature semantic mapper is used for converting the numerical features into semantic tokens; the prompt context table is used for caching the semantic token and the corresponding time range; and the Prompt splicing service is used for reading the semantic token from the prompt context table according to the time range of the external reasoning request and splicing the semantic token into a complete Prompt.

Description

Technical Field

[0001] This invention relates to the field of time series databases and artificial intelligence, specifically to an automatic generation system and method for time series features in large model prompting engineering. Background Technology

[0002] With the development of the Industrial Internet and the Internet of Things, the scale of time-series data is growing exponentially. To leverage large models for intelligent analysis of time-series data, existing technologies typically employ the following process:

[0003] ① Export the raw sequences from the TSDB (Time Series Database) using offline ETL;

[0004] ② Perform statistical, frequency domain, or deep learning feature extraction at the independent feature calculation node;

[0005] ③ Textify the features and then append them to the Prompt template;

[0006] ④ Call the large model API for inference.

[0007] The above process suffers from several drawbacks, including the separation of feature calculation and data storage, resulting in high latency, poor consistency, repeated ETL consuming additional computing and network resources, the need for real-time querying and formatting of Prompt concatenation adding millisecond to second-level waiting time, and the lack of an incremental update mechanism for high-frequency sliding window scenarios.

[0008] Therefore, how to eliminate the traditional ETL link, significantly reduce end-to-end inference latency, and ensure the time consistency between features and original data is a technical problem that urgently needs to be solved. Summary of the Invention

[0009] The technical objective of this invention is to provide an automatic temporal feature generation system and method for large model hinting engineering, in order to solve the problems of how to eliminate traditional ETL links, significantly reduce end-to-end inference latency, and ensure the temporal consistency between features and original data.

[0010] The technical objective of this invention is achieved as follows: an automatic generation system for temporal features in large-scale model-based engineering, comprising:

[0011] A prompt template compiler is used to parse declarative prompt templates into continuous query (CQ) execution plans;

[0012] The continuous query execution engine is used to trigger continuous query execution plans on the write path of the time series database (TSDB) to complete the sliding window feature calculation;

[0013] Feature semantic mapper, used to convert numerical features into semantic tokens;

[0014] The Prompt Context Table (PCT) is used to cache semantic tokens and their corresponding time ranges.

[0015] The Prompt concatenation service is used to read semantic tokens from the prompt context table based on the time range of the external inference request and concatenate them into a complete Prompt.

[0016] As a preferred option, the Prompt template is described using a declarative DSL (Domain-Specific Language), and its syntax elements include a time range placeholder {{window}}, a feature function placeholder {{agg(func,field)}}, and a text format placeholder {{format}}.

[0017] Among them, func supports mean, std, fft_top3 and ks_score;

[0018] The prompt word template compiler parses the DSL into an abstract syntax tree (AST) and generates the corresponding sequential query execution plan;

[0019] The abstract syntax tree plan nodes include sliding window operators, feature operators, textification operators, and output format operators;

[0020] The sliding window operator uses TSDB's native PIVOT RANGE EVERY semantics, and the window width and step size are parameterized by {{window}};

[0021] The feature operator calls the embedded User-Defined Aggregate Function (UDAF) library, supporting incremental computation;

[0022] The textualization operator maps numerical features to semantic tokens through a pluggable tokenizer, for example, mapping ks_score>0.73 to “significant drift”;

[0023] Output format operator: Generates key-value pairs {timestamp:_,features:_,semantic:_} that conform to the Prompt syntax.

[0024] Even better, the continuous query execution engine reuses the continuous query framework of the time-series database and adds the following extensions:

[0025] Write-Time Trigger: Immediately after a successful WAL commit and before the MemTable is flushed to disk, continuous query calculations are triggered to ensure that the feature visibility time is consistent with the original data.

[0026] Incremental state management: The CombineFn interface is used to maintain the intermediate results of the sliding window, and the state is stored in the StateStore of the time series database (based on LSM Tree), supporting fault recovery;

[0027] Concurrency control: The SeqLock mechanism is introduced to ensure that feature calculations are performed only once within the same window on the same timeline, thus avoiding duplication.

[0028] More specifically, the continuous query execution engine triggering process is as follows:

[0029] (1) Device-side initiation point: The device submits a time-series record with device ID, timestamp, and sample value to the TSDB write thread via MQTT / CoAP or local Socket, which is encapsulated as a DataPoint structure;

[0030] (2) Write thread receiving entry: The time series database write thread first appends the DataPoint structure to the currently active MemTable, and at the same time assigns a globally monotonically increasing log sequence number LSNn to the DataPoint structure to ensure that all subsequent derived data can be traced back to the same atomic point;

[0031] (3) WAL persistence: The write thread encodes the LSNn and DataPoint structure together into an 8-byte WAL record header + variable-length payload, and writes it to the WAL log of NOR Flash through the memory-mapped file interface (mmap); after successful flushing, the WAL log returns ACK to the write thread, at which point the data has crash-safe capability;

[0032] (4) Write-Time Hook Activation: The moment the WAL log returns a success, the time-series database calls the pre-embedded write_time_hook(LSNn) and immediately transfers the control flow to the Continuous Query Engine (CQ Engine); write_time_hook(LSNn) runs in user-mode interrupt context and must take less than 1ms, otherwise it will block subsequent writes;

[0033] (5) CQ plan parsing: The continuous query execution engine quickly locates the sliding window (window-id) into which the current write falls based on the pre-compiled AST, and checks whether the sliding window into which the current write falls already has an intermediate state (state-handle): if the intermediate state is empty, the aggregation buffer is initialized; if the intermediate state is not empty, it is reused directly to avoid reallocating memory.

[0034] (6) Feature UDAF calculation: The continuous query execution engine calls the embedded sliding window UDAF (user-defined aggregation function), such as mean(), std(), fft_top3(), etc., to perform incremental calculation on the numerical columns in the embedded sliding window UDAF; among them, the embedded sliding window UDAF uses SIMD instructions, and the time taken for 1000 points in a single window is about 90μs, and the result output is a 24B numerical feature vector;

[0035] (7) Semantic mapping: Numerical feature vectors are fed into the feature semantic mapper (Tokenizer). The feature semantic mapper maps floating-point features into semantic tokens (e.g., "significant drift") through a pre-set quantile bucket table, and then concatenates them into a prompt_kv object in Protocol Buffer format, with the size controlled within 256B, so as to facilitate the direct embedding of large model prompts in the future.

[0036] (8) Prompt context table insertion: The continuous query execution engine inserts the prompt_kv object along with LSNn and window start and end time as the primary key into the prompt context table. The prompt context table shares the same LSM-Tree with the original data table and uses an independent column family. Therefore, there is no extra disk flushing in the write path, and only one MemTable record needs to be appended.

[0037] (9) Completion notification: After the context table is successfully written, return SUCCESS to the continuous query execution engine. The continuous query execution engine then calls back cq_done(LSNn) to notify the writing thread that the derived data is now visible and the 4kB buffer occupied by this hook can be released.

[0038] (10) Response feedback: After receiving the notification, the write thread sends a WRITE_OK packet back to the device. Thus, the entire “write data → calculate features → cache prompt” chain is completed in a single system call.

[0039] More preferably, the feature semantic mapper incorporates a lightweight large model tokenizer (vocabulary ≤ 32k), loaded via memory mapping; the mapping process is as follows:

[0040] For continuous value features, quantile bucket encoding is used, and the bucket boundaries are obtained through offline statistics and cached in the TSDB metadata table;

[0041] For discrete features, dictionary mapping can be used directly;

[0042] For anomaly markers, the is_anomaly boolean value output by the drift detection UDAF is used and mapped to a "normal / abnormal" token;

[0043] The mapping results are written to the prompt context table in UTF-8 byte stream format, saving 30% of storage;

[0044] The context table is a system table within TSDB, with the schema: (timeline_id uint64, window_start timestamp, window_end timestamp, prompt_kv blob, ttl uint32). Here, prompt_kv is a Protocol Buffer-encoded key-value pair containing three fields: raw_features, semantic_tokens, and unit. The context table uses the same partitioning strategy as the data table, performing hash sharding based on timeline_id to ensure locality. Simultaneously, ttl is set to window_width * 2, allowing the time-series database to automatically reclaim expired contexts and prevent unlimited expansion.

[0045] More preferably, the Prompt concatenation service call process is as follows:

[0046] (1) Interface exposure: The Prompt splicing service exposes the standard gRPC method rpcGeneratePrompt(PromptRequest)returns(PromptReply) through prompt.proto; the transport layer uses HTTP / 2+TLS, port 443, and connection multiplexing keeps-alive for 300 seconds;

[0047] (2) Request parsing and validity verification, as detailed below:

[0048] ① After receiving the PromptRequest, first deduplicate and sort the timeline_id_list, limiting it to a maximum of 64 timelines;

[0049] ② Verify that query_start ≤ query_end and the interval ≤ 1h;

[0050] ③ Load the template AST from the memory dictionary based on prompt_template_id. If it does not exist, immediately return INVALID_ARGUMENT;

[0051] (3) Contextual Index Location: Use the combined index of "timeline + window start and end" (LSM-Tree pushdown) to generate a range scan plan:

[0052] SELECT prompt_kv,crc32 FROM pct

[0053] WHERE timeline_id IN(...)

[0054] AND window_start>=:query_start

[0055] AND window_end <= query_end

[0056] ORDER BY window_start ASC;

[0057] The corresponding query only accesses the column family of the context table and does not retrieve the original data table;

[0058] (4) Result expansion and template filling: Each returned prompt_kv is decoded into a triple {timestamp,features[],semantic_tokens[]} according to Protocol Buffer; and expanded in the order of timestamp and concatenated into a continuous text segment; the text segment is inserted into the user's Prompt template slot to generate a complete Prompt string;

[0059] (5) Correctness Guarantee (CRC32 Fast Check): Calculate the CRC32-C checksum for the complete Prompt string; and compare it with the accumulated CRC32 value of each record in the prompt context table query results:

[0060] ① If they match, return the Prompt text directly;

[0061] ② If inconsistent, trigger "rollback query": re-execute the context index positioning in step (3) and enable consistent read (read the latest LSN). If the second comparison still fails, return the DATA_INCONSISTENCY error code and the caller will retry.

[0062] (6) Output and Link Statistics: The final Prompt text is encapsulated into PromptReply, which also carries the actual interval, number of feature lines, and time consumption μs; it is returned to the caller without accessing the original time series data. The average latency is <5ms and P99 is <10ms.

[0063] An automatic generation method for temporal features in large-scale model-based engineering is proposed, as detailed below:

[0064] The prompt template compiler parses the prompt template into a continuous query (CQ) execution plan in the Time Series Database (TSDB) kernel.

[0065] On the data ingestion write path, feature extraction, semantic alignment, and context caching are synchronously completed through a feature semantic mapper, and the results are written in real time to a dedicated Prompt Context Table (PCT). The Prompt Context Table is used to cache semantic feature text and its corresponding time range.

[0066] When an external large model inference request arrives, the semantic feature text is read from the prompt context table according to the time range of the external large model push request and the Prompt template is completed.

[0067] As a preferred option, the Prompt template is described using a declarative DSL (Domain-Specific Language), and its syntax elements include a time range placeholder {{window}}, a feature function placeholder {{agg(func,field)}}, and a text format placeholder {{format}}.

[0068] Among them, func supports mean, std, fft_top3 and ks_score;

[0069] The prompt word template compiler parses the DSL into an abstract syntax tree (AST) and generates the corresponding sequential query execution plan;

[0070] The abstract syntax tree plan nodes include sliding window operators, feature operators, textification operators, and output format operators;

[0071] The sliding window operator adopts the native PIVOT RANGE EVERY semantics of time series databases, and the window width and step size are parameterized by {{window}};

[0072] The feature operator calls the embedded User-Defined Aggregate Function (UDAF) library, supporting incremental computation;

[0073] The textualization operator maps numerical features to semantic tokens through a pluggable tokenizer, for example, mapping ks_score>0.73 to “significant drift”;

[0074] Output format operator: Generates key-value pairs {timestamp:_,features:_,semantic:_} that conform to the Prompt syntax.

[0075] More preferably, the prompt context table is a system table in the time series database, with the schema: (timeline_iduint64, window_start timestamp, window_end timestamp, prompt_kv blob, ttluint32);

[0076] Among them, prompt_kv is a key-value pair encoded by Protocol Buffer, containing three fields: raw_features, semantic_tokens, and unit;

[0077] The context table uses the same partitioning strategy as the data table, performing hash sharding based on timeline_id to ensure locality; at the same time, ttl=window_width*2 is set so that the time series database can automatically reclaim expired contexts to avoid infinite expansion.

[0078] Better yet, the Prompt splicing service is exposed via a gRPC interface, as follows:

[0079] Receive external inference requests with parameters timeline_id_list, query_start, query_end, and prompt_template_id;

[0080] Based on the time range, execute the following query in the prompt context table: SELECT prompt_kv WHERE timeline_id IN (...) AND window_start >= query_start AND window_end <= query_end;

[0081] The returned prompt_kv is expanded in chronological order and inserted into the user-provided Prompt template slot;

[0082] Calculate the CRC32 checksum and compare it with the checksum cached in the context table:

[0083] If they match, return immediately;

[0084] If there is a discrepancy, a rollback query is triggered to ensure correctness;

[0085] Outputs a complete Prompt text for LLM inference, without needing to access the original data throughout the process, with an average latency of <5ms (P99 <10ms);

[0086] This method also includes end-to-end consistency guarantees, as detailed below:

[0087] Linear consistency: The continuous query execution engine performs calculations in a context where the WAL sequence number is monotonically increasing, and the feature results share the same log_sequence_id with the original data;

[0088] Atomic rollback: If writing to the context table fails, the continuous query execution engine throws a RetryableException, and the time-series data is automatically replayed to replay the corresponding batch of data;

[0089] Multi-replica synchronization: The context table is used as a system table and replicated to the majority of nodes via the Raft protocol to ensure that Prompt splicing can continue after a failover.

[0090] The automatic generation system and method for temporal features in large-model hinting engineering of the present invention has the following advantages:

[0091] (i) This invention compiles the "prompt word template" into a continuous query (CQ) in the kernel of a time series database (TSDB). Feature extraction, semantic alignment and context caching are completed synchronously on the write path of data entry into the database, and the results are written to a dedicated prompt context table (PCT) in real time. When an external large model inference request arrives, it is only necessary to concatenate the feature text already cached in the PCT according to the request time range to achieve zero-latency assembly of the prompt, eliminating the traditional ETL link, significantly reducing end-to-end inference latency, and ensuring the time consistency between features and original data.

[0092] (ii) This invention compiles the Prompt template into CQ, completes feature calculation and caching on the write path, achieves zero-latency Prompt splicing, strong consistency between features and original data, no need for external ETL, reduces resource consumption, and supports high-concurrency, high-throughput online inference scenarios.

[0093] (III) This invention achieves resource optimization; specifically: ① CPU: The feature UDAF adopts the SIMD instruction set (AVX-512), reducing the computation time by 42%; ② Memory: The incremental window state uses RoaringBitmap compression, with a single timeline memory usage of <2KB; ③ Disk: PCT enables TSDB's ZSTD block compression, with a compression rate of 55%, and shares block cache with the data table to avoid additional I / O;

[0094] (iv) This invention has good scalability, specifically: ① Horizontal scaling: The CQ execution plan can be pushed down to the TSDB storage node, corresponding one-to-one with the data shards, realizing "computation follows the data"; ② Hot template updating: The compiler supports online template replacement, which can be completed through the ALTER CONTINUOUS QUERY command without restarting the process; ③ Multi-model support: The semantic mapper is pluggable, and new tokenizers can be registered through CREATE MAPPING FUNCTION to adapt to different large model vocabularies;

[0095] (v) The present invention also has the following effects

[0096] ① End-to-end latency reduced by 10×: Prompt stitching reduced from an average of 800ms to 5ms;

[0097] ② 35% resource savings: Eliminating the offline ETL cluster significantly reduces CPU and network bandwidth;

[0098] ③ Strong consistency: Features and original data are generated within the same transaction boundary, avoiding "feature drift";

[0099] ④ High Availability: PCT uses Raft replication, enabling second-level failover in case of node failure;

[0100] ⑤ Easy to extend: Templates and mapping functions can be upgraded as plug-ins without modifying the kernel code. Attached Figure Description

[0101] The invention will be further described below with reference to the accompanying drawings.

[0102] Appendix Figure 1 A schematic diagram of an automatic generation system for temporal features in large-scale model-oriented engineering projects;

[0103] Appendix Figure 2 The sequence diagram for triggering the continuous query execution engine;

[0104] Appendix Figure 3 Construct a service call sequence diagram for Prompt. Detailed Implementation

[0105] The automatic generation system and method for temporal features of large model prompting projects according to the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0106] Example 1:

[0107] As attached Figure 1 As shown, this embodiment provides an automatic generation system for temporal features in large model hinting projects. The system includes:

[0108] A prompt template compiler is used to parse declarative prompt templates into continuous query (CQ) execution plans;

[0109] The continuous query execution engine is used to trigger continuous query execution plans on the write path of the time series database (TSDB) to complete the sliding window feature calculation;

[0110] Feature semantic mapper, used to convert numerical features into semantic tokens;

[0111] The Prompt Context Table (PCT) is used to cache semantic tokens and their corresponding time ranges.

[0112] The Prompt concatenation service is used to read semantic tokens from the prompt context table based on the time range of the external inference request and concatenate them into a complete Prompt.

[0113] In this embodiment, the Prompt template is described using a declarative DSL (Domain-Specific Language), and its syntax elements include a time range placeholder {{window}}, a feature function placeholder {{agg(func,field)}}, and a text format placeholder {{format}}.

[0114] Among them, func supports mean, std, fft_top3 and ks_score.

[0115] In this embodiment, the prompt word template compiler parses the DSL into an abstract syntax tree (AST) and generates a corresponding sequential query execution plan;

[0116] The abstract syntax tree plan nodes include sliding window operators, feature operators, textification operators, and output format operators;

[0117] The sliding window operator uses TSDB's native PIVOT RANGE EVERY semantics, and the window width and step size are parameterized by {{window}};

[0118] The feature operator calls the embedded User-Defined Aggregate Function (UDAF) library, supporting incremental computation;

[0119] The textualization operator maps numerical features to semantic tokens through a pluggable tokenizer, for example, mapping ks_score>0.73 to “significant drift”;

[0120] Output format operator: Generates key-value pairs {timestamp:_,features:_,semantic:_} that conform to the Prompt syntax.

[0121] In this embodiment, the continuous query execution engine reuses the continuous query framework of the time-series database and adds the following extensions:

[0122] Write-Time Trigger: Immediately after a successful WAL commit and before the MemTable is flushed to disk, continuous query calculations are triggered to ensure that the feature visibility time is consistent with the original data.

[0123] Incremental state management: The CombineFn interface is used to maintain the intermediate results of the sliding window, and the state is stored in the StateStore of the time series database (based on LSM Tree), supporting fault recovery;

[0124] Concurrency control: The SeqLock mechanism is introduced to ensure that feature calculations are performed only once within the same window on the same timeline, thus avoiding duplication.

[0125] As attached Figure 2 As shown, the specific triggering process of the continuous query execution engine in this embodiment is as follows:

[0126] (1) Device-side initiation point: The device submits a time-series record with device ID, timestamp, and sample value to the TSDB write thread via MQTT / CoAP or local Socket, which is encapsulated as a DataPoint structure;

[0127] (2) Write thread receiving entry: The time series database write thread first appends the DataPoint structure to the currently active MemTable, and at the same time assigns a globally monotonically increasing log sequence number LSNn to the DataPoint structure to ensure that all subsequent derived data can be traced back to the same atomic point;

[0128] (3) WAL persistence: The write thread encodes the LSNn and DataPoint structure together into an 8-byte WAL record header + variable-length payload, and writes it to the WAL log of NOR Flash through the memory-mapped file interface (mmap); after successful flushing, the WAL log returns ACK to the write thread, at which point the data has crash-safe capability;

[0129] (4) Write-Time Hook Activation: The moment the WAL log returns a success, the time-series database calls the pre-embedded write_time_hook(LSNn) and immediately transfers the control flow to the Continuous Query Engine (CQ Engine); write_time_hook(LSNn) runs in user-mode interrupt context and must take less than 1ms, otherwise it will block subsequent writes;

[0130] (5) CQ plan parsing: The continuous query execution engine quickly locates the sliding window (window-id) into which the current write falls based on the pre-compiled AST, and checks whether the sliding window into which the current write falls already has an intermediate state (state-handle): if the intermediate state is empty, the aggregation buffer is initialized; if the intermediate state is not empty, it is reused directly to avoid reallocating memory.

[0131] (6) Feature UDAF calculation: The continuous query execution engine calls the embedded sliding window UDAF (user-defined aggregation function), such as mean(), std(), fft_top3(), etc., to perform incremental calculation on the numerical columns in the embedded sliding window UDAF; among them, the embedded sliding window UDAF uses SIMD instructions, and the time taken for 1000 points in a single window is about 90μs, and the result output is a 24B numerical feature vector;

[0132] (7) Semantic mapping: Numerical feature vectors are fed into the feature semantic mapper (Tokenizer). The feature semantic mapper maps floating-point features into semantic tokens (e.g., "significant drift") through a pre-set quantile bucket table, and then concatenates them into a prompt_kv object in Protocol Buffer format, with the size controlled within 256B, so as to facilitate the direct embedding of large model prompts in the future.

[0133] (8) Prompt context table insertion: The continuous query execution engine inserts the prompt_kv object along with LSNn and window start and end time as the primary key into the prompt context table. The prompt context table shares the same LSM-Tree with the original data table and uses an independent column family. Therefore, there is no extra disk flushing in the write path, and only one MemTable record needs to be appended.

[0134] (9) Completion notification: After the context table is successfully written, return SUCCESS to the continuous query execution engine. The continuous query execution engine then calls back cq_done(LSNn) to notify the writing thread that the derived data is now visible and the 4kB buffer occupied by this hook can be released.

[0135] (10) Response feedback: After receiving the notification, the write thread sends a WRITE_OK packet back to the device. Thus, the entire “write data → calculate features → cache prompt” chain is completed in a single system call.

[0136] In this embodiment, the feature semantic mapper incorporates a lightweight large model Tokenizer (vocabulary ≤ 32k), which is loaded via memory mapping; the mapping process is as follows:

[0137] For continuous value features, quantile bucket encoding is used, and the bucket boundaries are obtained through offline statistics and cached in the TSDB metadata table;

[0138] For discrete features, dictionary mapping can be used directly;

[0139] For anomaly markers, the is_anomaly boolean value output by the drift detection UDAF is used and mapped to a "normal / abnormal" token;

[0140] The mapping results are written to the prompt context table in UTF-8 byte stream format, saving 30% of storage.

[0141] In this embodiment, the prompt context table is a system table within the TSDB, with the schema: (timeline_iduint64, window_start timestamp, window_end timestamp, prompt_kv blob, ttluint32). Here, prompt_kv is a Protocol Buffer-encoded key-value pair containing three fields: raw_features, semantic_tokens, and unit. The prompt context table uses the same partitioning strategy as the data table, performing hash sharding based on timeline_id to ensure locality. Simultaneously, ttl is set to window_width * 2, allowing the time-series database to automatically reclaim expired contexts, preventing infinite expansion.

[0142] As attached Figure 3 As shown, the Prompt splicing service invocation process in this embodiment is as follows:

[0143] (1) Interface exposure: The Prompt splicing service exposes the standard gRPC method rpcGeneratePrompt(PromptRequest)returns(PromptReply) through prompt.proto; the transport layer uses HTTP / 2+TLS, port 443, and connection multiplexing keeps-alive for 300 seconds;

[0144] (2) Request parsing and validity verification, as detailed below:

[0145] ① After receiving the PromptRequest, first deduplicate and sort the timeline_id_list, limiting it to a maximum of 64 timelines;

[0146] ② Verify that query_start ≤ query_end and the interval ≤ 1h;

[0147] ③ Load the template AST from the memory dictionary based on prompt_template_id. If it does not exist, immediately return INVALID_ARGUMENT;

[0148] (3) Contextual Index Location: Use the combined index of "timeline + window start and end" (LSM-Tree pushdown) to generate a range scan plan:

[0149] SELECT prompt_kv,crc32 FROM pct

[0150] WHERE timeline_id IN(...)

[0151] AND window_start>=:query_start

[0152] AND window_end <= query_end

[0153] ORDER BY window_start ASC;

[0154] The corresponding query only accesses the column family of the context table and does not retrieve the original data table;

[0155] (4) Result expansion and template filling: Each returned prompt_kv is decoded into a triple {timestamp,features[],semantic_tokens[]} according to Protocol Buffer; and expanded in the order of timestamp and concatenated into a continuous text segment; the text segment is inserted into the user's Prompt template slot to generate a complete Prompt string;

[0156] (5) Correctness Guarantee (CRC32 Fast Check): Calculate the CRC32-C checksum for the complete Prompt string; and compare it with the accumulated CRC32 value of each record in the prompt context table query results:

[0157] ① If they match, return the Prompt text directly;

[0158] ② If inconsistent, trigger "rollback query": re-execute the context index positioning in step (3) and enable consistent read (read the latest LSN). If the second comparison still fails, return the DATA_INCONSISTENCY error code and the caller will retry.

[0159] (6) Output and Link Statistics: The final Prompt text is encapsulated into PromptReply, which also carries the actual interval, number of feature lines, and time consumption μs; it is returned to the caller without accessing the original time series data. The average latency is <5ms and P99 is <10ms.

[0160] Example 2:

[0161] This embodiment provides a method for automatically generating time-series features for large-scale model-based projects. The method is as follows:

[0162] S1. The prompt template is parsed into a continuous query (CQ) execution plan in the Time Series Database (TSDB) kernel by the prompt word template compiler;

[0163] S2. On the data ingestion write path, feature extraction, semantic alignment, and context caching are synchronously completed through the feature semantic mapper, and the results are written in real time to a dedicated Prompt Context Table (PCT). The Prompt Context Table is used to cache semantic feature text and its corresponding time range.

[0164] S3. When an external large model inference request arrives, read the semantic feature text from the prompt context table according to the time range of the external large model push request and concatenate it to complete the Prompt template.

[0165] In this embodiment, the Prompt template is described using a declarative DSL (Domain-Specific Language), and its syntax elements include a time range placeholder {{window}}, a feature function placeholder {{agg(func,field)}}, and a text format placeholder {{format}}.

[0166] Among them, func supports mean, std, fft_top3 and ks_score.

[0167] In this embodiment, the prompt word template compiler parses the DSL into an abstract syntax tree (AST) and generates a corresponding sequential query execution plan;

[0168] The abstract syntax tree plan nodes include sliding window operators, feature operators, textification operators, and output format operators;

[0169] The sliding window operator adopts the native PIVOT RANGE EVERY semantics of time series databases, and the window width and step size are parameterized by {{window}};

[0170] The feature operator calls the embedded User-Defined Aggregate Function (UDAF) library, supporting incremental computation;

[0171] The textualization operator maps numerical features to semantic tokens through a pluggable tokenizer, for example, mapping ks_score>0.73 to “significant drift”;

[0172] Output format operator: Generates key-value pairs {timestamp:_,features:_,semantic:_} that conform to the Prompt syntax.

[0173] In this embodiment, the prompt context table is a system table in the time series database, with the schema being: (timeline_id uint64, window_start timestamp, window_end timestamp, prompt_kvblob, ttl uint32);

[0174] Among them, prompt_kv is a key-value pair encoded by Protocol Buffer, containing three fields: raw_features, semantic_tokens, and unit;

[0175] The context table uses the same partitioning strategy as the data table, performing hash sharding based on timeline_id to ensure locality; at the same time, ttl=window_width*2 is set so that the time series database can automatically reclaim expired contexts to avoid infinite expansion.

[0176] In this embodiment, the Prompt splicing service is exposed via a gRPC interface, as detailed below:

[0177] ① Receive external inference requests, with parameters including timeline_id_list, query_start, query_end, and prompt_template_id;

[0178] ② Based on the time range, execute the following query in the prompt context table: SELECT prompt_kv WHERE timeline_id IN (...) AND window_start >= query_start AND window_end <= query_end;

[0179] ③ Expand the returned prompt_kv in chronological order and insert it into the user-provided Prompt template slot;

[0180] ④ Calculate the CRC32 checksum and compare it with the checksum cached in the context table:

[0181] If they match, return immediately;

[0182] If there is a discrepancy, a rollback query is triggered to ensure correctness;

[0183] ⑤ Output a complete Prompt text for LLM inference. The entire process does not require access to the original data, and the average latency is <5ms (P99 <10ms).

[0184] This embodiment also includes end-to-end consistency guarantees, as detailed below:

[0185] Linear consistency: The continuous query execution engine performs calculations in a context where the WAL sequence number is monotonically increasing, and the feature results share the same log_sequence_id with the original data;

[0186] Atomic rollback: If writing to the context table fails, the continuous query execution engine throws a RetryableException, and the time-series data is automatically replayed to replay the corresponding batch of data;

[0187] Multi-replica synchronization: The context table is used as a system table and replicated to the majority of nodes via the Raft protocol to ensure that Prompt splicing can continue after a failover.

[0188] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An automatic generation system for temporal features in large-scale model-based engineering, characterized in that, The system includes: A prompt template compiler is used to parse declarative prompt templates into sequential query execution plans; The continuous query execution engine is used to trigger continuous query execution plans on the write path of the time series database to complete the sliding window feature calculation; Feature semantic mapper, used to convert numerical features into semantic tokens; A context table is provided to cache semantic tokens and their corresponding time ranges. The Prompt concatenation service is used to read semantic tokens from the prompt context table based on the time range of the external inference request and concatenate them into a complete Prompt.

2. The automatic generation system for temporal features of large-scale model-based engineering as described in claim 1, characterized in that, The Prompt template uses a declarative DSL description, with syntax elements including a time range placeholder {{window}}, a feature function placeholder {{agg(func,field)}}, and a text formatting placeholder {{format}}. Among them, func supports mean, std, fft_top3 and ks_score; The prompt word template compiler parses the DSL into an abstract syntax tree and generates the corresponding sequential query execution plan; The abstract syntax tree plan nodes include sliding window operators, feature operators, textification operators, and output format operators; The sliding window operator uses TSDB's native PIVOT RANGE EVERY semantics, and the window width and step size are parameterized by {{window}}; The feature operator calls the embedded User-Defined Aggregate Function library, supporting incremental computation; The textualization operator maps numerical features to semantic tokens through a pluggable tokenizer, for example, mapping ks_score>0.73 to "significant drift"; Output format operator: Generates key-value pairs {timestamp:_,features:_,semantic:_} that conform to the Prompt syntax.

3. The automatic generation system for temporal features of large-scale model-based engineering as described in claim 1 or 2, characterized in that, The continuous query execution engine reuses the continuous query framework of the time-series database and adds the following extensions: Write-on-demand: Immediately after successful WAL submission and before the in-memory table is flushed to disk, continuous query calculations are triggered to ensure that the feature visibility time is consistent with the original data; Incremental state management: The CombineFn interface is used to maintain the intermediate results of the sliding window, and the state is stored in the StateStore of the time series database, supporting fault recovery; Concurrency control: The SeqLock mechanism is introduced to ensure that feature calculations are performed only once within the same window on the same timeline, thus avoiding duplication.

4. The automatic generation system for temporal features of large-scale model-based engineering according to claim 3, characterized in that, The specific process of triggering the continuous query execution engine is as follows: (1) Device-side initiation point: The device submits a time-series record with device ID, timestamp, and sample value to the TSDB write thread via MQTT / CoAP or local Socket, which is encapsulated as a DataPoint structure; (2) Write thread receiving entry: The time series database write thread first appends the DataPoint structure to the currently active MemTable, and at the same time assigns a globally monotonically increasing log sequence number LSNn to the DataPoint structure to ensure that all subsequent derived data can be traced back to the same atomic point; (3) WAL persistence: The write thread encodes the LSNn and DataPoint structure together into an 8-byte WAL record header + variable-length payload, and writes it to the WAL log of NOR Flash through the memory-mapped file interface; after successful flushing, the WAL log returns ACK to the write thread, at which point the data has crash-safe capability. (4) Write-Time Hook Activation: The moment the WAL log returns a success, the time-series database calls the pre-embedded write_time_hook(LSNn) to immediately transfer the control flow to the continuous query execution engine; write_time_hook(LSNn) runs in user-mode interrupt context and must take less than 1ms, otherwise it will block subsequent writes; (5) CQ plan parsing: The continuous query execution engine quickly locates the sliding window into which the current write falls based on the pre-compiled AST, and checks whether the sliding window into which the current write falls already has an intermediate state: if the intermediate state is empty, the aggregation buffer is initialized; if the intermediate state is not empty, it is directly reused to avoid reallocating memory. (6) Feature UDAF calculation: The continuous query execution engine calls the embedded sliding window UDAF to perform incremental calculation on the numerical columns within the embedded sliding window UDAF; the embedded sliding window UDAF uses SIMD instructions, and takes about 90μs for 1000 points in a single window, and the result output is a 24B numerical feature vector. (7) Semantic mapping: Numerical feature vectors are fed into the feature semantic mapper. The feature semantic mapper maps floating-point features into semantic tokens through a pre-set quantile bucket table, and then concatenates them into a protocol buffer format prompt_kv object with a size controlled within 256B, which is convenient for direct embedding of large model prompts in the future. (8) Prompt context table insertion: The continuous query execution engine inserts the prompt_kv object along with LSNn and window start and end time as the primary key into the prompt context table. The prompt context table shares the same LSM-Tree with the original data table and uses an independent column family. Therefore, there is no extra disk flushing in the write path, and only one MemTable record needs to be appended. (9) Completion notification: After the context table is successfully written, return SUCCESS to the continuous query execution engine. The continuous query execution engine then calls back cq_done(LSNn) to notify the writing thread that the derived data is now visible and the 4kB buffer occupied by this hook can be released. (10) Response feedback: After receiving the notification, the write thread sends a WRITE_OK packet back to the device.

5. The automatic generation system for temporal features of large-scale model-based engineering according to claim 4, characterized in that, The feature semantic mapper incorporates a lightweight large model tokenizer, which is loaded via memory mapping; the mapping process is as follows: For continuous value features, quantile bucket encoding is used, and the bucket boundaries are obtained through offline statistics and cached in the TSDB metadata table; For discrete features, dictionary mapping can be used directly; For anomaly markers, the is_anomaly boolean value output by the drift detection UDAF is used and mapped to a "normal / abnormal" token; The mapping results are written to the prompt context table in UTF-8 byte stream format, saving 30% of storage; The context table is a system table within TSDB, with the schema: (timeline_id uint64, window_start timestamp, window_end timestamp, prompt_kv blob, ttl uint32). Here, prompt_kv is a Protocol Buffer-encoded key-value pair containing three fields: raw_features, semantic_tokens, and unit. The context table uses the same partitioning strategy as the data table, performing hash sharding based on timeline_id to ensure locality. Simultaneously, ttl is set to window_width * 2, allowing the time-series database to automatically reclaim expired contexts and prevent unlimited expansion.

6. The automatic generation system for temporal features of large-scale model-based engineering according to claim 5, characterized in that, The specific process of calling the Prompt splicing service is as follows: (1) Interface exposure: The Prompt splicing service exposes the standard gRPC method rpcGeneratePrompt(PromptRequest)returns(PromptReply) through prompt.proto; the transport layer uses HTTP / 2+TLS, port 443, and connection multiplexing keeps-alive for 300 seconds; (2) Request parsing and validity verification, as detailed below: ① After receiving the PromptRequest, first deduplicate and sort the timeline_id_list, limiting it to a maximum of 64 timelines; ② Verify that query_start ≤ query_end and the interval ≤ 1h; ③ Load the template AST from the memory dictionary based on prompt_template_id. If it does not exist, immediately return INVALID_ARGUMENT; (3) Contextual Index Positioning: Use the combined index of "timeline + window start and end" to generate a range scan plan: SELECT prompt_kv,crc32 FROM pct WHERE timeline_id IN(...) AND window_start>=:query_start AND window_end <= query_end ORDER BY window_start ASC; The corresponding query only accesses the column family of the context table and does not retrieve the original data table; (4) Result expansion and template filling: Each returned prompt_kv is decoded into a triple {timestamp,features[],semantic_tokens[]} according to Protocol Buffer; and expanded in the order of timestamp and concatenated into a continuous text segment; the text segment is inserted into the user's Prompt template slot to generate a complete Prompt string; (5) Correctness guarantee: Calculate the CRC32-C checksum for the complete Prompt string; and compare it with the accumulated CRC32 value of each record in the prompt context table query results: ① If they match, return the Prompt text directly; ② If inconsistent, trigger "rollback query": re-execute the context index positioning in step (3) and enable consistent read. If the second comparison still fails, return the DATA_INCONSISTENCY error code and have the caller retry. (6) Output and Link Statistics: The final Prompt text is encapsulated into PromptReply, which also carries the actual interval, number of feature lines, and time consumption μs; it is returned to the caller without accessing the original time series data. The average latency is <5ms and P99 is <10ms.

7. A method for automatically generating temporal features for large-scale model-based engineering, characterized in that, The method is as follows: The prompt template compiler parses the prompt template into a continuous query execution plan in the time-series database kernel; On the data ingestion write path, feature extraction, semantic alignment, and context caching are synchronously completed through a feature semantic mapper, and the results are written to a dedicated prompt context table in real time. The prompt context table is used to cache semantic feature text and its corresponding time range. When an external large model inference request arrives, the semantic feature text is read from the prompt context table according to the time range of the external large model push request and the Prompt template is completed.

8. The method for automatically generating temporal features for large-scale model-based hinting engineering according to claim 7, characterized in that, The Prompt template uses a declarative DSL description, with syntax elements including a time range placeholder {{window}}, a feature function placeholder {{agg(func,field)}}, and a text formatting placeholder {{format}}. Among them, func supports mean, std, fft_top3 and ks_score; The prompt word template compiler parses the DSL into an abstract syntax tree and generates the corresponding sequential query execution plan; The abstract syntax tree plan nodes include sliding window operators, feature operators, textification operators, and output format operators; The sliding window operator adopts the native PIVOT RANGE EVERY semantics of time series databases, and the window width and step size are parameterized by {{window}}; The feature operator calls the embedded User-Defined Aggregate Function library, supporting incremental computation; The textualization operator maps numerical features to semantic tokens through a pluggable tokenizer, for example, mapping ks_score>0.73 to "significant drift"; Output format operator: Generates key-value pairs {timestamp:_,features:_,semantic:_} that conform to the Prompt syntax.

9. The method for automatically generating temporal features for large-scale model-based projects according to claim 7 or 8, characterized in that, The context table is a system table in the time series database, with the schema: (timeline_id uint64, window_start timestamp, window_end timestamp, prompt_kv blob, ttl uint32); Among them, prompt_kv is a key-value pair encoded by Protocol Buffer, containing three fields: raw_features, semantic_tokens, and unit; The context table uses the same partitioning strategy as the data table, performing hash sharding based on timeline_id to ensure locality; at the same time, ttl is set to window_width*2, so that the time-series database can automatically reclaim expired contexts to avoid infinite expansion.

10. The method for automatically generating temporal features for large-scale model-based hinting engineering according to claim 9, characterized in that, The Prompt splicing service is exposed via a gRPC interface, as detailed below: Receive external inference requests with parameters timeline_id_list, query_start, query_end, and prompt_template_id; Based on the time range, execute the following query in the prompt context table: SELECT prompt_kv WHERE timeline_id IN (...) AND window_start >= query_start AND window_end <= query_end; The returned prompt_kv is expanded in chronological order and inserted into the user-provided Prompt template slot; Calculate the CRC32 checksum and compare it with the checksum cached in the context table: If they match, return immediately; If there is a discrepancy, a rollback query is triggered to ensure correctness; Output the complete Prompt text for LLM inference; This method also includes end-to-end consistency guarantees, as detailed below: Linear consistency: The continuous query execution engine performs calculations in a context where the WAL sequence number is monotonically increasing, and the feature results share the same log_sequence_id with the original data; Atomic rollback: If writing to the context table fails, the continuous query execution engine throws a RetryableException, and the time-series data is automatically replayed to replay the corresponding batch of data; Multi-replica synchronization: The context table is used as a system table and replicated to the majority of nodes via the Raft protocol to ensure that Prompt splicing can continue after a failover.

Images