Query method based on time series data generated modeling and hierarchical storage

Abstract

Embodiments of the present disclosure disclose a query method based on time series data generative modeling and hierarchical storage. The specific implementation of the method comprises: dividing a dynamic time window when the sampling segment information entropy rate reaches a preset mutation threshold; extracting a set of generation parameters including mean vector, covariance matrix, baseline trend polynomial coefficient, residual standard deviation and implicit space distribution factor into a warm storage layer, and compressing the fitting residual compensation into a cold storage layer; when the model reconstruction variance is not greater than a preset query tolerance error threshold, the query result is analyzed and output in the warm storage layer, when the threshold is greater than the threshold and the Euclidean state distance of the implicit space distribution factor of the adjacent window is less than the preset isomorphic latent variable threshold, the corresponding physical block address is spliced into a single discrete aggregation read instruction, the residual compensation is decompressed by the cold storage layer, and the query result is reconstructed and output by the generation parameter set, and by decoupling physical reading and logical query, the cold data decompression overhead and read amplification blocking are eliminated.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of big data storage and database query technology, specifically to a query method based on time-series data generative modeling and hierarchical storage. Background Technology

[0002] Static tiered storage based on data generation time is a common technical solution in big data storage systems. This solution typically stores recently generated, frequently accessed "hot" data in high-speed storage media, while older, less frequently accessed "cold" data is compressed and migrated to lower-cost object storage media. To balance data storage costs and read / write efficiency, existing tiered storage methods are prone to mismatches between physical data storage and query requirements when performing aggregation queries on historical data spanning long periods. Cold data is usually stored in lossless compression in fixed physical blocks. When the query engine needs to retrieve data within a certain time range, it often needs to read the entire compressed block into memory, decompress it, restore the data points, and then perform the query calculation. This process generates a large amount of data transfer and decompression overhead. In addition, when querying historical intervals with fluctuating device status or sudden changes, the system usually needs to continuously send a large number of discrete small data block read requests to the slower object storage media. Since the time overhead of network connection, handshake, and addressing is difficult to effectively amortize, it can easily lead to physical read amplification and cause data transmission channels and computing nodes to become blocked.

[0003] To reduce the burden of reading historical data, existing technologies typically employ downsampling or increased hardware resources. While downsampling can reduce data storage, it disrupts the statistical distribution characteristics of the original time-series data and loses some details of abrupt changes, making it difficult to meet the requirements of high-precision analysis. Increasing cache capacity or transmission bandwidth can improve query performance, but it increases system construction costs and cannot change the dependence of query calculations on the reading and decompression of underlying data. For example, Chinese invention patent application CN119646533A discloses a method and system for detecting anomalies in satellite time-series data. It uses a bidirectional LSTM network to construct a variational autoencoder and introduces adjusted cosine similarity. However, when processing long-period telemetry sequences from satellites, the inherent full-time-series dependency calculation of the bidirectional structure makes the forward and backward states susceptible to concurrent burst noise interference during deep network transmission. Furthermore, its distance correction mechanism does not fully consider the dynamic coupling characteristics between multi-dimensional channels, resulting in limited mapping accuracy of latent space distribution parameters under strong nonlinear and non-stationary conditions, making it difficult to balance low false alarm rate and high sensitivity.

[0004] Therefore, how to decouple physical reading from logical querying and eliminate decompression overhead and discrete network request overhead in the cold data range aggregation query process without reducing the accuracy of data reconstruction or increasing the burden of cold storage media retrieval has become a technical problem that needs to be solved. Summary of the Invention

[0005] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0006] Some embodiments of this disclosure propose a query method based on time-series data generative modeling and hierarchical storage to solve the technical problems mentioned in the background section above.

[0007] In a first aspect, some embodiments of this disclosure provide a query method based on generative modeling and hierarchical storage of time-series data. The method includes: Step S1, acquiring a continuous time-series data stream, calculating the information entropy change rate of sampled segments in the continuous time-series data stream in real time, and defining a dynamic time window when the information entropy change rate reaches a preset mutation threshold; Step S2, constructing a generative parameter model for the original time-series data within each dynamic time window, extracting a generative parameter set containing a mean vector, covariance matrix, baseline trend polynomial coefficients, residual standard deviation, and implicit spatial distribution factors, storing it in a warm storage layer, and compressing and storing the residual compensation amount between the original time-series data and the fitted curve of the generative parameter model in a cold storage layer; Step S3, receiving a query request containing a target time range and query type, calculating the model reconstruction variance of the dynamic time window covered by the target time range in the warm storage layer based on the generative parameter set; Step S4, in the model reconstruction... When the model reconstruction variance is not greater than the preset query tolerance error threshold, the query result is output in the warm storage layer based on the generated parameter set; Step S5, when the model reconstruction variance is greater than the preset query tolerance error threshold, the Euclidean state distance between the latent spatial distribution factor of the current dynamic time window and the latent spatial distribution factor of the subsequent adjacent dynamic time windows is calculated in the warm storage layer; Step S6, when the Euclidean state distance is less than the preset isomorphic latent variable threshold, the physical block address of the residual compensation amount in the cold storage layer for the corresponding dynamic time window is concatenated into a single discrete aggregation read instruction, which is then retrieved by the cold storage layer, decompressed, and reconstructed with the generated parameter set to output the query result; when the Euclidean state distance is not less than the preset isomorphic latent variable threshold, the discrete aggregation read queue is truncated at the corresponding dynamic time window, and a residual compensation amount retrieval instruction for the cold storage layer is issued independently for the current dynamic time window, which is then decompressed, reconstructed, and the query result is output.

[0008] Preferably, step S1 includes the following sub-steps: Step S11, dividing the data amplitude of the continuous time-series data stream into a preset number of equidistant intervals, and calculating the probability that data points fall into each equidistant interval within the current sampling time segment; Step S12, calculating the Shannon entropy of the time-series data amplitude distribution of the current sampling time segment based on the probability of each equidistant interval; Step S13, calculating the absolute value of the difference between the Shannon entropy of the time-series data amplitude distribution of the current sampling time segment and the Shannon entropy of the time-series data amplitude distribution of the previous sampling time segment, and dividing the absolute value by the duration of the sampling time segment to obtain the information entropy change rate.

[0009] Preferably, the preset mutation threshold for the information entropy change rate is set in the range of 0.10 to 0.80, and the upper limit of the number of sampling points in the dynamic time window is set to 500 to 50,000.

[0010] Preferably, after concatenating the physical block address of the residual compensation amount in the cold storage layer for the corresponding dynamic time window into a single discrete aggregation read instruction, step S6 includes the following sub-steps: Step S61, decompress the residual compensation amount retrieved by the single discrete aggregation read instruction and store it in the system memory buffer pool; Step S62, when reconstructing the output query result for subsequent adjacent dynamic time windows, directly read the corresponding residual compensation amount from the system memory buffer pool to avoid repeatedly issuing read instructions for the cold storage layer.

[0011] Preferably, the mean vector, covariance matrix, baseline trend polynomial coefficients, residual standard deviation, and implicit spatial distribution factor of the generated parameter set are all stored in a warm storage layer using a fixed-byte numerical array.

[0012] Preferably, in step S3, calculating the model reconstruction variance of the dynamic time window covered by the target time range based on the generated parameter set specifically involves: extracting the residual standard deviation from the generated parameter set, dividing the square of the residual standard deviation by the number of sampling points within the dynamic time window covered by the target time range, and obtaining the model reconstruction variance of the corresponding dynamic time window under the mean query type.

[0013] Preferably, the warm storage layer is deployed on a solid-state drive medium, and the cold storage layer is deployed on an object storage medium; the method further includes storing the latest generated original time-series data in the continuous time-series data stream through a hot storage layer deployed on a non-volatile memory medium, and the hot storage layer adopts an append-write log structure.

[0014] Preferably, the preset query tolerance error threshold is set within the range of 0.0001 to 0.2000.

[0015] Preferably, after the reconstruction of all dynamic time windows covered by the target time range is completed, the residual compensation memory space occupied in the system memory buffer pool is released.

[0016] The above-described embodiments of this disclosure have the following beneficial effects: 1. In the query of generative modeling and hierarchical storage of time series data, dynamic time windows are divided according to the information entropy change rate, so that the window boundaries can be adjusted with the fluctuation and mutation of the time series data. This reduces the fitting bias caused by the fixed time window spanning different data states. By storing the generation parameter set, which reflects the data distribution and change trend, including the mean vector, covariance matrix, baseline trend polynomial coefficients, residual standard deviation and implicit spatial distribution factors, in the warm storage layer, and compressing the residual compensation amount and storing it in the cold storage layer, the statistical distribution characteristics and mutation details of the original time series data can be preserved in a smaller storage space, thus balancing storage space occupation and data reconstruction accuracy.

[0017] 2. Determine whether the generated parameter set can meet the current query based on the query type and accuracy requirements. When the model reconstruction variance is within the allowable range of the query, the query calculation can be completed directly in the warm storage layer without reading and decompressing the original residual data in the cold storage layer. This can reduce the amount of disk reads and decompression calculations required for historical range aggregation queries, shorten the query response time, and adapt to the accuracy requirements of different queries.

[0018] 3. When the generated parameter set cannot meet the query accuracy requirements, the data state is judged to be similar based on the implicit spatial distribution factor of adjacent dynamic time windows. The residual compensation of windows with similar states is read in a concentrated manner. When the Euclidean state distance exceeds the preset isomorphic latent variable threshold, the read queue is truncated to issue read instructions independently. This reduces the number of network connections, handshakes and addressing times of the object storage medium, reduces the additional overhead caused by discrete reading, and alleviates the read blocking that is easy to occur during the query of fluctuating data. Attached Figure Description

[0019] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0020] Figure 1 This is a flowchart of the time-series data hierarchical storage and query process of the present invention; Figure 2 This is the model reconstruction variance verification and residual reconstruction diagram of the present invention; Figure 3 This is a comparison chart of the entropy change rate and mutation threshold of the sampled fragment information in this invention. Detailed Implementation

[0021] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0022] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0023] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0024] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0025] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0026] Before performing any of the operations involving the collection, storage, processing, or use of the target images disclosed herein, the relevant organizations or individuals shall fulfill their obligations, including conducting information security impact assessments, informing the information subjects, and obtaining prior authorization and consent from the information subjects.

[0027] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0028] refer to Figure 1 The diagram illustrates the flow of some embodiments of a query method based on time-series data generative modeling and hierarchical storage according to this disclosure. A query method based on time-series data generative modeling and hierarchical storage includes the following steps: Step S1: Acquire a continuous time-series data stream, calculate the information entropy change rate of the sampled segments in the continuous time-series data stream in real time, and define a dynamic time window when the information entropy change rate reaches a preset mutation threshold. Step S2: Construct a generative parametric model for the original time series data within each dynamic time window, extract the generative parameter set containing the mean vector, covariance matrix, baseline trend polynomial coefficients, residual standard deviation and implicit spatial distribution factor and store it in the warm storage layer, and compress and store the residual compensation between the original time series data and the fitted curve of the generative parametric model in the cold storage layer. Step S3: Receive a query request containing the target time range and query type, and calculate the model reconstruction variance of the dynamic time window covered by the target time range in the warm storage layer based on the generated parameter set; Step S4: When the model reconstruction variance is not greater than the preset query tolerance error threshold, the query result is parsed and output in the warm storage layer based on the generated parameter set. Step S5: When the model reconstruction variance is greater than the preset query tolerance error threshold, calculate the Euclidean state distance between the latent spatial distribution factor of the current dynamic time window and the latent spatial distribution factor of the subsequent adjacent dynamic time windows in the warm storage layer. Step S6: When the Euclidean state distance is less than the preset isomorphic latent variable threshold, the physical block address of the residual compensation amount in the cold storage layer for the corresponding dynamic time window is concatenated into a single discrete aggregation read instruction. The residual compensation amount is retrieved by the cold storage layer, decompressed, and reconstructed with the generated parameter set to output the query result. When the Euclidean state distance is not less than the preset isomorphic latent variable threshold, the discrete aggregation read queue is truncated at the corresponding dynamic time window, and a residual compensation amount retrieval instruction for the cold storage layer is issued independently for the current dynamic time window. After decompression and reconstruction, the query result is output.

[0029] Preferably, step S1 includes the following sub-steps: Step S11, dividing the data amplitude of the continuous time-series data stream into a preset number of equidistant intervals, and calculating the probability that data points fall into each equidistant interval within the current sampling time segment; Step S12, calculating the Shannon entropy of the time-series data amplitude distribution of the current sampling time segment based on the probability of each equidistant interval; Step S13, calculating the absolute value of the difference between the Shannon entropy of the time-series data amplitude distribution of the current sampling time segment and the Shannon entropy of the time-series data amplitude distribution of the previous sampling time segment, and dividing the absolute value by the duration of the sampling time segment to obtain the information entropy change rate.

[0030] Preferably, the preset mutation threshold for the information entropy change rate is set in the range of 0.10 to 0.80, and the upper limit of the number of sampling points in the dynamic time window is set to 500 to 50,000.

[0031] Preferably, after concatenating the physical block address of the residual compensation amount in the cold storage layer for the corresponding dynamic time window into a single discrete aggregation read instruction, step S6 includes the following sub-steps: Step S61, decompress the residual compensation amount retrieved by the single discrete aggregation read instruction and store it in the system memory buffer pool; Step S62, when reconstructing the output query result for subsequent adjacent dynamic time windows, directly read the corresponding residual compensation amount from the system memory buffer pool to avoid repeatedly issuing read instructions for the cold storage layer.

[0032] Preferably, the mean vector, covariance matrix, baseline trend polynomial coefficients, residual standard deviation, and implicit spatial distribution factor of the generated parameter set are all stored in a warm storage layer using a fixed-byte numerical array.

[0033] Preferably, in step S3, calculating the model reconstruction variance of the dynamic time window covered by the target time range based on the generated parameter set specifically involves: extracting the residual standard deviation from the generated parameter set, dividing the square of the residual standard deviation by the number of sampling points within the dynamic time window covered by the target time range, and obtaining the model reconstruction variance of the corresponding dynamic time window under the mean query type.

[0034] Preferably, the warm storage layer is deployed on a solid-state drive medium, and the cold storage layer is deployed on an object storage medium; the method further includes storing the latest generated original time-series data in the continuous time-series data stream through a hot storage layer deployed on a non-volatile memory medium, and the hot storage layer adopts an append-write log structure.

[0035] Preferably, the preset query tolerance error threshold is set within the range of 0.0001 to 0.2000.

[0036] Preferably, after the reconstruction of all dynamic time windows covered by the target time range is completed, the residual compensation memory space occupied in the system memory buffer pool is released.

[0037] Example 1: In the current industrial-grade smart grid high-frequency sensor monitoring system, the intelligent terminal node continuously collects time-series data streams of power grid transformer winding temperature and vibration amplitude at a sampling frequency of 50Hz. The system uses 1 second as a sampling time segment, with each sampling time segment containing 50 sampling points. The data amplitude is divided into 256 equidistant intervals, and the probability of a sampling point falling into each equidistant interval is statistically analyzed. The Shannon entropy of the amplitude distribution of the time series data for the t-th sampling time segment is calculated according to the following formula. ,in, This represents the probability that the data amplitude falls into the i-th equidistant interval within the t-th sampling time segment. The system calculates the absolute value of the Shannon entropy difference between adjacent sampling time segments according to the following formula, and divides the absolute value by a fixed 1-second time interval to calculate the Shannon entropy change per unit time as the information entropy rate of change, which is used to characterize the rate of change of the probability distribution of time-series data in the time dimension. The preset change threshold η is 0.45. When the information entropy rate of change... When the preset mutation threshold η is reached, or when the number of sampling points in the current window reaches 3000, the current window ends and a dynamic time window is defined. .

[0038] For dynamic time windows The system constructs a generative parametric model based on a conditional variational encoder for the original time-series data, fits the probability distribution and baseline trend of the original time-series data, and extracts the mean vector. Covariance matrix Baseline trend polynomial coefficients , and residual standard deviation and two-dimensional implicit spatial distribution factor To form a set of generation parameters Generate parameter set The mean vector, covariance matrix, baseline trend polynomial coefficients, residual standard deviation, and latent spatial distribution factor are all stored in a warm storage layer using fixed-byte numerical arrays. The system calculates the difference between the original time-series data and the fitted curve of the generative parametric model to obtain the residual compensation amount. .

[0039] In a three-tiered physical storage topology, the latest generated raw time-series data is stored in a hot storage layer composed of non-volatile memory media in an append-write log structure, generating parameter sets. and dynamic time window The time boundary index is stored in a warm storage layer composed of solid-state drives, and the residual compensation amount... After block-level lossless compression, the data is stored in a cold storage layer composed of object storage media, while the mean vector in the warm storage layer is stored in a cold storage layer. Covariance Matrix Used for probability distribution queries and anomaly probability interval estimation; when the query type requires the confidence probability that the data falls within a specific amplitude interval, the query routing engine calls the mean vector. Covariance Matrix Construct a multidimensional Gaussian probability density function and integrate it over the target amplitude interval to reveal the implicit spatial distribution factor. Used to calculate the Euclidean state distance between adjacent dynamic time windows, and based on this, to determine the aggregate read range of residual compensation in the cold storage layer.

[0040] The system receives the target time range, mean query type, and preset query tolerance error threshold. The query request locates the dynamic time window covered by the target time range. And read the corresponding generation parameter set from the warm storage layer. The query routing engine determines the calculation method for model reconstruction variance based on the query type; for mean-based queries, it calculates the residual standard deviation. Squared by the dynamic time window Number of sampling points within The model reconstruction variance was obtained. The model reconstruction variance is a specific numerical estimation error statistic bound to the currently received query type. For mean query types, the model reconstruction variance is specifically defined as the squared mean estimate calculated by multiplying the residual standard deviation stored in the parameter set by the square of the residual standard deviation, and then dividing by the total number of discrete sampling points within the current dynamic time window. For extreme value or quantile query types, the model reconstruction variance is specifically defined as the weighted sum of the baseline trend polynomial fitting variance and the residual variance, used to uniformly quantify the upper bound of the statistical variance generated when reconstructing query results solely based on the warm storage layer model parameters. In this embodiment, the residual standard deviation... Take 0.80, number of sampling points Set the value to 100, and calculate the model reconstruction variance. The preset query tolerance error threshold is 0.0064. We set it to 0.01, due to the model reconstruction variance. Not greater than the preset query tolerance error threshold The query routing engine directly calls the parameter set generated in the warm storage layer. The mean query result is parsed. For other query types, the query routing engine generates a parameter set, substitutes it into the corresponding query operation, and uses the variance of the result obtained from the query operation as the model reconstruction variance for the corresponding dynamic time window.

[0041] Based on baseline trend polynomial coefficients and The average analytical value of the temperature storage layer is calculated according to the following formula. Among them, the baseline trend polynomial coefficients and The sampling points were set to 20.0 and 0.10 respectively. Taking 100, the average analytical value of the temperature storage layer was calculated. The value is 25.00, the sum of the original sampled data is 2530.0, and the corresponding actual accurate average value is 25.30. The measured relative error between the average value resolved by the temperature storage layer and the actual accurate average value is 0.01186. This measured relative error is used to record the deviation between the resolved result and the accurate result of the original sampled data. A preset query tolerance error threshold is set. Then it is used as the variance of model reconstruction. The upper limit of the comparison is that the two are not considered the same comparison metric because of the model reconstruction variance. The preset query tolerance error threshold was not exceeded. This query does not require retrieving the residual compensation amount in the cold storage layer, which can reduce the physical disk read volume and control the query response time to within 15ms.

[0042] The aforementioned analytical average value of the warm storage layer is rapidly estimated using a continuous time domain integral form for the discrete sampled sequence. When performing discrete calculations according to the sampling point number increasing from 1 to 100, the discrete sampling interval correction is included in the average value of the baseline trend polynomial. The corrected discrete analytical average value is 25.05. The deviation between this analytical average value and the actual accurate average value of 25.30 comes from the non-zero mean portion of the discrete residual sequence. When retrieving the residual compensation amount for reconstruction, the system accumulates the residuals and includes them in the query result to compensate for this deviation.

[0043] When the system receives a high-precision audit query request, a preset query tolerance error threshold is established. Set to 0.001, due to the model reconstruction variance The value is 0.0064, which is greater than the preset query tolerance error threshold. The query routing engine enters the cold / warm I / O decoupling pre-aggregation process to extract the current dynamic time window. Implicit spatial distribution factor And sequentially read the subsequent adjacent dynamic time windows that have not yet been parsed within the target time range. and Implicit spatial distribution factor and Current dynamic time window With any subsequent adjacent dynamic time window Euclidean distance between Calculate according to the following formula: ,in, Represents a dynamic time window The latent spatial distribution factor has eigencomponents in the m-th dimension. Represents a dynamic time window The latent spatial distribution factor has eigencomponents in the m-th dimension, and the dynamic time window. With dynamic time window Euclidean distance between The value is 0.0361, which is less than the preset isomorphic latent variable threshold of 0.10. Therefore, the query routing engine will use a dynamic time window. Merged into the discrete-aggregate read queue, dynamic time window With dynamic time window Euclidean distance between The value is 0.7159, which is greater than the preset isomorphic latent variable threshold of 0.10. Therefore, the query routing engine will read the discrete aggregation queue within the dynamic time window. Cut off at the current dynamic time window. and Issuing a single discrete aggregation read command for a dynamic time window exceeding the threshold of isomorphic latent variables. Then, an independent instruction will be issued and processed.

[0044] Query the routing engine to obtain the dynamic time window and dynamic time window The physical block address of the corresponding residual compensation amount in the cold storage layer is concatenated into a single discrete aggregate read instruction. The data structure of this single discrete aggregate read instruction specifically includes a unique identifier of the target cold storage layer object and a Hypertext Transfer Protocol byte range request header data frame consisting of multiple non-contiguous physical start byte offsets and length values. By simultaneously encapsulating the read range parameters of multiple discrete data blocks in a single network socket connection, multiple independent physical read tasks are merged into a single application layer data acquisition command and sent to the cold storage layer. The cold storage layer object storage medium adopts an object-oriented application layer transport protocol. The query routing engine converts each physical block address into a byte range field that the cold storage layer read interface can recognize, and writes multiple byte range fields in the same composite Hypertext Transfer Protocol request. The cold storage layer locates the data segment corresponding to each physical block address based on these fields and returns continuous or discrete compressed residual data through a single network connection. This conversion is only used to adapt the cold storage layer's read interface, and the read target is still the physical block address of the residual compensation amount concatenated in the single discrete aggregate read instruction.

[0045] After the cold storage layer returns the compressed residual data, the system stores it in the system memory buffer pool and decompresses it. The query routing engine calculates the dynamic time window according to the following formula. The final reconstruction average ,in, This represents the average resolution of the temperature storage layer. Indicates residual compensation amount The i-th array element in the array represents the residual compensation amount. The sum of the array elements is 30.0, and the number of sampling points is... Taking 100, the final reconstructed average value is calculated. The value was 25.30, consistent with the actual precise average of the original sampled data, followed by processing of the dynamic time window. At this time, the query routing engine directly reads the corresponding residual compensation amount from the system memory buffer pool and participates in the reconstruction, without repeatedly sending read commands to the cold storage layer.

[0046] When the model reconstruction variance is not greater than the preset query tolerance error threshold, the query routing engine directly uses the generated parameter set in the warm storage layer to output the query results. When the model reconstruction variance is greater than the preset query tolerance error threshold, the aggregation range of the residual compensation amount is determined according to the Euclidean state distance between adjacent dynamic time windows. Multiple physical block addresses are concatenated into a single discrete aggregation read instruction, and decompression and reconstruction are completed in the system memory buffer pool. When the Euclidean state distance is not less than the preset isomorphic latent variable threshold, the read queue is truncated and independent read is issued. This process reduces the number of network connections, protocol handshakes and addressing times in the cold storage layer, while reducing physical read amplification and channel blocking during the discrete residual data reading process.

[0047] Example 2: This example uses a cluster system composed of 8 distributed computing nodes. The hot storage layer is deployed on non-volatile memory media, the warm storage layer is deployed on solid-state drive media, and the cold storage layer is deployed on object storage media based on the HTTP protocol. The test data is a high-frequency sampling dataset of power transformer temperature and vibration sensors containing 10 million records. Gaussian white noise with a signal-to-noise ratio of 20dB and 50Hz power frequency harmonic interference are superimposed on the original signal to test the query response, data reconstruction, and cold storage layer reading of the query method of time-series data generative modeling and hierarchical storage under interference conditions.

[0048] During parameter configuration, a preset mutation threshold η is used to determine the granularity of the dynamic time window. When the preset mutation threshold η is below 0.10, the information entropy change rate caused by environmental noise easily reaches this threshold, increasing the frequency of dynamic time window segmentation and the number of generated parameter sets and time boundary indexes in the warm storage layer. When the preset mutation threshold η is above 0.80, some waveform mutations cannot trigger window segmentation in time, and the fitting residual of the generative parameter model increases accordingly. The isomorphic latent variable threshold τ is determined based on the range of values ​​of the two-dimensional geometric coordinate points of the latent spatial distribution factor in Euclidean space and the discrete distance threshold. Specifically, it is calculated by checking whether the straight-line distance between the two sets of two-dimensional floating-point coordinates corresponding to adjacent time windows in the Cartesian coordinate system is less than the preset discrete value threshold. This is used to control the number of physical blocks of residual compensation included in a single discrete aggregation read. A preset query tolerance error threshold is also included. Based on the accuracy requirements of the business query, the error is configured in stages within the range of 0.0001 to 0.2000, and serves as the upper limit for comparing the variance of the model reconstruction. The measured relative error of each group of records is used to characterize the deviation between the query result and the accurate result of the original data, and does not exceed the preset query tolerance error threshold. Direct comparison.

[0049] The experiment included sample group 1, sample group 2, sample group 3 of the present invention, and 4 control groups, for a total of 7 groups. Sample group 1 of the present invention had a preset mutation threshold η set to 0.45 and an isomorphic latent variable threshold. Set to 0.10, preset query tolerance error threshold Set to 0.0100; Sample group 2 of this invention will use the preset mutation threshold. Set the threshold to 0.10, the isomorphic latent variable threshold τ to 0.10, and the preset query tolerance error threshold. Set to 0.0010; in sample group 3 of this invention, the preset mutation threshold η is set to 0.80, the isomorphic latent variable threshold τ is set to 0.10, and the preset query tolerance error threshold is set to 0.0010. Set it to 0.1000.

[0050] Control group 1 set the preset mutation threshold η to 0.45 and the preset query tolerance error threshold. The value was set to 0.0100, and the Euclidean state distance determination based on the implicit spatial distribution factor and the pre-aggregation and splicing of physical block addresses were removed; control group 2 set the preset mutation threshold η to 0.05 and the preset query tolerance error threshold to 0.05. Set to 0.0100; control group 3 sets the preset mutation threshold η to 0.90 and the preset query tolerance error threshold. Set to 0.0100; control group 4 uses a static 60-second fixed time window and stores all original time series data in a cold storage layer without loss of compression, without performing warm storage layer parsing queries based on the generated parameter set.

[0051] Under the same concurrent aggregation query pressure, sample group 1 of the present invention is based on a preset mutation threshold. Define a dynamic time window, and ensure that the variance of model reconstruction does not exceed a preset query tolerance error threshold. When the mean query results are directly parsed in the warm storage layer based on the generated parameter set, the query response latency for this group is 12.3ms, the disk read reduction rate is 88.6%, the CPU decompression utilization rate is 4.2%, and the measured relative error is 0.01186. Control group 4 needs to read and decompress all relevant original time series data from the cold storage layer, and its query response latency is 215.4ms, the disk read reduction rate is 0.0%, the CPU decompression utilization rate is 82.3%, and the measured relative error is 0.00000.

[0052] Sample group 2 of this invention uses a preset mutation threshold η of 0.10 and a preset query tolerance error threshold of 0.0010. With this parameter configuration, the query response latency is 14.8ms, the disk read reduction rate is 85.2%, the CPU decompression utilization rate is 8.6%, and the measured relative error is 0.00085. The low preset query tolerance error threshold allows queries with excessive model reconstruction variance to enter the cold storage layer for reconstruction processing, while retaining the single discrete aggregation read process based on Euclidean state distance.

[0053] Sample group 3 of this invention uses a preset mutation threshold η of 0.80 and a preset query tolerance error threshold of 0.1000. The query response latency for this group is 8.5ms, the disk read reduction rate is 94.1%, the CPU decompression utilization rate is 1.8%, and the measured relative error is 0.08240. Under this configuration, the model reconstruction variance of most queries is within the preset query tolerance error threshold range, and the query routing engine directly uses the generated parameter set in the warm storage layer to output the query results.

[0054] In control group 1, the dynamic time window division parameters and preset query tolerance error threshold are consistent with those of sample group 1 of this invention, but the Euclidean state distance is no longer calculated based on the implicit spatial distribution factor of adjacent dynamic time windows, and the physical block address of the residual compensation amount is not concatenated into a single discrete aggregation read instruction. When the model reconstruction variance is greater than the preset query tolerance error threshold, At that time, the system continuously sent 120 discrete data acquisition requests to the cold storage layer. The disk read reduction rate of this group was still 88.6%, and the measured relative error was still 0.01186. However, the multiple network connections, protocol handshakes and addressing increased the query response latency to 86.7ms and the CPU decompression utilization rate to 28.4%.

[0055] In control group 2, the preset mutation threshold η was 0.05, which is lower than 0.10. The Gaussian white noise and power frequency harmonic interference superimposed on the original signal caused the information entropy change rate to reach the threshold multiple times within 1 second. The dynamic time window was frequently divided, and the number of generated parameter sets in the warm storage layer increased accordingly. The query response latency of this group was 45.2ms, the disk read reduction rate was 62.4%, the CPU decompression utilization rate was 12.1%, and the measured relative error was 0.00920.

[0056] In control group 3, the preset mutation threshold η was 0.90, which was higher than 0.80. Some high-frequency waveform mutations failed to trigger dynamic time window segmentation. The generative parameter model was fitted across different data states, the residual compensation amount increased, and the number of times the model reconstruction variance exceeded the preset query tolerance error threshold increased accordingly. During the test, 92.4% of the query requests entered the cold storage layer for reading and reconstruction processing. The query response latency of this group was 168.4ms, the disk read reduction rate was 22.8%, the CPU decompression utilization rate was 68.5%, and the measured relative error was 0.01186.

[0057] All the above groups use the same dataset and concurrent query conditions. After defining the dynamic time window according to the preset mutation threshold, the generated parameter set is stored in the warm storage layer. The query result can be directly output when the model reconstruction variance meets the query accuracy condition. When the model reconstruction variance exceeds the preset query tolerance error threshold, the residual compensation amount is aggregated according to the Euclidean state distance of the latent spatial distribution factor, which can reduce the discrete read requests of the cold storage layer. When the distance exceeds the isomorphic latent variable threshold, the queue is truncated to avoid cross-state read amplification. When the preset mutation threshold is too low, the frequency of dynamic time window segmentation increases. When the preset mutation threshold is too high, the fitting residual of the generative parameter model increases, and the number of queries entering the cold storage layer for reconstruction processing increases accordingly.

[0058] Example 3: Current industrial-grade smart grid high-frequency sensor monitoring systems incorporate the continuous time-series data streams collected by sensor nodes into dynamic time windows. Subsequently, the system processor uses the least squares polynomial regression algorithm to calculate the baseline trend polynomial coefficients based on the sampling time index and the measured amplitude column vector. , and And generate a dynamic time window accordingly. The baseline trend fitting curve within the range is used to subtract the calculated value of the fitting curve at the corresponding sampling point from the original sampling amplitude to obtain the fluctuation residual sequence.

[0059] The conditional variational encoder neural network takes a fixed-length one-dimensional tensor of fluctuation residuals with 100 sampling points as input. This tensor passes through two one-dimensional convolutional layers and one fully connected layer. The kernel size of the two one-dimensional convolutional layers is 3, and the number of channels is 16 and 32, respectively. The number of nodes in the fully connected layer is 64. After feature extraction, two independent linear fully connected output layers output a mean vector of dimension 2. The covariance matrix with a main diagonal dimension of 2 The two-dimensional latent spatial distribution factor was obtained through reparameterized sampling. The two-dimensional latent space distribution factor is specifically a two-dimensional numerical array composed of two discrete floating-point numbers, which corresponds to a two-dimensional eigenvector in the low-dimensional latent space coordinate system. It is used to characterize the probability distribution morphology of the fluctuation residual within a specific time window. The system also calculates the distribution standard deviation of the fluctuation residual sequence to obtain the residual standard deviation. The discrete difference between the original sampled data and the fitted curve of the generative parametric model is saved as the residual compensation amount. Baseline trend polynomial coefficients , and Mean vector Covariance matrix residual standard deviation and implicit spatial distribution factors Together they constitute the generation parameter set It is stored in the warm storage layer composed of solid-state drive media in the form of a numerical array of fixed byte length.

[0060] During model training, the conditional variational encoder neural network uses the sampling period corresponding to the dynamic time window as the conditional vector and adopts a composite loss function composed of the mean square error loss function and the variational boundary divergence loss function. Offline parameter training is completed through the backpropagation algorithm. During online inference, the encoding layer maps the fluctuation residual sequence to a low-dimensional latent space and outputs a two-dimensional latent space distribution factor through reparameterized sampling to characterize the distribution state of the fluctuation residual within the dynamic time window.

[0061] During deployment initialization, the system measures the single HTTP network connection handshake latency of the cold storage layer object storage medium and the single block decompression time of the system memory for ZSTD compressed residual blocks. Based on the ratio of network connection overhead to decompression computation overhead, and combined with historical sampling streams under different vibration frequencies, the probability density distribution of latent spatial distribution factors in two-dimensional Euclidean space is statistically analyzed. For adjacent dynamic time windows, the routing engine calculates the Euclidean state distance between their latent spatial distribution factors. When the Euclidean state distance is less than the preset isomorphic latent variable threshold... When the corresponding dynamic time window is included in the same discrete aggregation read queue, and the system memory overhead occupied by the pre-aggregated residual block is lower than the latency loss caused by repeatedly establishing network connections, this threshold is used as the judgment boundary for aggregation read. At a sampling frequency of 50Hz, the preset isomorphic latent variable threshold τ is calibrated to 0.10.

[0062] Euclidean state distance is only used to determine the range of dynamic time windows that need to be merged and read. The query routing engine obtains the physical block addresses of the residual compensation amount corresponding to these dynamic time windows based on the time boundary index saved in the warm storage layer, and concatenates the physical block addresses into a single discrete aggregation read instruction. Each physical block can be stored continuously or dispersedly in the cold storage layer. The cold storage layer retrieves the corresponding compressed residual data according to the physical block address recorded in the read instruction, without inferring the actual storage location of the physical block based on the similarity of the implicit spatial distribution factor.

[0063] After a single discrete aggregation read command returns data, the system memory buffer pool allocates contiguous memory pages to hold the compressed residual data retrieved in batches. After decompression, the system maintains the resident status of each residual block through a doubly linked list and the least recently used eviction rule. During the precise reconstruction in the current dynamic time window and subsequent adjacent dynamic time windows merged into the same read queue, the system sets a memory read lock flag for the corresponding residual compensation amount, enabling the query routing engine to directly read the decompressed residual compensation amount from the system memory buffer pool.

[0064] When multiple concurrent queries read the same residual data block simultaneously, the system uses hardware-level atomic comparison and exchange instructions to set the memory read lock flag and increment the lock count to distinguish the query tasks currently using the residual data block. Each memory read lock flag is configured with a 500-millisecond timeout automatic forced release timer. When an abnormal query task times out and does not actively release the read lock, the system forcibly decrements the corresponding lock count. After all dynamic time windows covered by the target time range have completed query reconstruction and output the query results, the query routing engine releases the read lock flag, calls the memory release interface to clear the decompressed residual data, and reclaims the memory space occupied by the residual compensation in the system memory buffer pool.

[0065] Example 4: This example combines Figures 1 to 3 This section explains the query method based on generative modeling and hierarchical storage of time-series data, such as... Figure 1As shown, in step S1, a continuous time-series data stream is acquired, and the information entropy rate of change of the sampled segments in the continuous time-series data stream is calculated in real time. When the information entropy rate of change reaches a preset mutation threshold, a dynamic time window is defined. In step S2, a generative parametric model is constructed for the original time-series data within each dynamic time window. The generative parameter set, which includes the mean vector, covariance matrix, baseline trend polynomial coefficients, residual standard deviation, and implicit spatial distribution factor, is extracted and stored in a warm storage layer. The residual compensation between the original time-series data and the fitted curve of the generative parametric model is compressed and stored in a cold storage layer. In step S3, a query request containing the target time range and query type is received. In the warm storage layer, the model reconstruction variance of the dynamic time window covered by the target time range is calculated based on the generative parameter set. In step S4, when the model reconstruction variance is not greater than a preset query tolerance error threshold, the model is stored in a cold storage layer. In the reservoir, the query result is output based on the generated parameter set. In step S5, when the model reconstruction variance is greater than the preset query tolerance error threshold, the Euclidean state distance between the latent spatial distribution factor of the current dynamic time window and the latent spatial distribution factor of the subsequent adjacent dynamic time windows is calculated in the warm storage layer. In step S6, when the Euclidean state distance is less than the preset isomorphic latent variable threshold, the physical block address of the residual compensation amount in the cold storage layer for the corresponding dynamic time window is concatenated into a single discrete aggregation read instruction. The residual compensation amount is retrieved by the cold storage layer, decompressed, and reconstructed with the generated parameter set to output the query result. When the Euclidean state distance is not less than the preset isomorphic latent variable threshold, the discrete aggregation read queue is truncated at the corresponding dynamic time window, and a residual compensation amount retrieval instruction for the cold storage layer is issued independently for the current dynamic time window. After decompression and reconstruction, the query result is output.

[0066] like Figure 2 As shown, the system receives query requests and calculates the model reconstruction variance. When the variance is not greater than a preset query tolerance threshold, it is parsed and output by the warm storage layer. When it is greater than the preset query tolerance threshold, the Euclidean state distance is calculated. When it is less than a preset isomorphic latent variable threshold, a single discrete aggregation read instruction is concatenated, retrieved and decompressed by the cold storage layer, and stored in the system memory buffer pool. The query result is then reconstructed and output after reconstruction with the generated parameter set. For subsequent adjacent dynamic time windows, the Euclidean state distance is calculated, and the corresponding residual compensation amount is directly read from the system memory buffer pool to participate in the reconstruction and output of the query result. When the Euclidean state distance is not less than the preset isomorphic latent variable threshold, the read queue is truncated and a read instruction is issued independently. After all dynamic time window queries and reconstructions are completed, the residual compensation amount memory space is released.

[0067] like Figure 3As shown, the horizontal axis represents the sampling segment number and the vertical axis represents the information entropy change rate. In the legend, the solid line represents the information entropy change rate of the sampling segment and the dashed line represents the preset mutation threshold. During the sequence change of sampling segment numbers 1 to 10, the information entropy change rate of the sampling segment fluctuates within the range of 0.00 to 0.70. The preset mutation threshold is set as the horizontal reference line located at the 0.45 mark, and the information entropy change rate of the sampling segment exceeds the preset mutation threshold of 0.45 at sampling segment numbers 4 and 7.

[0068] Example 5: In the current industrial transformer deployment environment, there is frequent electromagnetic interference. Before the intelligent terminal node goes online, 300 seconds of background electromagnetic noise and thermal noise data streams are collected. The data amplitude is divided into 256 equidistant intervals and the background Shannon entropy baseline is calculated. Then, the information entropy rate baseline value is determined based on the fluctuation variance of the background Shannon entropy under the condition of no physical fault. When the signal-to-noise ratio of the sensor drifts within the range of 15dB to 30dB due to equipment aging or changes in ambient temperature, the system recalculates the average value of the background Shannon entropy with a sliding period of 3600 seconds and corrects the preset mutation threshold η to 3 times the standard deviation of the background Shannon entropy, so that the preset mutation threshold η is kept within the range of 0.10 to 0.80, reducing the impact of sensor hardware drift on the division of dynamic time windows.

[0069] During high-frequency data stream transmission, the system verifies the difference in sampling timestamps. When the number of consecutively missing sampling points does not exceed 5, linear interpolation is performed using the amplitude of the previous sampling point and the amplitude of the next sampling point to fill the missing sampling points, ensuring that the length of the sampling point sequence is consistent with the time base. When the number of consecutively missing sampling points exceeds 5, or the duration of the current dynamic time window reaches the maximum time limit, the system forcibly terminates the current dynamic time window and limits the maximum number of sampling points to between 500 and 50,000. Subsequently, the generated parameter set in the warm storage layer is... The time boundary index is synchronously written to the non-volatile storage medium, enabling the system to recover historical range statistical queries based on the data stored in the warm storage layer after a power failure.

[0070] The preset mutation threshold and the upper limit of the number of sampling points jointly control the division of the dynamic time window. When the preset mutation threshold is lower than 0.10 or the upper limit of the number of sampling points is lower than 500, high-frequency random noise in the environment is more likely to trigger window splitting, which increases the number of generated parameter sets and their indexes in the warm storage layer. When the preset mutation threshold is higher than 0.80 or the upper limit of the number of sampling points is higher than 50,000, some violent mutations cannot form new dynamic time windows in time, and the fitting residuals of the generative parameter model increase accordingly. More queries need to retrieve the residual compensation amount in the cold storage layer for reconstruction.

[0071] Example 6: After the historical range variance statistics query reaches the query routing engine, the system locates the dynamic time window covered by the target time range and reads the corresponding generated parameter set from the warm storage layer composed of solid-state drive media. When the query type is variance calculation, the query routing engine substitutes the generated parameter set into the variance query operation. Based on the calculated value of the baseline trend polynomial at each sampling point, the residual standard deviation, and the residual mean component in the mean vector corresponding to the dimension of the current query data, the model reconstruction variance corresponding to this query type is obtained. When the model reconstruction variance is not greater than the preset query tolerance error threshold, the variance query result is directly parsed in the warm storage layer; when the model reconstruction variance is greater than the preset query tolerance error threshold, the residual compensation amount in the cold storage layer is retrieved, decompressed, and used together with the generated parameter set to reconstruct the query result.

[0072] For dynamic time windows where the model reconstruction variance is no greater than a preset query tolerance error threshold The query routing engine uses baseline trend polynomial coefficients. , and Calculate the trend second moment within the window and combine it with the residual standard deviation. The square and the mean component of the residuals The analytical variance of the temperature storage layer is calculated according to the following formula. ,in, Represents a dynamic time window The number of sampling points within the sequence, where i represents the index of the sampling point sequence. This represents the average resolution of the temperature storage layer. This represents the residual mean component in the mean vector corresponding to the dimension of the currently queried data. Represents the standard deviation of the residuals. This represents the resolution variance of the warm storage layer; the query routing engine uses this resolution variance as a dynamic time window. Output the variance query results.

[0073] When the query type is an extreme value query, the query routing engine extracts the baseline trend polynomial coefficients from the generated parameter set, calculates the first derivative of the baseline trend curve, and finds the sampling positions where the derivative is 0 within the sampling interval corresponding to the dynamic time window. These sampling positions and the two endpoints of the dynamic time window are used as candidate positions. The amplitude of the baseline trend curve at each candidate position is calculated. Then, the residual standard deviation is multiplied by the normal distribution quantile coefficient corresponding to the query probability to obtain the residual fluctuation boundary. This boundary is then superimposed on the baseline trend amplitude at the candidate positions to form the maximum value estimate and the minimum value estimate.

[0074] When the query type is quantile query, the query routing engine reads the baseline trend polynomial coefficients and residual standard deviations from the generated parameter set according to the probability specified in the query request. It multiplies the residual standard deviations by the normal distribution quantile coefficients corresponding to the probability and adds them to the amplitude of the baseline trend curve at the target sampling position to obtain the corresponding quantile estimate. For extreme value queries and quantile queries, the system also calculates the model reconstruction variance of the corresponding query result according to the generated parameter set. When the model reconstruction variance is not greater than the preset query tolerance error threshold, the estimation result is directly output from the warm storage layer. When it exceeds the threshold, the residual compensation amount is retrieved for decompression and reconstruction.

[0075] During the long-term operation of the high-frequency monitoring sensor, the system continuously monitors the residual standard deviation of each dynamic time window. When the average residual standard deviation of 30 consecutive dynamic time windows increases by more than 20% relative to the initial calibration baseline, the system generates a model update signal and retrieves the original time series data of the most recent 1000 dynamic time windows from the hot storage layer during the system's off-peak period. The fully connected mapping layer parameters of the conditional variational encoder neural network are retrained. After training, the low-dimensional latent space projection matrix used to generate the hidden spatial distribution factor in the hot storage layer is updated.

[0076] After updating the low-dimensional latent space projection matrix, the system re-determines the single network connection handshake latency of the cold storage layer object storage medium based on the updated latent space distribution, and measures the single-block decompression time of the system memory for the compressed residual block. The query routing engine recalibrates the preset isomorphic latent variable threshold based on the ratio of the network connection handshake latency to the decompression time. This preset isomorphic latent variable threshold corresponds to the dimensionless Euclidean geometric distance critical value between two points in the two-dimensional latent space coordinate system in the physical dimension. Its calibration logic first determines the single hypertext transfer protocol connection and handshake latency initiated for the cold storage layer object storage medium through network probing. The average network latency in milliseconds was measured, and the average CPU computation time in milliseconds required to decompress a single residual data block using a compression algorithm was measured in system memory. The benefit ratio was obtained by dividing the network latency in milliseconds by the decompression time in milliseconds. When the Euclidean geometric distance between the latent spatial distribution factors of two adjacent dynamic time windows in the latent space was less than the distance threshold, it was determined that the network connection overhead saved by merging and reading the residuals was greater than the additional decompression computation overhead. The distance was adjusted to the range of 0.10 to 0.15 so that the Euclidean state distance determination of subsequent adjacent dynamic time windows was consistent with the updated latent spatial distribution.

[0077] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A query method based on time-series data generative modeling and hierarchical storage, characterized in that, include: Step S1: Acquire a continuous time-series data stream, calculate the information entropy change rate of the sampled segments in the continuous time-series data stream in real time, and define a dynamic time window when the information entropy change rate reaches a preset mutation threshold. Step S2: Construct a generative parametric model for the original time series data within each dynamic time window, extract the generative parameter set containing the mean vector, covariance matrix, baseline trend polynomial coefficients, residual standard deviation and implicit spatial distribution factor and store it in the warm storage layer, and compress and store the residual compensation between the original time series data and the fitted curve of the generative parametric model in the cold storage layer. Step S3: Receive a query request containing the target time range and query type, and calculate the model reconstruction variance of the dynamic time window covered by the target time range in the warm storage layer based on the generated parameter set; Step S4: When the model reconstruction variance is not greater than the preset query tolerance error threshold, the query result is parsed and output in the warm storage layer based on the generated parameter set. Step S5: When the model reconstruction variance is greater than the preset query tolerance error threshold, calculate the Euclidean state distance between the latent spatial distribution factor of the current dynamic time window and the latent spatial distribution factor of the subsequent adjacent dynamic time windows in the warm storage layer. Step S6: When the Euclidean state distance is less than the preset isomorphic latent variable threshold, the physical block address of the residual compensation amount in the cold storage layer for the corresponding dynamic time window is concatenated into a single discrete aggregation read instruction. The residual compensation amount is retrieved by the cold storage layer, decompressed, and reconstructed with the generated parameter set to output the query result. When the Euclidean state distance is not less than the preset isomorphic latent variable threshold, the discrete aggregation read queue is truncated at the corresponding dynamic time window, and a residual compensation amount retrieval instruction for the cold storage layer is issued independently for the current dynamic time window. After decompression and reconstruction, the query result is output.

2. The query method based on time-series data generative modeling and hierarchical storage according to claim 1, characterized in that, Step S1 includes the following sub-steps: Step S11: Divide the data amplitude of the continuous time-series data stream into a preset number of equidistant intervals, and calculate the probability that the data points fall into each equidistant interval within the current sampling time segment; Step S12: Calculate the Shannon entropy of the amplitude distribution of the time series data for the current sampling time segment based on the probability of each equidistant interval. Step S13: Calculate the absolute value of the difference between the Shannon entropy of the amplitude distribution of the time series data in the current sampling time segment and the Shannon entropy of the amplitude distribution of the time series data in the previous sampling time segment. Divide the absolute value by the duration of the sampling time segment to obtain the information entropy change rate.

3. The query method based on time-series data generative modeling and hierarchical storage according to claim 2, characterized in that, The preset mutation threshold for information entropy change rate is set between 0.10 and 0.80, and the maximum number of sampling points in the dynamic time window is set between 500 and 50,000.

4. The query method based on time-series data generative modeling and hierarchical storage according to claim 1, characterized in that, After concatenating the physical block addresses of the residual compensation amount in the cold storage layer corresponding to the dynamic time window into a single discrete aggregate read instruction, step S6 includes the following sub-steps: Step S61: Decompress the residual compensation amount retrieved by the single discrete aggregation read instruction and store it in the system memory buffer pool; In step S62, when reconstructing the output query results for subsequent adjacent dynamic time windows, the corresponding residual compensation amount is directly read from the system memory buffer pool to avoid repeatedly issuing read instructions for the cold storage layer.

5. The query method based on time-series data generative modeling and hierarchical storage according to claim 1, characterized in that, The mean vector, covariance matrix, baseline trend polynomial coefficients, residual standard deviation, and implicit spatial distribution factor of the generated parameter set are all stored in a warm storage layer using a fixed-byte numerical array.

6. The query method based on time-series data generative modeling and hierarchical storage according to claim 1, characterized in that, In step S3, the calculation of the model reconstruction variance of the dynamic time window covered by the target time range based on the generated parameter set is specifically as follows: extract the residual standard deviation from the generated parameter set, divide the square of the residual standard deviation by the number of sampling points within the dynamic time window covered by the target time range, and obtain the model reconstruction variance of the corresponding dynamic time window under the mean query type.

7. The query method based on time-series data generative modeling and hierarchical storage according to claim 1, characterized in that, The warm storage layer is deployed on solid-state drive media, and the cold storage layer is deployed on object storage media; the method also includes storing the latest generated original time-series data in the continuous time-series data stream through a hot storage layer deployed on non-volatile memory media, and the hot storage layer adopts an append-write log structure.

8. The query method based on time-series data generative modeling and hierarchical storage according to claim 1, characterized in that, The preset query tolerance error threshold is set within the range of 0.0001 to 0.2000.

9. A query method based on time-series data generative modeling and hierarchical storage according to claim 4, characterized in that, After the reconstruction of all dynamic time window queries covering the target time range is completed, the residual compensation memory space occupied in the system memory buffer pool is released.

Images