A distributed storage method and device based on a time series database

Abstract

The application discloses a kind of distributed storage method and device based on time series database, it is related to data storage technical field;Original time series data is obtained to carry out time clustering segmentation and obtain time series data segment set;Time series data segment in time series data segment set is carried out data merging and obtains merging data segment set;The storage location of each merging data segment in merging data segment set is determined;Time query index is constructed to the data in each merging data segment;For the data in each merging data segment, the storage address of the data is obtained, and the storage address is corresponded with the time query index of the data, and the data is saved in the storage address.Time series data is clustered and then segmented, and then the segmented data is merged, to reduce storage fragmentation, and finally by constructing query tree to associate storage address and time query index, when facing mass data, efficient storage is still satisfied, to improve data management efficiency.

Description

Technical Field

[0001] This invention belongs to the field of data storage technology, specifically relating to a distributed storage method and apparatus based on a time-series database. Background Technology

[0002] With the rapid development of fields such as the Internet of Things, financial transactions, and industrial monitoring, massive amounts of time-series data are experiencing explosive growth. Time-series data records data points in chronological order, characterized by unique timestamps, strong real-time requirements, large data volume, and rapid growth. Traditional relational databases suffer from low storage efficiency and poor query performance when dealing with this type of data, making it difficult to meet the needs of efficient storage, writing, and querying of massive amounts of time-series data.

[0003] Existing distributed storage solutions for time-series databases mostly employ simple data sharding strategies, dividing and storing data according to time ranges or device identifiers. However, in practical applications, this approach easily leads to uneven data writes, with some nodes experiencing excessive load and severely impacting write efficiency. Simultaneously, uneven query load distribution results in poor query performance. Furthermore, deficiencies in data compression, consistency maintenance, and fault recovery significantly limit the efficient management of large-scale time-series data.

[0004] Patent CN108268217A discloses a hierarchical storage method based on hot and cold classification of time-series data. It establishes a UID system to uniformly identify time-series data; it categorizes data into real-time data, weekly data, historical data, and cold data according to specific time rules, establishing real-time data layers, weekly data layers, historical data layers, and cold data layers, which are stored sequentially in memory, SSD storage, HDD storage, and the distributed storage system HDFS, respectively; and it establishes a unified data insertion and query interface, routing query requests to the appropriate storage query interface based on the query time range, providing different storage access for different data. However, this solution still cannot achieve efficient storage when dealing with massive amounts of time-series data, resulting in low data management efficiency. Summary of the Invention

[0005] The purpose of this invention is to solve the problem that when dealing with the storage of massive amounts of time-series data, it is still impossible to meet the requirements of efficient storage, resulting in low data management efficiency. Therefore, this invention proposes a distributed storage method and device based on time-series databases.

[0006] In a first aspect of this invention, a distributed storage method based on a time-series database is proposed, the method comprising: Obtain the original time series data, and perform time clustering to segment the original time series data to obtain a set of time series data segments; each time series data segment corresponds to a unique block index; The time series data segments in the time series data segment set are merged to obtain a merged data segment set; Determine the storage location of each merged data segment in the merged data segment set; Build a time-based query index for the data within each merged data segment; For each data segment in the merged data, obtain the storage address of the data, match the storage address with the time query index of the data, and save the data to the storage address.

[0007] Optionally, performing time clustering to segment the original time series data to obtain a set of time series data segments includes: Step S11: The original time series data is segmented using multiple sets of random breakpoints to obtain multiple sets of data segments; the random breakpoint sets include a preset number of breakpoint times; Step S12: Generate a chromosome for each set of data fragments, calculate the empirical mean and empirical covariance of the corresponding fragments for each chromosome, and determine the total score of the chromosome based on the empirical mean and empirical covariance. Step S13: Determine the common breakpoints of the chromosome ranked first and second in total score to obtain the effective breakpoint set; Step S14: Randomly generate breakpoint times to supplement the effective breakpoint set to obtain multiple updated random breakpoint sets; the updated random breakpoint sets have the same number of breakpoints as the random breakpoint sets. Step S15: Update multiple sets of random breakpoints according to multiple sets of updated random breakpoints, repeat steps S11 to S14 until the preset number of loops is reached, and obtain the data fragment set corresponding to the chromosome with the highest total score as the time series data fragment set.

[0008] Optionally, merging the time series data segments in the time series data segment set to obtain a merged data segment set includes: Step S21: For each time series data segment in the time series data segment set, calculate the similarity between adjacent segments; Step S22: If the similarity is greater than the preset similarity threshold, then the adjacent time series data segments are recorded as candidate data segment groups; Step S23: Calculate the seek time reduction and merging / compression cost for each candidate data segment group to obtain the target benefit; Step S24: If the target return is greater than the preset return threshold, then merge adjacent time series data segments in the candidate data segment group; Step S25: Update the data in the time series data segment set according to the merged data, return to step S21, until the preset conditions are met, and obtain all merged time series data segments to obtain the merged data segment set.

[0009] Optionally, step S23 includes: Through formula Achieve the target return; in, The reduction in seek time is represented by N, where N is the original number of fragments and N0 is the number of fragments after merging. To reduce the cost of merging, Z represents the original data volume, and Z0 represents the compressed data volume after merging. and is the weighting coefficient, and C is the target return.

[0010] Optionally, determining the storage location of each merged data segment in the merged data segment set includes: Acquire existing storage node resources; the storage node resources include storage location and storage capacity; Extract data features from the target data segment; the target data segment is any one of the merged data segments in the merged data segment set; the data features include data size, access frequency, and real-time requirements; Substitute the data features and the existing storage node resources into a preset decision tree model to obtain a candidate storage node set; For each candidate node in the candidate storage node set, calculate the fit score between the candidate node and the target data segment, and obtain the candidate node with the highest fit score as the storage location of the target data segment.

[0011] In a second aspect of the invention, a distributed storage device based on a time-series database is provided, comprising: The data segmentation module is used to acquire raw time series data, perform time clustering on the raw time series data to obtain a set of time series data segments; each time series data segment corresponds to a unique block index. The data merging module is used to merge time series data segments in the time series data segment set to obtain a merged data segment set; A storage location determination module is used to determine the storage location of each merged data segment in the merged data segment set; The query index building module is used to build a time query index for the data within each merged data segment; The data storage module is used to obtain the storage address of the data in each merged data segment, match the storage address with the time query index of the data, and save the data to the storage address.

[0012] Optionally, the data segmentation module includes: The data segmentation module is used to segment the original time series data into multiple sets of data segments by using multiple sets of random breakpoints; the random breakpoint sets include a preset number of breakpoint times; The total score calculation module is used to generate a chromosome for each set of data segments, calculate the empirical mean and empirical covariance of the corresponding segments for each chromosome, and determine the total score of the chromosome based on the empirical mean and empirical covariance. The effective breakpoint set determination module is used to determine the common breakpoints of the chromosome ranked first and second in total score to obtain the effective breakpoint set; The random breakpoint set update module is used to randomly generate breakpoint times to supplement the valid breakpoint set and obtain multiple updated random breakpoint sets; the number of breakpoints in the updated random breakpoint sets is the same as the number of breakpoints in the random breakpoint sets. The time series data fragment set generation module is used to update multiple sets of random breakpoint sets according to multiple sets of updated random breakpoint sets, repeat the data segmentation module to the random breakpoint set update module until a preset number of loops is reached, and obtain the data fragment set corresponding to the chromosome with the highest total score as the time series data fragment set.

[0013] Optionally, the data merging module includes: The similarity calculation module is used to calculate the similarity between adjacent segments for each time series data segment in the time series data segment set; The candidate data segment group determination module is used to record adjacent time series data segments as candidate data segment groups if the similarity is greater than a preset similarity threshold. The target benefit determination module is used to calculate the reduction in seek time and the merging and compression cost for each candidate data segment group to obtain the target benefit. The time series data segment merging module is used to merge adjacent time series data segments in the candidate data segment group if the target return is greater than a preset return threshold. The merged data fragment set generation module is used to update the data in the time series data fragment set according to the merged data, return to the similarity calculation module, until the preset conditions are met, and obtain all merged time series data fragments to obtain the merged data fragment set.

[0014] Optionally, the target return determination module includes: Through formula Achieve the target return; in, The reduction in seek time is represented by N, where N is the original number of fragments and N0 is the number of fragments after merging. To reduce the cost of merging, Z represents the original data volume, and Z0 represents the compressed data volume after merging. and is the weighting coefficient, and C is the target return.

[0015] Optionally, the storage location determination module includes: The storage node resource determination module is used to obtain existing storage node resources; the storage node resources include storage location and storage capacity. The data feature extraction module is used to extract data features of the target data segment; the target data segment is any one of the merged data segments in the merged data segment set; the data features include data size, access frequency, and real-time requirements; The candidate storage node set determination module is used to substitute the data features and the existing storage node resources into a preset decision tree model to obtain a candidate storage node set. The storage location generation module is used to calculate the fit score between each candidate node in the candidate storage node set and the target data segment, and obtain the candidate node with the highest fit score as the storage location of the target data segment.

[0016] The beneficial effects of this invention are: This invention proposes a distributed storage method based on a time-series database. The method involves acquiring raw time-series data, performing time clustering to segment the data into time-series data fragment sets, and assigning a unique block index to each fragment. The time-series data fragments in the fragment sets are then merged to obtain a merged data fragment set. The storage location of each merged data fragment is determined, and a time-based query index is constructed for the data within each merged data fragment. For each data fragment, its storage address is obtained, and this address is mapped to its time-based query index before being saved to that address. By clustering and segmenting the time-series data, the data storage structure better aligns with the data generation logic. Compared to traditional methods of sharding data within fixed time ranges, this method more accurately captures the temporal distribution patterns of the data. Merging the sharded data reduces storage fragmentation, and the merged data fragments are evenly distributed across storage nodes, avoiding continuous storage at the same address. Finally, a query tree is constructed to associate the storage addresses and time-based query indexes, ensuring efficient storage even with massive amounts of data and improving data management efficiency. Attached Figure Description

[0017] The invention will now be further described with reference to the accompanying drawings.

[0018] Figure 1 A flowchart illustrating a distributed storage method based on a time-series database, provided as an embodiment of the present invention; Figure 2A flowchart of a time clustering segmentation method provided in an embodiment of the present invention; Figure 3 A flowchart illustrating a method for merging time series data segments provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a distributed storage device based on a time-series database, provided as an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0020] Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] This invention provides a distributed storage method based on a time-series database. See also... Figure 1 , Figure 1 A flowchart illustrating a distributed storage method based on a time-series database, provided as an embodiment of the present invention. The method includes the following steps: S101, Obtain the original time series data, and perform time clustering segmentation on the original time series data to obtain a set of time series data segments; S102, merge the time series data segments in the time series data segment set to obtain a merged data segment set; S103, determine the storage location of each merged data segment in the merged data segment set; S104, Build a time query index for the data within each merged data segment; S105: For each data segment in the merged data segment, obtain the storage address of the data, match the storage address with the time query index of the data, and save the data to the storage address.

[0022] Each time series data segment corresponds to a unique block index; Based on the distributed storage method for time-series databases provided in this invention, the data is clustered and segmented to make the data storage structure more closely match the data generation logic. Compared with the traditional method of sharding according to a fixed time range, it can more accurately capture the time distribution pattern of data. The sharded data is then merged to reduce storage fragmentation. The merged data fragments are evenly distributed to each storage node, and the same address is no longer continuously stored. Finally, a query tree is constructed to associate the storage address and the time query index, so that efficient storage can still be achieved when facing massive amounts of data, thus improving data management efficiency.

[0023] In one implementation, before acquiring the raw time series data, all the time data to be collected is acquired first, and these time data are preprocessed (using existing common data preprocessing methods) to remove invalid data.

[0024] In one implementation, time clustering segmentation can divide data with similar time patterns into the same segment, making it easier to determine the storage location for different segments. Each segment corresponds to a unique block index, forming a structured data organization method, which facilitates quick location and management of data in a specific time period and reduces the complexity of data retrieval.

[0025] In one implementation, merging time series segments can integrate adjacent or similar data segments into larger logical units, reducing data fragmentation. During the merging process, data deduplication or redundancy filtering can be performed simultaneously to improve data quality and reduce invalid storage. After merging data segments, the data is stored in larger logical units, which can make full use of the block space of the storage medium and reduce the waste of fragmented space.

[0026] In one implementation, a time query index (B-tree, hash index, or time range index, etc.) is built for each merged data fragment. This allows for quick location of data storage directly via timestamp or time range, avoiding full scans. This transforms time-dimensional query operations into direct access to physical storage locations, reducing intermediate conversion overhead.

[0027] In one embodiment, see Figure 2 , Figure 2 The present invention provides a flowchart of a time clustering segmentation method, which specifically includes: The time series data fragment set obtained by performing time clustering on the original time series data includes: S11: The original time series data is segmented by multiple sets of random breakpoints to obtain multiple sets of data segments; S12: Generate a chromosome for each set of data fragments, calculate the empirical mean and empirical covariance of the corresponding fragments for each chromosome, and determine the total score of the chromosome based on the empirical mean and empirical covariance. S13: Determine the common breakpoints of the chromosome ranked first and second in total score to obtain the effective breakpoint set; S14: Randomly generate breakpoint times to supplement the valid breakpoint set and obtain multiple sets of updated random breakpoint sets; S15: Update multiple sets of random breakpoints according to multiple sets of updated random breakpoints, repeat steps S11 to S14 until the preset number of loops is reached, and obtain the data fragment set corresponding to the chromosome with the highest total score as the time series data fragment set. The random breakpoint set includes a preset number of breakpoint times; the number of breakpoints in the updated random breakpoint set is the same as the number of breakpoints in the random breakpoint set.

[0028] In one implementation, the original data is segmented by multiple sets of random breakpoints, which avoids the limitations of a single fixed breakpoint and covers the possibility of segmentation in different time intervals and different modes. The total score is calculated based on the empirical mean and empirical covariance of the segments, which can quantify the rationality of the segmentation from a statistical perspective.

[0029] In one implementation, the common breakpoints of the two chromosomes with the highest total scores are selected as the effective breakpoint set. A consensus mechanism is used to filter out the influence of random noise. The existence of common breakpoints indicates that these breakpoints can improve the segmentation quality in different random segmentation schemes. The breakpoint set is updated by iterative generation, so that the breakpoint set gradually converges to the true feature points of the data. After the segmentation results are filtered by the scores, the data features within each data segment are more consistent, which is convenient for subsequent analysis.

[0030] In one implementation, the preset breakpoint times are determined by technical personnel.

[0031] In one implementation, for each chromosome corresponding to segment X... i Calculate the empirical mean and experience covariance C i Through formula The total score is calculated, where F is the total score. =|X i | represents the segment length, tr() is the matrix trace operation, x is a segment in a chromosome, λ is the regularization parameter determined by the technician, and I is the identity matrix.

[0032] In one implementation, the randomness of the initial breakpoint and the random supplementation during the iteration process make the algorithm independent of the local features of specific data samples. Based on the scoring and ranking mechanism of multiple chromosomes, the algorithm makes the final segmentation result statistically reliable through group optimization rather than single-solution decision-making.

[0033] In one implementation, the generation of multiple sets of random breakpoints and the calculation of scores can be processed in parallel, improving the processing efficiency of large-scale time series data.

[0034] In one embodiment, see Figure 3 , Figure 3 The present invention provides a flowchart of a method for merging time series data segments, specifically including: S21: For each time series data segment in the time series data segment set, calculate the similarity between adjacent segments; S22: If the similarity is greater than the preset similarity threshold, then the adjacent time series data segments are recorded as candidate data segment groups; S23: Calculate the reduction in seek time and the merging and compression cost for each candidate data segment group to obtain the target benefit; S24: If the target return is greater than the preset return threshold, then merge adjacent time series data segments in the candidate data segment group; S25: Update the data in the time series data segment set according to the merged data, return to step S21, until the preset conditions are met, and obtain all merged time series data segments to obtain the merged data segment set.

[0035] In one implementation, merging adjacent segments with high similarity can eliminate duplicate or highly similar data, reducing storage requirements. The merged data segments may exhibit stronger regularity, thereby improving the compression ratio of the compression algorithm and further reducing storage space.

[0036] In one implementation, the similarity between adjacent segments can be determined by calculating Euclidean distance, Manhattan distance, Pearson correlation coefficient, autoencoder, etc.

[0037] In one implementation, the number of merged fragments is reduced, and the number of data blocks that need to be read during a query is reduced, thereby reducing disk seek time and speeding up the query. The merged data fragments are more independent, which facilitates parallel querying or distributed processing and improves system throughput.

[0038] In one implementation, after similar segments are merged, subsequent analysis only needs to process a smaller amount of data. By continuously optimizing data segments through iterative loops, the system can maintain efficient operation under different data distributions.

[0039] In one implementation, the preset similarity threshold, revenue threshold, and preset conditions (number of merges, revenue increase threshold) are determined by technical personnel to avoid information loss or waste of computing resources.

[0040] In one embodiment, step S23 includes: Through formula Achieve the target return; in, The reduction in seek time is represented by N, where N is the original number of fragments and N0 is the number of fragments after merging. To reduce the cost of merging, Z represents the original data volume, and Z0 represents the compressed data volume after merging. and denoted as a weighting coefficient, and C represents the target return.

[0041] In one implementation, the improvement in physical storage continuity is measured by the difference between the original number of fragments (N) and the merged number of fragments (N0); the difference between the original data volume (Z) and the merged and compressed data volume (Z0) quantifies the savings in storage and transmission costs. and The weighting is adjusted to suit the priorities of different business scenarios; focusing on storage costs can increase... Focusing on I / O performance can increase The specific values ​​will be determined by technical personnel. and The sum is 1.

[0042] In one embodiment, determining the storage location of each merged data fragment in the merged data fragment set includes: Acquire existing storage node resources; storage node resources include storage location and storage capacity. Extract the data features of the target data segment; the target data segment is any one of the merged data segments in the merged data segment set; the data features include data size, access frequency, and real-time requirements; Substitute the data characteristics and existing storage node resources into a pre-defined decision tree model to obtain a set of candidate storage nodes; For each candidate node in the candidate storage node set, calculate the fit score between the candidate node and the target data segment, and select the candidate node with the highest fit score as the storage location of the target data segment.

[0043] In one implementation, nodes with sufficient remaining capacity are matched with storage nodes based on the amount of data in the target data segment to prevent small data from occupying a large amount of space. In a distributed storage system, large files are preferentially stored in high-capacity nodes, and small files are stored in low-capacity nodes, thereby improving space utilization.

[0044] In one implementation, the data characteristics of the target data segment (data size, access frequency, real-time requirements, etc.) are determined by computer statistics.

[0045] In one implementation, data with high real-time requirements is preferentially allocated to storage nodes in the same rack or data center as the computing nodes to reduce network RTT; the decision tree model can simultaneously handle storage location, storage capacity and access frequency to replace manual rule configuration; traditional manual configuration requires writing rules for each data type, while the decision tree can automatically learn the optimal mapping relationship through training, improving configuration efficiency; the preset decision tree model is obtained by training through historical data.

[0046] In one implementation, the fit score between the candidate node and the target data segment is calculated using an SVM model. The fit score is the probability that the SVM model is suitable for the node to store a specific data segment. The higher the score, the more suitable the node is for storing the data segment.

[0047] In one implementation, if multiple data segments use the same storage node, the segment with the highest adaptation score is stored in that node, and the storage location of the remaining data segments is recalculated.

[0048] Based on the same inventive concept, this invention also provides a distributed storage device based on a time-series database. See also Figure 4 , Figure 4 A schematic diagram of a distributed storage device based on a time-series database provided in an embodiment of the present invention includes: The data segmentation module is used to acquire raw time series data, perform time clustering on the raw time series data to obtain a set of time series data segments; each time series data segment corresponds to a unique block index. The data merging module is used to merge time series data segments in a time series data segment set to obtain a merged data segment set. The storage location determination module is used to determine the storage location of each merged data segment in the merged data segment set; The query index building module is used to build a time query index for the data within each merged data segment; The data storage module is used to obtain the storage address of the data in each merged data segment, match the storage address with the time query index of the data, and save the data to the storage address.

[0049] Based on the distributed storage device based on time-series database provided by the embodiments of the present invention, the data storage structure is more in line with the data generation logic by clustering and segmenting the time-series data. Compared with the traditional method of sharding according to a fixed time range, it can more accurately capture the time distribution pattern of data. Then, the sharded data is merged to reduce storage fragmentation. The merged data fragments are evenly distributed to each storage node, and the same address is no longer continuously stored. Finally, by constructing a query tree to associate the storage address and the time query index, it can still meet the requirements of efficient storage when facing massive data, and improve the data management efficiency.

[0050] In one embodiment, the data segmentation module includes: The data segmentation module is used to segment the original time series data into multiple sets of data segments by using multiple sets of random breakpoints; the random breakpoint sets include a preset number of breakpoint times; The total score calculation module is used to generate a chromosome for each set of data segments, calculate the empirical mean and empirical covariance of the corresponding segments for each chromosome, and determine the total score of the chromosome based on the empirical mean and empirical covariance. The effective breakpoint set determination module is used to determine the common breakpoints of the chromosome ranked first and second in total score to obtain the effective breakpoint set; The random breakpoint set update module is used to randomly generate breakpoint times to supplement the valid breakpoint set and obtain multiple sets of updated random breakpoint sets; the number of breakpoints in the updated random breakpoint set is the same as that in the random breakpoint set. The time series data fragment set generation module is used to update multiple sets of random breakpoint sets based on multiple sets of updated random breakpoint sets, from the repeated data segmentation module to the random breakpoint set update module, until a preset number of cycles is reached, and obtain the data fragment set corresponding to the chromosome with the highest total score as the time series data fragment set.

[0051] In one embodiment, the data merging module includes: The similarity calculation module is used to calculate the similarity between adjacent segments for each time series data segment in the time series data segment set; The candidate data segment group determination module is used to record adjacent time series data segments as candidate data segment groups if the similarity is greater than a preset similarity threshold. The target benefit determination module is used to calculate the reduction in seek time and the merging and compression cost for each candidate data segment group to obtain the target benefit. The time series data segment merging module is used to merge adjacent time series data segments in the candidate data segment group if the target return is greater than a preset return threshold. The merged data fragment set generation module is used to update the data in the time series data fragment set based on the merged data, return to the similarity calculation module, and continue until the preset conditions are met to obtain all merged time series data fragments to obtain the merged data fragment set.

[0052] In one embodiment, the target return determination module includes: Through formula Achieve the target return; in, The reduction in seek time is represented by N, where N is the original number of fragments and N0 is the number of fragments after merging. To reduce the cost of merging, Z represents the original data volume, and Z0 represents the compressed data volume after merging. and denoted as a weighting coefficient, and C represents the target return.

[0053] In one embodiment, the storage location determination module includes: The storage node resource determination module is used to obtain existing storage node resources; storage node resources include storage location and storage capacity. The data feature extraction module is used to extract data features from the target data segment; the target data segment is any one of the merged data segments in the merged data segment set; the data features include data size, access frequency, and real-time requirements; The candidate storage node set determination module is used to input data features and existing storage node resources into a preset decision tree model to obtain a candidate storage node set. The storage location generation module is used to calculate the fit score between each candidate node in the candidate storage node set and the target data segment, and to obtain the candidate node with the highest fit score as the storage location of the target data segment.

[0054] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A distributed storage method based on a time-series database, characterized in that, The method includes: Obtain the original time series data, and perform time clustering to segment the original time series data to obtain a set of time series data segments; each time series data segment corresponds to a unique block index; The time series data segments in the time series data segment set are merged to obtain a merged data segment set; Determine the storage location of each merged data segment in the merged data segment set; Build a time-based query index for the data within each merged data segment; For each data segment in the merged data, obtain the storage address of the data, match the storage address with the time query index of the data, and save the data to the storage address; The time series data fragment set obtained by performing time clustering on the original time series data includes: Step S11: The original time series data is segmented using multiple sets of random breakpoints to obtain multiple sets of data segments; the random breakpoint sets include a preset number of breakpoint times; Step S12: Generate a chromosome for each set of data fragments, calculate the empirical mean and empirical covariance of the corresponding fragments for each chromosome, and determine the total score of the chromosome based on the empirical mean and empirical covariance. Step S13: Determine the common breakpoints of the chromosome ranked first and second in total score to obtain the effective breakpoint set; Step S14: Randomly generate breakpoint times to supplement the effective breakpoint set to obtain multiple updated random breakpoint sets; the updated random breakpoint sets have the same number of breakpoints as the random breakpoint sets. Step S15: Update multiple sets of random breakpoints according to multiple sets of updated random breakpoints, repeat steps S11 to S14 until the preset number of loops is reached, and obtain the data fragment set corresponding to the chromosome with the highest total score as the time series data fragment set.

2. The distributed storage method based on a time-series database according to claim 1, characterized in that, Merging time series data segments in the aforementioned time series data segment set to obtain a merged data segment set includes: Step S21: For each time series data segment in the time series data segment set, calculate the similarity between adjacent segments; Step S22: If the similarity is greater than the preset similarity threshold, then the adjacent time series data segments are recorded as candidate data segment groups; Step S23: Calculate the seek time reduction and merging / compression cost for each candidate data segment group to obtain the target benefit; Step S24: If the target return is greater than the preset return threshold, then merge adjacent time series data segments in the candidate data segment group; Step S25: Update the data in the time series data segment set according to the merged data, return to step S21, until the preset conditions are met, and obtain all merged time series data segments to obtain the merged data segment set.

3. The distributed storage method based on a time-series database according to claim 2, characterized in that, Step S23 includes: Through formula Achieve the target return; in, The reduction in seek time is represented by N, where N is the original number of fragments and N0 is the number of fragments after merging. To reduce the cost of merging, Z represents the original data volume, and Z0 represents the compressed data volume after merging. and is the weighting coefficient, and C is the target return.

4. The distributed storage method based on a time-series database according to claim 1, characterized in that, Determining the storage location of each merged data segment in the merged data segment set includes: Acquire existing storage node resources; the storage node resources include storage location and storage capacity; Extract data features from the target data segment; the target data segment is any one of the merged data segments in the merged data segment set; the data features include data size, access frequency, and real-time requirements; Substitute the data features and the existing storage node resources into a preset decision tree model to obtain a candidate storage node set; For each candidate node in the candidate storage node set, calculate the fit score between the candidate node and the target data segment, and obtain the candidate node with the highest fit score as the storage location of the target data segment.

5. A distributed storage device based on a time-series database, characterized in that, The device includes: The data segmentation module is used to acquire raw time series data, perform time clustering on the raw time series data to obtain a set of time series data segments; each time series data segment corresponds to a unique block index. The data merging module is used to merge time series data segments in the time series data segment set to obtain a merged data segment set; A storage location determination module is used to determine the storage location of each merged data segment in the merged data segment set; The query index building module is used to build a time query index for the data within each merged data segment; The data storage module is used to obtain the storage address of each data segment within the merged data segment, match the storage address with the time query index of the data, and save the data to the specified storage address. The data segmentation module includes: The data segmentation module is used to segment the original time series data into multiple sets of data segments by using multiple sets of random breakpoints; the random breakpoint sets include a preset number of breakpoint times; The total score calculation module is used to generate a chromosome for each set of data segments, calculate the empirical mean and empirical covariance of the corresponding segments for each chromosome, and determine the total score of the chromosome based on the empirical mean and empirical covariance. The effective breakpoint set determination module is used to determine the common breakpoints of the chromosome ranked first and second in total score to obtain the effective breakpoint set; The random breakpoint set update module is used to randomly generate breakpoint times to supplement the valid breakpoint set and obtain multiple updated random breakpoint sets; the updated random breakpoint sets have the same number of breakpoints as the random breakpoint sets. The time series data fragment set generation module is used to update multiple sets of random breakpoint sets according to multiple sets of updated random breakpoint sets, repeating the data segmentation module to the random breakpoint set update module until a preset number of loops is reached, and obtaining the data fragment set corresponding to the chromosome with the highest total score as the time series data fragment set.

6. A distributed storage device based on a time-series database according to claim 5, characterized in that, The data merging module includes: The similarity calculation module is used to calculate the similarity between adjacent segments for each time series data segment in the time series data segment set; The candidate data segment group determination module is used to record adjacent time series data segments as candidate data segment groups if the similarity is greater than a preset similarity threshold. The target benefit determination module is used to calculate the reduction in seek time and the merging and compression cost for each candidate data segment group to obtain the target benefit. The time series data segment merging module is used to merge adjacent time series data segments in the candidate data segment group if the target return is greater than a preset return threshold. The merged data fragment set generation module is used to update the data in the time series data fragment set according to the merged data, return to the similarity calculation module, until the preset conditions are met, and obtain all merged time series data fragments to obtain the merged data fragment set.

7. A distributed storage device based on a time-series database according to claim 6, characterized in that, The target return determination module includes: Through formula Achieve the target return; in, The reduction in seek time is represented by N, where N is the original number of fragments and N0 is the number of fragments after merging. To reduce the cost of merging, Z represents the original data volume, and Z0 represents the compressed data volume after merging. and is the weighting coefficient, and C is the target return.

8. A distributed storage device based on a time-series database according to claim 5, characterized in that, The storage location determination module includes: The storage node resource determination module is used to obtain existing storage node resources; the storage node resources include storage location and storage capacity. The data feature extraction module is used to extract data features of the target data segment; the target data segment is any one of the merged data segments in the merged data segment set; the data features include data size, access frequency, and real-time requirements; The candidate storage node set determination module is used to substitute the data features and the existing storage node resources into a preset decision tree model to obtain a candidate storage node set. The storage location generation module is used to calculate the fit score between each candidate node in the candidate storage node set and the target data segment, and obtain the candidate node with the highest fit score as the storage location of the target data segment.

Images