A time series data synchronization method and device, electronic equipment and storage medium

Abstract

The application discloses a time series data synchronization method and device, electronic equipment and storage medium. The method comprises the following steps: acquiring a to-be-synchronized time series data set, dividing the to-be-synchronized time series data set into a plurality of to-be-synchronized sub-data sets according to the index dimension of the time series data; dividing the to-be-synchronized sub-data sets into a plurality of segmented data sets belonging to different time periods according to the collection time range of the to-be-synchronized sub-data sets; and performing data synchronization on each segmented data set of each to-be-synchronized sub-data set respectively, so as to realize synchronization of the to-be-synchronized time series data set. The application solves the technical problems that the time series data synchronization is prone to errors and the reliability is difficult to be ensured in the prior art.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and specifically to a time-series data synchronization method, apparatus, electronic device, and storage medium. Background Technology

[0002] The log data of the bank's monitoring system is mostly stored in time-series databases. Time-series databases have insufficient support for application-related queries, low performance, poor scalability, and can only support a small number of data sources, a small number of data source collection indicators, and a small number of tags. They cannot adapt to the bank's high-frequency collection, storage, calculation, and application of a large number of indicators from a large number of data sources.

[0003] To address the aforementioned shortcomings, a storage and query optimization scheme based on ClickHouse has been proposed, which improves the performance of aggregation queries involving multiple indexes. However, when migrating time-series data to ClickHouse using tools, the large volume of historical data makes the synchronization process prone to errors, compromising data reliability. Summary of the Invention

[0004] The purpose of this invention is to overcome the above-mentioned technical deficiencies and provide a time-series data synchronization method, apparatus, electronic device and storage medium that can be used in financial technology or other related technical fields, solving the technical problems of easy errors and difficulty in guaranteeing reliability in the prior art when synchronizing time-series data.

[0005] To achieve the above-mentioned technical objectives, the present invention adopts the following technical solution:

[0006] In a first aspect, the present invention provides a time-series data synchronization method, comprising the following steps:

[0007] Obtain the time series dataset to be synchronized, and divide the time series dataset to be synchronized into multiple sub-datasets to be synchronized according to the indicator dimensions of the time series data;

[0008] Based on the collection time range of the sub-dataset to be synchronized, the sub-dataset to be synchronized is divided into multiple segmented datasets belonging to different time periods;

[0009] Data synchronization is performed on each segment of the dataset to be synchronized in order to achieve synchronization of the time series dataset.

[0010] In some embodiments, obtaining the time-series dataset to be synchronized involves dividing the time-series dataset to be synchronized into multiple sub-datasets to be synchronized based on the metric dimensions of the time-series data, including:

[0011] Obtain the time series dataset to be synchronized, and determine the indicator dimensions of each time series data in the time series dataset to be synchronized;

[0012] Based on the indicator dimensions of each time series data, the time series dataset to be synchronized is divided to obtain multiple sub-datasets to be synchronized with rates belonging to different indicator dimensions.

[0013] In some embodiments, dividing the dataset to be synchronized into multiple segmented datasets belonging to different time periods according to the collection time range of the dataset to be synchronized includes:

[0014] Obtain the time collection range of the sub-dataset to be synchronized;

[0015] The time acquisition range is divided into multiple time periods according to preset time division levels;

[0016] Based on the defined time periods, the dataset to be synchronized is divided into multiple segmented datasets belonging to different time periods.

[0017] In some embodiments, the step of synchronizing each segment of the dataset in each sub-dataset to be synchronized includes:

[0018] Data synchronization is performed on each segment of the dataset to be synchronized, and a fixed time label is generated for each time series data during the synchronization process.

[0019] In some embodiments, the time-series data synchronization method further includes:

[0020] If any of the datasets to be synchronized fails to synchronize, the synchronization task for that dataset will be executed again after a preset interval, until the dataset is successfully synchronized or the number of executions exceeds the preset number.

[0021] In some embodiments, the time-series data synchronization method further includes:

[0022] When the data retrieval of any segment dataset of any sub-dataset to be synchronized times out, the preset time division level is reduced, the time collection range is divided according to the reduced time division level, and the sub-dataset to be synchronized is divided into multiple segment datasets according to the newly divided time period. Data synchronization is performed on multiple segment datasets until the data retrieval of each segment dataset is successful.

[0023] In some embodiments, when the data retrieval of the segmented dataset divided into time periods according to the smallest time division level fails, the segmented dataset to be synchronized is marked and automatic synchronization is stopped.

[0024] Secondly, the present invention also provides a timing data synchronization device, comprising:

[0025] The first partitioning module is used to obtain the time series dataset to be synchronized and to divide the time series dataset to be synchronized into multiple sub-datasets to be synchronized according to the indicator dimensions of the time series data.

[0026] The second partitioning module is used to divide the dataset to be synchronized into multiple segmented datasets belonging to different time periods according to the collection time range of the dataset to be synchronized.

[0027] The synchronization module is used to synchronize the data of each segment of the dataset to be synchronized, so as to realize the synchronization of the time series dataset.

[0028] Thirdly, the present invention also provides an electronic device, comprising: a processor and a memory;

[0029] The memory stores computer programs that can be executed by the processor;

[0030] When the processor executes the computer program, it implements the steps in the timing data synchronization method described above.

[0031] Fourthly, the present invention also provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the time-series data synchronization method described above.

[0032] Compared with existing technologies, the time-series data synchronization method, apparatus, electronic device, and storage medium provided by this invention first acquire the time-series dataset to be synchronized. Based on the indicator dimensions of the time-series data, the dataset is divided into multiple sub-datasets to be synchronized. Then, based on the collection time range of each sub-dataset, it is further divided into multiple segmented datasets belonging to different time periods. Finally, data synchronization is performed on each segmented dataset of each sub-dataset to achieve synchronization of the time-series dataset. By dividing the time-series dataset to be synchronized according to indicator dimensions, and then further dividing the resulting sub-datasets according to time periods, the entire time-series dataset is divided into multiple smaller datasets before synchronization, reducing the probability of problems during the synchronization process and ensuring data reliability. Attached Figure Description

[0033] Figure 1 This is a flowchart of the timing data synchronization method provided in the embodiments of the present invention;

[0034] Figure 2 This is a flowchart of step S100 in the time-series data synchronization method provided in the embodiments of the present invention;

[0035] Figure 3This is a flowchart of step S200 in the time-series data synchronization method provided in the embodiments of the present invention;

[0036] Figure 4 This is a schematic diagram of the functional modules of the timing data synchronization device provided in an embodiment of the present invention;

[0037] Figure 5 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0039] Please see Figure 1 , Figure 1 This is a flowchart illustrating a time-series data synchronization method provided by an embodiment of the present invention. The time-series data synchronization method involved in this invention can be used in time-series data synchronization in the banking and other financial fields to optimize the time-series data synchronization process and improve the reliability of synchronized data. The time-series data synchronization method involved in this invention can be executed by an electronic device capable of receiving or sending data. This electronic device can be various electronic devices with a display screen and supporting web browsing, including but not limited to smartphones, tablets, portable computers, and desktop servers. Figure 1 As shown, the method specifically includes the following steps S100 to S300.

[0040] S100. Obtain the time series dataset to be synchronized, and divide the time series dataset to be synchronized into multiple sub-datasets to be synchronized according to the indicator dimensions of the time series data.

[0041] In this embodiment, when migrating the time-series dataset to be synchronized to ClickHouse (a columnar database management system for online analytical processing), to avoid data migration errors due to an excessively large dataset, which could affect the reliability of the synchronized data, the dataset is divided into multiple sub-datasets according to the metric dimension. This prevents data migration errors caused by an excessively large single migration task. Specifically, this embodiment divides the entire migration task into multiple sub-tasks according to the metric dimension, with each sub-task synchronizing only the data for one metric.

[0042] Among them, metrics are the basic units of time-series monitoring data. Subtasks and metrics are one-to-one. Therefore, when a subtask is abnormal, it will be marked, and the specific time-series data of the erroneous metric can be quickly located, making it easy to troubleshoot. Moreover, each subtask is independent of the others and does not affect each other. Therefore, when one or more subtasks fail to migrate, it will not block the data migration of other subtasks, thus avoiding poor data reliability due to excessive data volume.

[0043] S200. Based on the collection time range of the sub-dataset to be synchronized, the sub-dataset to be synchronized is divided into multiple segmented datasets belonging to different time periods.

[0044] In this embodiment, in order to further reduce the probability of data synchronization errors, based on the data migration work being divided into individual sub-tasks according to the indicator dimension, for each sub-task, the entire time span is divided into multiple small time segments according to the time range of the migrated data. This avoids the problem of poor efficiency or even crash of time series database API due to the possibility of some complex indicators pulling massive amounts of data. One segmented dataset corresponds to one segmented synchronization task under one sub-task, and one segmented task only synchronizes the data of the time period corresponding to the segmented task.

[0045] When an anomaly occurs in a segment task, it will be marked, which can quickly locate the time series data of the specific time period in which the error occurred, making it easy to troubleshoot. Moreover, each segment task is independent of the others and does not affect each other. Therefore, when one or more segment tasks fail to migrate, it will not block the data migration of other segment tasks, thus avoiding poor data reliability due to excessive data volume.

[0046] S300. Perform data synchronization on each segment of the dataset to be synchronized to achieve synchronization of the time series dataset.

[0047] In this embodiment, once the dataset to be synchronized is divided, the data synchronization process can begin, synchronizing each segment dataset to ClickHouse. Since each dataset to be synchronized is independent of the others, and each segment dataset within each dataset to be synchronized is also independent of the others, the segments dataset do not affect each other during the synchronization process. If one segment dataset encounters a synchronization problem, the other segments datasets can still perform data synchronization normally, thereby improving the efficiency of data synchronization and facilitating the troubleshooting of synchronization problems.

[0048] In addition, during the execution of subtasks, there may be situations such as system crashes or process interruptions. Since the information of previously executed subtasks has been persisted in the database, in this embodiment, the migration task will continue to be executed after the service restarts, thereby avoiding duplicate work and speeding up the synchronization process.

[0049] In this embodiment of the invention, the time-series dataset to be synchronized is first acquired. Based on the indicator dimensions of the time-series data, the dataset is divided into multiple sub-datasets to be synchronized. Then, based on the collection time range of each sub-dataset, it is further divided into multiple segmented datasets belonging to different time periods. Finally, data synchronization is performed on each segmented dataset of each sub-dataset to achieve the synchronization of the time-series dataset. By dividing the time-series dataset to be synchronized according to indicator dimensions, and then further dividing the resulting sub-datasets according to time periods, the entire time-series dataset is divided into multiple smaller datasets before synchronization, reducing the probability of problems during the synchronization process and ensuring data reliability.

[0050] In some embodiments, please refer to Figure 2 Step S100 specifically includes:

[0051] S110. Obtain the time series dataset to be synchronized and determine the indicator dimensions of each time series data in the time series dataset to be synchronized.

[0052] S120. Based on the indicator dimensions of each time series data, the time series dataset to be synchronized is divided to obtain multiple sub-datasets to be synchronized with different rates belonging to different indicator dimensions.

[0053] In this embodiment, in order to divide the time series dataset to be synchronized according to the indicator dimension, the indicator dimension of each time series dataset to be synchronized is first obtained. The indicator dimension can be obtained through the attributes of the time series data. After obtaining the indicator dimension of the time series data, the time series dataset to be synchronized can be divided according to the indicator dimension. The time series dataset to be synchronized is divided into multiple sub-datasets with different indicators according to the indicator dimension, thereby splitting the huge time series dataset to be synchronized into multiple subsets and reducing the high probability of data errors caused by the excessive size of the time series dataset to be synchronized.

[0054] In some embodiments, please refer to Figure 3 Step S200 specifically includes:

[0055] S210. Obtain the time collection range of the sub-dataset to be synchronized;

[0056] S220. Divide the time acquisition range into multiple time periods according to preset time division levels;

[0057] S230. Based on the divided time periods, the dataset to be synchronized is divided into multiple segmented datasets belonging to different time periods.

[0058] In this embodiment, to avoid a single synchronization subset being too large, the data migration work is divided into individual subtasks according to the indicator dimension. For each subtask, the dataset to be synchronized is divided into multiple segmented datasets based on the time range of the migrated data. Specifically, the time collection range is divided into multiple small time segments according to the start and end times of the time-series data collection in the dataset to be synchronized, using a preset time division level. Then, the dataset to be synchronized is split according to each small time segment to obtain multiple segmented datasets belonging to different time periods.

[0059] The preset time intervals can be set according to actual conditions, such as 4 hours, 1 hour, 30 minutes, 20 minutes, 10 minutes, 5 minutes, 1 minute, etc.

[0060] In some embodiments, step S300, wherein synchronizing the data of each segment of each subset of the dataset to be synchronized includes:

[0061] Data synchronization is performed on each segment of the dataset to be synchronized, and a fixed time label is generated for each time series data during the synchronization process.

[0062] In this embodiment, since data duplication may occur when a subtask is abnormally stopped and then re-executed, a fixed timestamp is generated for each piece of time-series data during the synchronization process. Duplicate time-series data will be automatically overwritten according to the timestamp, thereby avoiding data duplication.

[0063] In some embodiments, the method further includes:

[0064] If any of the datasets to be synchronized fails to synchronize, the synchronization task for that dataset will be executed again after a preset interval, until the dataset is successfully synchronized or the number of executions exceeds the preset number.

[0065] In this embodiment, to ensure the reliability of data migration for the sub-datasets to be synchronized in each indicator dimension, a subtask-level retry mechanism is set up. When a task encounters a problem, the subtask will automatically attempt to re-execute after a period of time until it succeeds. If the number of retries for a single subtask exceeds a preset number, the task will be suspended and handled manually. Preferably, the preset number is 5 times, that is, when the number of retries for a single subtask exceeds 5, the task will be suspended and handled manually.

[0066] In some embodiments, the method further includes:

[0067] When the data retrieval of any segment dataset of any sub-dataset to be synchronized times out, the preset time division level is reduced, the time collection range is divided according to the reduced time division level, and the sub-dataset to be synchronized is divided into multiple segment datasets according to the newly divided time period. Data synchronization is performed on multiple segment datasets until the data retrieval of each segment dataset is successful.

[0068] In this embodiment, the time division is divided into 7 levels from largest to smallest: 4 hours, 1 hour, 30 minutes, 20 minutes, 10 minutes, 5 minutes, and 1 minute. Since the finer the data segmentation, the less data is synchronized per word, the more operations are required, resulting in more time consumption during the interaction process and affecting the overall synchronization speed. Therefore, this application uses the largest time division to divide the time collection range during the initial synchronization. The amount of data synchronized at one time is larger, which improves the synchronization speed to a certain extent. When the response times out, the time division is automatically reduced to the next level to try to retrieve the data. If the retrieval is successful, the subsequent synchronization actions of this task will use the same time division. Otherwise, the downgrading attempt continues until the retrieval is successful. This ensures the synchronization speed when network and other hardware resources allow.

[0069] In some embodiments, when the data retrieval of the segmented dataset divided into time periods according to the smallest time division level fails, the segmented dataset to be synchronized is marked and automatic synchronization is stopped.

[0070] In this embodiment, if the retrieval of each segment dataset corresponding to the time period divided by the 1-minute time division still times out, it is considered that the historical data of the indicator is abnormally large. At this time, the dataset to be synchronized is marked and automatic synchronization is stopped. The task is marked as abnormal and then the task is exited. It will not be executed again. This abnormal task requires manual intervention and is handled separately.

[0071] The technical solution provided by this invention first acquires a time-series dataset to be synchronized. Based on the indicator dimensions of the time-series data, the dataset is divided into multiple sub-datasets to be synchronized. Then, based on the collection time range of each sub-dataset, it is further divided into multiple segmented datasets belonging to different time periods. Finally, data synchronization is performed on each segmented dataset of each sub-dataset to achieve synchronization of the time-series dataset. By dividing the time-series dataset to be synchronized according to indicator dimensions, and then further dividing the resulting sub-datasets according to time periods, the entire time-series dataset is divided into multiple smaller datasets before synchronization, reducing the probability of problems during the synchronization process and ensuring data reliability.

[0072] Another embodiment of the present invention provides a timing data synchronization device; please refer to [link to relevant documentation]. Figure 4 The timing data synchronization device includes a first partitioning module 11, a second partitioning module 12, and a synchronization module 13.

[0073] The first partitioning module 11 is used to obtain the time series dataset to be synchronized and to divide the time series dataset to be synchronized into multiple sub-datasets to be synchronized according to the indicator dimensions of the time series data.

[0074] The second partitioning module 12 is used to divide the dataset to be synchronized into multiple segmented datasets belonging to different time periods according to the collection time range of the dataset to be synchronized.

[0075] Synchronization module 13 is used to synchronize the data of each segment of each sub-data set to be synchronized, so as to realize the synchronization of the time series dataset to be synchronized.

[0076] This invention first acquires the time-series dataset to be synchronized. Based on the metrics dimensions of the time-series data, the dataset is divided into multiple sub-datasets. Then, based on the collection time range of each sub-dataset, it is further divided into multiple segmented datasets belonging to different time periods. Finally, data synchronization is performed on each segmented dataset of each sub-dataset to achieve synchronization of the time-series dataset. By dividing the time-series dataset according to metrics dimensions, and then further dividing the resulting sub-datasets according to time periods, the entire time-series dataset is divided into multiple smaller datasets before synchronization, reducing the probability of problems during synchronization and ensuring data reliability.

[0077] It should be noted that the module referred to in this invention refers to a series of computer program instruction segments that can perform specific functions. It is more suitable than a program for describing the execution process of timing data synchronization. For the specific implementation of each module, please refer to the corresponding method embodiments above, which will not be repeated here.

[0078] In some embodiments, the first segmentation module 11 includes an acquisition unit and a first classification unit.

[0079] The acquisition unit is used to acquire the time series dataset to be synchronized and to determine the indicator dimensions of each time series data in the time series dataset to be synchronized.

[0080] The first classification unit is used to divide the time series dataset to be synchronized according to the indicator dimensions of each time series data, so as to obtain multiple sub-datasets to be synchronized with rates belonging to different indicator dimensions.

[0081] In some embodiments, the second division module 12 includes a time range acquisition unit, a segmentation unit, and a second classification unit.

[0082] The time range acquisition unit is used to obtain the time collection range of the sub-dataset to be synchronized.

[0083] The segmentation unit is used to divide the time acquisition range into multiple time periods according to preset time division levels.

[0084] The second classification unit is used to divide the dataset to be synchronized into multiple segmented datasets belonging to different time periods, based on the divided time periods.

[0085] In some embodiments, the synchronization module 13 is specifically used for:

[0086] Data synchronization is performed on each segment of the dataset to be synchronized, and a fixed time label is generated for each time series data during the synchronization process.

[0087] In some embodiments, the synchronization module 13 is further configured to:

[0088] If any of the datasets to be synchronized fails to synchronize, the synchronization task for that dataset will be executed again after a preset interval, until the dataset is successfully synchronized or the number of executions exceeds the preset number.

[0089] In some embodiments, the second partitioning module 12 is further configured to

[0090] When the data retrieval of any segment dataset of any sub-dataset to be synchronized times out, the preset time division level is reduced, the time collection range is divided according to the reduced time division level, and the sub-dataset to be synchronized is divided into multiple segment datasets according to the newly divided time period. Data synchronization is performed on multiple segment datasets until the data retrieval of each segment dataset is successful.

[0091] In some embodiments, when the data retrieval of the segmented dataset divided into time periods according to the smallest time division level fails, the segmented dataset to be synchronized is marked and automatic synchronization is stopped.

[0092] Another embodiment of the present invention provides an electronic device, such as... Figure 5 As shown, the electronic device 10 includes:

[0093] One or more processors 110 and memory 120, Figure 5 The following description uses a processor 110 as an example. The processor 110 and the memory 120 can be connected via a bus or other means. Figure 5 Taking the example of a connection between China and Israel via a bus.

[0094] Processor 110 is used to perform various control logics of electronic device 10. It can be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), microcontroller, ARM (Acorn RISC Machine) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components. Furthermore, processor 110 can also be any conventional processor, microprocessor, or state machine. Processor 110 can also be implemented as a combination of computing devices, such as a combination of DSP and microprocessor, multiple microprocessors, one or more microprocessors combined with DSP and / or any other such configuration.

[0095] The memory 120, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions corresponding to the timing data synchronization method in the embodiments of the present invention. The processor 110 executes various functional applications and data processing of the electronic device 10 by running the non-volatile software programs, instructions, and units stored in the memory 120, thereby implementing the timing data synchronization method in the above-described method embodiments.

[0096] The memory 120 may include a program storage area and a data storage area. The program storage area may store applications required for the operating platform and at least one function; the data storage area may store data created based on the use of the electronic device 10. Furthermore, the memory 120 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 120 may optionally include memory remotely located relative to the processor 110, and these remote memories may be connected to the electronic device 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0097] One or more units are stored in memory 120. When executed by one or more processors 110, they perform the timing data synchronization method in any of the above method embodiments, for example, the method described above. Figure 1 The method steps S100 to S300.

[0098] Another embodiment of the present invention provides a computer-readable storage medium storing computer-executable instructions that are executed by one or more processors, for example, to perform the above-described instructions. Figure 1 The method steps S100 to S300.

[0099] As an example, computer-readable storage media can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) as external cache memory. By way of illustration and not limitation, RAM can be obtained in many forms such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM). The memory components or memories disclosed in the operating environment described herein are intended to include one or more of these and / or any other suitable types of memory.

[0100] In summary, the time-series data synchronization method, apparatus, electronic device, and storage medium provided by this invention first acquire the time-series dataset to be synchronized. Then, based on the indicator dimensions of the time-series data, the dataset is divided into multiple sub-datasets to be synchronized. Next, based on the collection time range of each sub-dataset, it is further divided into multiple segmented datasets belonging to different time periods. Finally, data synchronization is performed on each segmented dataset of each sub-dataset to achieve synchronization of the time-series dataset. By dividing the time-series dataset to be synchronized according to indicator dimensions, and then further dividing the resulting sub-datasets according to time periods, the entire time-series dataset is divided into multiple smaller datasets before synchronization, reducing the probability of problems during the synchronization process and ensuring data reliability.

[0101] The specific embodiments of the present invention described above do not constitute a limitation on the scope of protection of the present invention. Any other corresponding changes and modifications made in accordance with the technical concept of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A time-series data synchronization method, characterized in that, Includes the following steps: Obtain the time series dataset to be synchronized, and divide the time series dataset to be synchronized into multiple sub-datasets to be synchronized according to the indicator dimensions of the time series data; Based on the time collection range of the dataset to be synchronized, the dataset to be synchronized is divided into multiple segmented datasets belonging to different time periods; Data synchronization is performed on each segment of the dataset to be synchronized in order to achieve synchronization of the time series dataset. The process involves dividing the dataset to be synchronized into multiple segmented datasets belonging to different time periods, based on the time collection range of the dataset to be synchronized. Obtain the time collection range of the sub-dataset to be synchronized; The time acquisition range is divided into multiple time periods according to preset time division levels; Based on the defined time periods, the dataset to be synchronized is divided into multiple segmented datasets belonging to different time periods; When the data retrieval of any segment dataset of any sub-dataset to be synchronized times out, the preset time division level is reduced, the time collection range is divided according to the reduced time division level, and the sub-dataset to be synchronized is divided into multiple segment datasets according to the newly divided time period. Data synchronization is performed on multiple segment datasets until the data retrieval of each segment dataset is successful. During the initial synchronization, the largest time division is used to divide the time collection range. The amount of data synchronized in a single time is large, which improves the synchronization speed to a certain extent. When the response times out, the time division is automatically reduced to the next level to try to retrieve the data. If the retrieval is successful, the subsequent synchronization actions of this task will use the same time division. Otherwise, the downgrading attempt will continue until the retrieval is successful.

2. The time-series data synchronization method according to claim 1, characterized in that, The process of obtaining the time-series dataset to be synchronized involves dividing it into multiple sub-datasets based on the metrics dimensions of the time-series data, including: Obtain the time series dataset to be synchronized, and determine the indicator dimensions of each time series data in the time series dataset to be synchronized; Based on the indicator dimensions of each time series data, the time series dataset to be synchronized is divided to obtain multiple sub-datasets to be synchronized with rates belonging to different indicator dimensions.

3. The time-series data synchronization method according to claim 1, characterized in that, The process of synchronizing data for each segment of the dataset to be synchronized includes: Data synchronization is performed on each segment of the dataset to be synchronized, and a fixed time label is generated for each time series data during the synchronization process.

4. The time-series data synchronization method according to claim 1, characterized in that, Also includes: If any of the datasets to be synchronized fails to synchronize, the synchronization task for that dataset will be executed again after a preset interval, until the dataset is successfully synchronized or the number of executions exceeds the preset number.

5. The time-series data synchronization method according to claim 1, characterized in that, When data retrieval fails for the segmented datasets divided into time periods according to the smallest time division level, the segmented dataset to be synchronized is marked and automatic synchronization is stopped.

6. A timing data synchronization device, characterized in that, include: The first partitioning module is used to obtain the time series dataset to be synchronized and to divide the time series dataset to be synchronized into multiple sub-datasets to be synchronized according to the indicator dimensions of the time series data. The second partitioning module is used to divide the dataset to be synchronized into multiple segmented datasets belonging to different time periods according to the time collection range of the dataset to be synchronized. The synchronization module is used to synchronize the data of each segment of each sub-dataset to be synchronized, so as to realize the synchronization of the time series dataset to be synchronized. The second segmentation module includes a time range acquisition unit, a segmentation unit, and a second classification unit; The time range acquisition unit is used to obtain the time collection range of the sub-dataset to be synchronized; The segmentation unit is used to divide the time acquisition range into multiple time periods according to preset time division levels; The second classification unit is used to divide the sub-dataset to be synchronized into multiple segmented datasets belonging to different time periods according to the divided time periods; The second partitioning module is also used to: when the data retrieval of each segment dataset of any subset dataset to be synchronized times out, reduce the preset time partitioning level, divide the time collection range according to the reduced time partitioning level, and divide the subset dataset to be synchronized into multiple segment datasets according to the newly divided time period, and synchronize the data of the multiple segment datasets until the data retrieval of each segment dataset is successful. During the initial synchronization, the largest time division is used to divide the time collection range. The amount of data synchronized in a single time is large, which improves the synchronization speed to a certain extent. When the response times out, the time division is automatically reduced to the next level to try to retrieve the data. If the retrieval is successful, the subsequent synchronization actions of this task will use the same time division. Otherwise, the downgrading attempt will continue until the retrieval is successful.

7. An electronic device, characterized in that, include: Processor and memory; The memory stores computer programs that can be executed by the processor; When the processor executes the computer program, it implements the steps in the timing data synchronization method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the steps in the timing data synchronization method as described in any one of claims 1-5.

Images