Distributed multi-modal efficient time series data storage method
By analyzing the data generation amount of the time series data generation device and the storage amount and life value of the distributed storage device, determining the target and non-target storage devices, and generating storage matching strategies, the problem of low life management accuracy of the storage device in the distributed data storage system is solved, and the data storage capability and stability of the time series database are improved.
Patent Information
- Application Number
- CN202510405873.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-02
- Publication Date
- 2025-04-29
- Estimated Expiration
- 2045-04-02
AI Technical Summary
The distributed data storage system has difficulty in timing data storage allocation and storage device management, resulting in low accuracy in storage device life management, which affects the data storage capability and stability of the timing database.
By collecting the unit time data generation amount of each time data generation device, the real-time data storage amount of each distributed time series data storage device and the remaining service life value of the storage device are analyzed, the target update storage device and the non-target update storage device are determined, and the timing data storage matching strategy is generated to control the storage actions of the timing data generation device.
It realizes high-precision life management of storage devices in distributed data storage systems, efficient device replacement and data migration, and improves the data storage capabilities and stability of timing databases.
Smart Images

Figure CN119902723A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of time series data storage, and in particular to a distributed multi-modal efficient time series data storage method. Background Art
[0002] Time series databases are widely used in the fields of Internet of Things, finance, industrial control, etc., mainly for storing and querying large-scale time series data. In the existing technology, time series database systems have efficient data insertion and query capabilities, but there are challenges in data storage and scalability. Distributed storage systems provide reliable storage solutions, but distributed storage systems still have many limitations in time series data storage allocation and storage device management.
[0003] Specifically, since different distributed time series data storage devices in a distributed data storage system may have different storage capacities (usually referring to the number of time series data groups of a unit size that can be stored in the storage device) and storage device service life (usually referring to the maximum number of write times of a time series data group of a unit size in the storage device), different data sources store different distributed time series data storage devices in the distributed storage system, and the amount of data generated per unit time is also different, the distributed data storage system has low accuracy in life management of several distributed time series data storage devices within it, which may easily cause the service life of multiple distributed time series data storage devices to reach the upper limit at the same time, and the storage devices need to be replaced and data migrated at the same time, rather than reaching the upper limit and replacing and migrating the storage devices in sequence. Such a situation will affect the data storage capacity of the entire time series database, and further affect the business operation of the actual application scenario. At the same time, due to the different data retention requirements of different data sources, the time series data generated by each time series data generation device has different expiration deletion times when it is stored in the distributed time series data storage device, which further increases the difficulty of the distributed data storage system in allocating time series data and managing storage devices.
[0004] Therefore, how to reduce the difficulty of distributed data storage systems in allocating time series data storage and managing storage devices, improve the data storage capacity and stability of time series databases, and realize high-precision life management of storage devices in distributed data storage systems and efficient equipment replacement and data migration processes is a technical problem that needs to be solved urgently. Summary of the invention
[0005] The present invention provides a distributed multi-modal efficient time series data storage method, aiming to solve at least one of the above technical problems.
[0006] To achieve the above object, the present invention provides a distributed multi-modal efficient time series data storage method, the method comprising the following steps: S1: collecting time series data features of each time series data generating device within the area, and generating a unit time data generation amount of each time series data generating device according to the time series data features; S2: Obtaining the device identification and storage information of each distributed time series data storage device in the distributed data storage system, and determining the remaining service life value of the storage device of each distributed time series data storage device and the real-time data storage capacity of several groups of time series data; S3: determining the target update storage device and the non-target update storage device in the current storage device update cycle according to the remaining service life value of the storage device of each distributed time-series data storage device; S4: Based on the remaining service life values of the target update storage devices and the non-target update storage devices, the real-time data storage volume, and the data generation volume per unit time of each time series data generation device, a time series data storage matching strategy for each distributed time series data storage device is generated; wherein the time series data storage matching strategy includes a first time series data storage path between a plurality of first time series data generation devices and each target update storage device and a second time series data storage path between a plurality of second time series data generation devices and each non-target update storage device; S5: Based on the time series data storage matching strategy, control each time series data generating device to execute a corresponding time series data storage action within the current storage device update cycle.
[0007] Optionally, step S1 specifically includes: S11: collecting time series data characteristics of each time series data generating device within the area; wherein the time series data characteristics include the time series data type and the time series data upload frequency; S12: Determine the data volume of a single time series data transmission executed by each time series data generating device according to the time series data type, and calculate the data generation volume per unit time of each time series data generating device by using the data volume and the time series data upload frequency.
[0008] Optionally, step S2 specifically includes: S21: Obtaining a device identification and storage information of each distributed time series data storage device in the distributed data storage system; wherein the storage information includes a historical storage data volume and real-time storage information; S22: Calculate the remaining service life value of the storage device of each distributed time series data storage device by using the device identification and the amount of historical storage data in the storage information; S23: Determine the real-time data storage capacity of each distributed time series data storage device by using the real-time storage information in the storage information.
[0009] Optionally, step S22 specifically includes: S221: using the device identifier, querying the initial service life value of the storage device of each distributed time series data storage device, and extracting the historical storage data volume and real-time data storage information in the storage information; S222: Calculate the remaining service life of each distributed time series data storage device according to the corresponding relationship between the amount of historical storage data and the reduced value of the service life of the storage device by using the amount of historical storage data in the historical storage information and the service life of the storage device.
[0010] Optionally, step S23 specifically includes: S231: using the real-time data storage information, determining the time series data types of several groups of time series data stored in each distributed time series data storage device; S232: Calculate the real-time data storage capacity in each distributed time series data storage device according to the time series data type of each group of time series data.
[0011] Optionally, step S3 specifically includes: S31: sorting the remaining service life value of each distributed time series data storage device from low to high to generate a sorted list of remaining service life values of the distributed time series data storage devices; S32: The first distributed time series data storage device in the remaining life value sorted list is used as a target update storage device in the current storage device update cycle, and the remaining distributed time series data storage devices are used as non-target update storage devices.
[0012] Optionally, step S4 specifically includes: S41: Based on the remaining service life value of the target updated storage device and the duration of the current storage device update cycle, and on the principle that the remaining service life value of the target updated storage device at the end of the current storage device update cycle is lower than the device update threshold, calculate the first unit time data storage amount when the target updated storage device receives the time series data in the current storage device; S42: According to the first unit time data storage capacity and the unit time data generation capacity of each time series data generation device, matching a plurality of first time series data generation devices whose sum of unit time data generation capacity is closest to the first unit time data storage capacity among all time series data generation devices; S43: adding the first time series data storage paths of the plurality of first time series data generating devices and the target update storage device to the time series data storage matching strategy as the time series data storage mode of the plurality of first time series data generating devices; S44: Based on the first ratio of the remaining service life values of the non-target update storage devices, a plurality of second time series data generating devices are matched for each non-target update storage device in the remaining time series data generating devices, on the principle that the second ratio of the sum of the unit time data generation amount allocated by the remaining time series data generating devices to each non-target update storage device is closest to the first ratio; S45: adding the second time series data storage paths of the plurality of second time series data generating devices and the corresponding non-target update storage devices to the time series data storage matching strategy as the time series data storage mode of the plurality of second time series data generating devices.
[0013] Optionally, step S4 further includes: S46: Obtaining the data storage period of each time series data generating device, the upper limit of the data storage capacity of each distributed time series data storage device, and the data stamp information of each set of stored time series data; S47: Based on the data storage period and data stamp information, and taking into account the real-time data storage capacity and the upper limit of the data storage capacity of each distributed time series data storage device, it is determined whether to re-execute the determination of the target update storage device and the non-target update storage device.
[0014] Optionally, the step S47 specifically includes: S471: Calculate the periodic deletion time of each group of time series data in each distributed time series data storage device according to the data storage period and data stamp information, consider the real-time data storage capacity and the data storage capacity upper limit of each distributed time series data storage device, and determine whether the real-time data storage capacity of each distributed time series data storage device is higher than the data storage capacity upper limit when the time series data storage path corresponding to the time series data storage matching strategy is executed by the time series data storage device of the time series data generation device; S472: If yes, remove the distributed time series data storage device, and return to step S3 to determine the target update storage device and the non-target update storage device.
[0015] Optionally, the method further includes step S6: at the end of the current storage device update cycle, importing the data in the target update storage device into the newly connected distributed time series data storage device, and re-determining the time series data storage matching strategy for the next storage device update cycle.
[0016] The beneficial effects of the present invention are as follows: a distributed multi-modal efficient time series data storage method is proposed. By considering multiple influencing factors of time series data to storage devices, the storage path generation of the target update storage device based on the current storage device update cycle is first executed, and then the storage path generation of the non-target update storage device based on the real-time data storage volume ratio is executed, so as to drive the life management and data migration of several storage devices in the distributed data storage system to match each storage device update cycle, reduce the difficulty of the distributed data storage system in time series data storage allocation and storage device management, improve the data storage capacity and stability of the time series database, and realize the high-precision life management of storage devices in the distributed data storage system and the process of efficient equipment replacement and data migration. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 It is a flow chart of an embodiment of a distributed multi-modal efficient time series data storage method of the present invention.
[0018] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0019] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0020] The embodiment of the present invention provides a distributed multi-modal efficient time series data storage method, referring to Figure 1 , Figure 1 It is a flow chart of an embodiment of a distributed multi-modal efficient time series data storage method of the present invention.
[0021] In this embodiment, a distributed multi-modal efficient time series data storage method is provided, the method comprising the following steps: S1: collecting time series data features of each time series data generating device within the area, and generating a unit time data generation amount of each time series data generating device according to the time series data features; S2: Obtaining the device identification and storage information of each distributed time series data storage device in the distributed data storage system, and determining the remaining service life value of the storage device of each distributed time series data storage device and the real-time data storage capacity of several groups of time series data; S3: determining the target update storage device and the non-target update storage device in the current storage device update cycle according to the remaining service life value of the storage device of each distributed time-series data storage device; S4: Based on the remaining service life values of the target update storage devices and the non-target update storage devices, the real-time data storage volume, and the data generation volume per unit time of each time series data generation device, a time series data storage matching strategy for each distributed time series data storage device is generated; wherein the time series data storage matching strategy includes a first time series data storage path between a plurality of first time series data generation devices and each target update storage device and a second time series data storage path between a plurality of second time series data generation devices and each non-target update storage device; S5: Based on the time series data storage matching strategy, control each time series data generating device to execute a corresponding time series data storage action within the current storage device update cycle.
[0022] It should be noted that, since different distributed time series data storage devices in the distributed data storage system may have different storage capacities (usually referring to the number of time series data groups of unit size that can be stored in the storage device) and storage device service life (usually referring to the maximum number of write times of time series data groups of unit size in the storage device), different data sources store different distributed time series data storage devices in the distributed storage system, and the amount of data generated per unit time is also different, the distributed data storage system has low accuracy in life management of several distributed time series data storage devices inside it, which may easily cause the service life of multiple distributed time series data storage devices to reach the upper limit at the same time, and the storage devices need to be replaced and data migrated at the same time, rather than reaching the upper limit and replacing and migrating the storage devices in sequence. Such a situation will affect the data storage capacity of the entire time series database, and further affect the business operation of the actual application scenario. At the same time, due to the different data retention requirements of different data sources, the time series data generated by each time series data generation device has different expiration deletion times when it is stored in the distributed time series data storage device, which further increases the difficulty of the distributed data storage system in allocating time series data and managing storage devices.
[0023] In order to solve the above problems, this embodiment collects the data generation per unit time of each time series data generation device, analyzes the real-time data storage capacity and the remaining service life value of each distributed time series data storage device, determines the target update storage device and the non-target update storage device, executes the storage path generation from the target update storage device and the non-target update storage device to the time series data generation device, and controls the time series data storage action of the time series data generation device. Therefore, by considering multiple influencing factors of time series data and storage devices, the life management and data migration of several storage devices of the distributed data storage system can be driven to match the update cycle of each storage device, which can reduce the difficulty of the distributed data storage system in time series data storage allocation and storage device management, and improve the data storage capacity and stability of the time series database.
[0024] In a preferred embodiment, step S1 specifically includes: S11: collecting time series data characteristics of each time series data generating device within the area; wherein the time series data characteristics include the time series data type and the time series data upload frequency; S12: Determine the data volume of a single time series data transmission executed by each time series data generating device according to the time series data type, and calculate the data generation volume per unit time of each time series data generating device by using the data volume and the time series data upload frequency.
[0025] In this embodiment, by collecting the time series data type and the time series data upload frequency when each time series data generating device in the area uploads the time series data, the data volume of a single time series data transmission is determined according to the time series data type, and then the data generation volume per unit time of each time series data generating device is calculated by multiplying the time series data upload frequency by the data volume of a single time series data transmission. By accurately analyzing the data generation volume per unit time of each time series data generating device, data support is provided for the subsequent high-precision management of the life of the storage device and the generation of data storage allocation strategies.
[0026] In a preferred embodiment, step S2 specifically includes: S21: Obtaining a device identification and storage information of each distributed time series data storage device in the distributed data storage system; wherein the storage information includes a historical storage data volume and real-time storage information; S22: Calculate the remaining service life value of the storage device of each distributed time series data storage device by using the device identification and the amount of historical storage data in the storage information; S23: Determine the real-time data storage capacity of each distributed time series data storage device by using the real-time storage information in the storage information.
[0027] Furthermore, step S22 specifically includes: S221: using the device identifier, querying the initial service life value of the storage device of each distributed time series data storage device, and extracting the historical storage data volume and real-time data storage information in the storage information; S222: Calculate the remaining service life of each distributed time series data storage device according to the corresponding relationship between the amount of historical storage data and the reduced value of the service life of the storage device by using the amount of historical storage data in the historical storage information and the service life of the storage device.
[0028] Furthermore, step S23 specifically includes: S231: using the real-time data storage information, determining the time series data types of several groups of time series data stored in each distributed time series data storage device; S232: Calculate the real-time data storage capacity in each distributed time series data storage device according to the time series data type of each group of time series data.
[0029] In this embodiment, the device identification and storage information of each distributed time series data storage device are first obtained, the initial service life value of the storage device is queried using the device identification, and then the remaining service life value of the storage device is calculated by converting the historical storage data volume into the storage device service life reduction value; then, the time series data type is determined based on the real-time data storage information, and the real-time data storage volume in the distributed time series data storage device is calculated.
[0030] In a preferred embodiment, step S3 specifically includes: S31: sorting the remaining service life value of each distributed time series data storage device from low to high to generate a sorted list of remaining service life values of the distributed time series data storage devices; S32: The first distributed time series data storage device in the remaining life value sorted list is used as a target update storage device in the current storage device update cycle, and the remaining distributed time series data storage devices are used as non-target update storage devices.
[0031] In this embodiment, the storage device with the lowest remaining life value is used as the target update storage device, and the remaining storage devices are used as non-target update storage devices. It should be noted that the target update storage device is the storage device that needs to be replaced and data migrated after the current storage update cycle ends, and the non-target update storage device is the storage device that needs to be replaced and data migrated in the subsequent storage update cycle.
[0032] In a preferred embodiment, step S4 specifically includes: S41: Based on the remaining service life value of the target updated storage device and the duration of the current storage device update cycle, and on the principle that the remaining service life value of the target updated storage device at the end of the current storage device update cycle is lower than the device update threshold, calculate the first unit time data storage amount when the target updated storage device receives the time series data in the current storage device; S42: According to the first unit time data storage capacity and the unit time data generation capacity of each time series data generation device, matching a plurality of first time series data generation devices whose sum of unit time data generation capacity is closest to the first unit time data storage capacity among all time series data generation devices; S43: adding the first time series data storage paths of the plurality of first time series data generating devices and the target update storage device to the time series data storage matching strategy as the time series data storage mode of the plurality of first time series data generating devices; S44: Based on the first ratio of the remaining service life values of the non-target update storage devices, a plurality of second time series data generating devices are matched for each non-target update storage device in the remaining time series data generating devices, on the principle that the second ratio of the sum of the unit time data generation amount allocated by the remaining time series data generating devices to each non-target update storage device is closest to the first ratio; S45: adding the second time series data storage paths of the plurality of second time series data generating devices and the corresponding non-target update storage devices to the time series data storage matching strategy as the time series data storage mode of the plurality of second time series data generating devices.
[0033] In this embodiment, by considering multiple influencing factors of time series data to storage devices, the storage path generation of the target update storage device based on the current storage device update cycle is first executed, and then the storage path generation of the non-target update storage device based on the real-time data storage volume ratio is executed, so as to drive the life management and data migration of several storage devices in the distributed data storage system to match each storage device update cycle, reduce the difficulty of the distributed data storage system in allocating time series data and managing storage devices, improve the data storage capacity and stability of the time series database, and realize the high-precision life management of storage devices in the distributed data storage system and efficient equipment replacement and data migration process.
[0034] In a preferred embodiment, step S4 further includes: S46: Obtaining the data storage period of each time series data generating device, the upper limit of the data storage capacity of each distributed time series data storage device, and the data stamp information of each set of stored time series data; S47: Based on the data storage period and data stamp information, and taking into account the real-time data storage capacity and the upper limit of the data storage capacity of each distributed time series data storage device, it is determined whether to re-execute the determination of the target update storage device and the non-target update storage device.
[0035] Furthermore, the step S47 specifically includes: S471: Calculate the periodic deletion time of each group of time series data in each distributed time series data storage device according to the data storage period and data stamp information, consider the real-time data storage capacity and the data storage capacity upper limit of each distributed time series data storage device, and determine whether the real-time data storage capacity of each distributed time series data storage device is higher than the data storage capacity upper limit when the time series data storage path corresponding to the time series data storage matching strategy is executed by the time series data storage device of the time series data generation device; S472: If yes, remove the distributed time series data storage device, and return to step S3 to determine the target update storage device and the non-target update storage device.
[0036] Considering the different data retention requirements of different data sources, the time-limit deletion time of the time-limit data generated by each time-limit data generation device when it is stored in the distributed time-limit data storage device is different, which further increases the difficulty of the distributed data storage system in allocating time-limit data and managing storage devices. In this embodiment, after the time-limit data storage matching strategy is generated according to the above steps, it is necessary to determine whether the real-time data storage capacity of a distributed time-limit data storage device will be higher than the upper limit of the data storage capacity when each distributed time-limit data storage device performs time-limit data storage according to the time-limit data storage matching strategy (because the time-limit data is continuously imported and stored, and is also continuously deleted by the time limit). If so, it indicates that the current time-limit data storage matching strategy cannot meet the space limit of the storage device. At this time, it is necessary to temporarily remove the distributed time-limit data storage device from the current storage update cycle (the reason is that there is a large amount of new data in the distributed time-limit data storage device, and it is impossible to maintain the balance of data storage capacity by deleting the time limit on one side and importing the time-limit data on the other side). After the removal, return to step S3 to perform the determination of the target update storage device and the non-target update storage device and the subsequent generation of the time-limit data storage matching strategy.
[0037] In a preferred embodiment, the method further includes step S6: at the end of the current storage device update cycle, the data in the target update storage device is imported into the newly connected distributed time series data storage device, and the time series data storage matching strategy for the next storage device update cycle is re-determined.
[0038] In this embodiment, after obtaining a determined time series data storage matching strategy, time series data storage is performed according to the time series data storage matching strategy. After that, when the current storage device update cycle ends, the storage device is replaced and the storage data is migrated, and a new time series data storage matching strategy is re-determined, thereby realizing the storage allocation of time series data and storage device management of the distributed data storage system.
[0039] In addition, the present invention also proposes a distributed multimodal efficient time series data storage device, which includes: a memory, a processor, and a distributed multimodal efficient time series data storage program stored in the memory and runnable on the processor. When the distributed multimodal efficient time series data storage program is executed by the processor, the steps of the distributed multimodal efficient time series data storage method as described above are implemented.
[0040] The specific implementation of the distributed multi-modal efficient time series data storage device of the present application is basically the same as the above-mentioned embodiments of the distributed multi-modal efficient time series data storage method, and will not be repeated here.
[0041] It is understood that, in the description of this specification, the description with reference to the terms "one embodiment", "another embodiment", "other embodiments", or "first to Nth embodiments" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0042] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.
[0043] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A distributed multi-modal efficient time series data storage method, characterized in that: The method comprises the following steps: S1: collecting time series data features of each time series data generating device within the area, and generating a unit time data generation amount of each time series data generating device according to the time series data features; S2: Obtaining the device identification and storage information of each distributed time series data storage device in the distributed data storage system, and determining the remaining service life value of the storage device of each distributed time series data storage device and the real-time data storage capacity of several groups of time series data; S3: determining the target update storage device and the non-target update storage device in the current storage device update cycle according to the remaining service life value of the storage device of each distributed time-series data storage device; S4: Based on the remaining service life values of the target update storage devices and the non-target update storage devices, the real-time data storage volume, and the data generation volume per unit time of each time series data generation device, a time series data storage matching strategy for each distributed time series data storage device is generated; wherein the time series data storage matching strategy includes a first time series data storage path between a plurality of first time series data generation devices and each target update storage device and a second time series data storage path between a plurality of second time series data generation devices and each non-target update storage device; S5: Based on the time series data storage matching strategy, control each time series data generating device to execute a corresponding time series data storage action within the current storage device update cycle.
2. The distributed multi-modal efficient time series data storage method according to claim 1, characterized in that: Step S1 specifically includes: S11: collecting time series data characteristics of each time series data generating device within the area; wherein the time series data characteristics include the time series data type and the time series data upload frequency; S12: Determine the data volume of a single time series data transmission executed by each time series data generating device according to the time series data type, and calculate the data generation volume per unit time of each time series data generating device by using the data volume and the time series data upload frequency.
3. The distributed multi-modal efficient time series data storage method according to claim 1, characterized in that: Step S2 specifically includes: S21: Obtaining a device identification and storage information of each distributed time series data storage device in the distributed data storage system; wherein the storage information includes a historical storage data volume and real-time storage information; S22: Calculate the remaining service life value of the storage device of each distributed time series data storage device by using the device identification and the amount of historical storage data in the storage information; S23: Determine the real-time data storage capacity of each distributed time series data storage device by using the real-time storage information in the storage information.
4. The distributed multi-modal efficient time series data storage method according to claim 3, characterized in that: Step S22 specifically includes: S221: using the device identifier, querying the initial service life value of the storage device of each distributed time series data storage device, and extracting the historical storage data volume and real-time data storage information in the storage information; S222: Calculate the remaining service life of each distributed time series data storage device according to the corresponding relationship between the amount of historical storage data and the reduced value of the service life of the storage device by using the amount of historical storage data in the historical storage information and the service life of the storage device.
5. The distributed multi-modal efficient time series data storage method according to claim 3, characterized in that: Step S23 specifically includes: S231: using the real-time data storage information, determining the time series data types of several groups of time series data stored in each distributed time series data storage device; S232: Calculate the real-time data storage capacity in each distributed time series data storage device according to the time series data type of each group of time series data.
6. The distributed multi-modal efficient time series data storage method according to claim 1, characterized in that: Step S3 specifically includes: S31: sorting the remaining service life value of each distributed time series data storage device from low to high to generate a sorted list of remaining service life values of the distributed time series data storage devices; S32: The first distributed time series data storage device in the remaining life value sorted list is used as a target update storage device in the current storage device update cycle, and the remaining distributed time series data storage devices are used as non-target update storage devices.
7. The distributed multi-modal efficient time series data storage method according to claim 6, characterized in that: Step S4 specifically includes: S41: Based on the remaining service life value of the target updated storage device and the duration of the current storage device update cycle, and on the principle that the remaining service life value of the target updated storage device at the end of the current storage device update cycle is lower than the device update threshold, calculate the first unit time data storage amount when the target updated storage device receives the time series data in the current storage device; S42: According to the first unit time data storage capacity and the unit time data generation capacity of each time series data generation device, matching a plurality of first time series data generation devices whose sum of unit time data generation capacity is closest to the first unit time data storage capacity among all time series data generation devices; S43: adding the first time series data storage paths of the plurality of first time series data generating devices and the target update storage device to the time series data storage matching strategy as the time series data storage mode of the plurality of first time series data generating devices; S44: Based on the first ratio of the remaining service life values of the non-target update storage devices, a plurality of second time series data generating devices are matched for each non-target update storage device in the remaining time series data generating devices, on the principle that the second ratio of the sum of the unit time data generation amount allocated by the remaining time series data generating devices to each non-target update storage device is closest to the first ratio; S45: adding the second time series data storage paths of the plurality of second time series data generating devices and the corresponding non-target update storage devices to the time series data storage matching strategy as the time series data storage mode of the plurality of second time series data generating devices.
8. The distributed multi-modal efficient time series data storage method according to claim 7, characterized in that: Step S4 also includes: S46: Obtaining the data storage period of each time series data generating device, the upper limit of the data storage capacity of each distributed time series data storage device, and the data stamp information of each set of stored time series data; S47: Based on the data storage period and data stamp information, and taking into account the real-time data storage capacity and the upper limit of the data storage capacity of each distributed time series data storage device, it is determined whether to re-execute the determination of the target update storage device and the non-target update storage device.
9. The distributed multi-modal efficient time series data storage method according to claim 8, characterized in that: The step S47 specifically includes: S471: Calculate the periodic deletion time of each group of time series data in each distributed time series data storage device according to the data storage period and data stamp information, consider the real-time data storage capacity and the data storage capacity upper limit of each distributed time series data storage device, and determine whether the real-time data storage capacity of each distributed time series data storage device is higher than the data storage capacity upper limit when the time series data storage path corresponding to the time series data storage matching strategy is executed by the time series data storage device of the time series data generation device; S472: If yes, remove the distributed time series data storage device, and return to step S3 to determine the target update storage device and the non-target update storage device.
10. The distributed multi-modal efficient time series data storage method according to claim 1, characterized in that: The method further includes step S6: when the current storage device update cycle ends, the data in the target update storage device is imported into the newly connected distributed time series data storage device, and the time series data storage matching strategy for the next storage device update cycle is re-determined.
Citation Information
Patent Citations
Data migration method, device and equipment and computer readable storage medium
CN113126924A
Data storage, query and management method and device, equipment, system and program
CN113590675A
Storage management method, device and system and computer readable storage medium
CN114217740A
Time series data storage method and device
CN118466851A
Distributed storage data self-adaptive load balancing method and system
CN119201003A