Data storage method, device and system, electronic equipment and storage medium

Abstract

The invention relates to the technical field of data storage, and provides a data storage method, device and system, electronic equipment and a storage medium. According to the method, multiple pieces of first to-be-stored data with the same first to-be-stored time identifier are obtained, and the first to-be-stored data are grouped according to first to-be-stored time; the method comprises the steps of obtaining at least two first data sets, creating data sub-tables of the number of the first data sets, storing first to-be-stored data in the first data sets in the corresponding data sub-tables, and storing the first to-be-stored data in the first data sets in the corresponding data sub-tables. And recording the sub-table identifier of the data sub-table where the first to-be-stored data corresponding to each first to-be-stored time identifier is located, so that the time identifiers of the data stored in the same data sub-table can be ensured to be different, and the problem of data loss caused by the fact that the data with the same time identifier exists in the same sub-table is avoided.

Description

Technical Field

[0001] The present application relates to the technical field of data storage, and in particular to a data storage method, device, system, electronic device and storage medium. Background Art

[0002] TDengine is a big data platform designed and optimized for scenarios such as the Internet of Things and the Industrial Internet. Its core module is a high-performance, clustered open source, cloud-native, and minimalist time series database (TSDB). It can safely and efficiently aggregate, store, analyze, and distribute TB or even PB-level data generated by a large number of devices and data collectors every day, monitor and warn business operation status in real time, and provide real-time business insights.

[0003] ts is a default time field in the tdengine table, and all data is based on this time. When storing data, Tdengine uses super tables as units. A super table consists of multiple sub tables. Since tdengine is a time series database, if data with the same ts time is inserted into the same sub table at the same time, that is, the ts time is the same, but the data content is different, some data comes from multiple devices, or data generated by different services of the same device, the later data will overwrite the previous data, leaving only the last piece of data, and the previous data will be lost.

[0004] If, under certain scenarios, a large amount of data with the same ts field is stored in the same sub-table at the same time, a large amount of data will be lost, leaving only the last piece of data, resulting in data loss. Summary of the invention

[0005] The embodiments of the present application provide a data storage method, device, vehicle and storage medium to solve the technical problem in the related art that when a large amount of data with the same ts field is stored in the same sub-table at the same time, a large amount of data will be lost, leaving only the last piece of data, resulting in data loss.

[0006] An embodiment of the present application provides a data storage method, the method comprising: obtaining a plurality of first data to be stored, wherein the first time identifiers to be stored of at least two of the first data to be stored are the same; grouping the plurality of first data to be stored according to the first time identifier to be stored of each first data to be stored to obtain at least two first data sets, wherein the first time identifiers to be stored of the first data to be stored in the same first data set are different; creating data subtables of the first number of sets of first data sets, and allocating a data subtable to each first data set, wherein the data subtables corresponding to the first data sets are different; storing all the first data to be stored in each first data set into the corresponding data subtable, and recording the subtable identifier of the data subtable in which the first data to be stored corresponding to each first time identifier to be stored is stored.

[0007] In one embodiment of the present application, multiple first data to be stored are grouped according to the first time identifier of each first data to be stored to obtain at least two first data sets, including: repeatedly executing the grouping strategy until the first time identifiers of the first data to be stored that have not yet been added to the first data set are all different, so as to obtain multiple first data sets, wherein the number of first sets of the first data sets is equal to the repetition maximum value, and the repetition maximum value is the maximum value of the number of first data to be stored with the same first time identifier in multiple first data to be stored; wherein the grouping strategy includes sorting the first data to be stored that have not yet been added to the first data set according to the first time identifier to be stored, and selecting one first data to be stored corresponding to each first time identifier to be stored to generate a first data set.

[0008] In one embodiment of the present application, before creating a first set number of data sub-tables of the first data set, the data storage method also includes: generating a sub-table name corresponding to each first data set according to the set identifier of the first data set; and creating a data sub-table corresponding to each first data set based on the generated sub-table names.

[0009] In one embodiment of the present application, after storing all the first data to be stored in each first data set in the corresponding data subtable, the data storage method also includes: obtaining multiple second data to be stored, the second data to be stored including a second time identifier to be stored and a second business data; if each of the second time identifiers to be stored is different from each of the first time identifiers to be stored, grouping the multiple second data to be stored according to the second time identifier to be stored of each second data to be stored to obtain one or more second data sets, wherein the second time identifiers to be stored of the second data in the same second data set are different; if the number of second sets of the second data set is less than or equal to the number of the first sets, assigning a created data subtable to one second data set, and each second data set has a plurality of second data sets. If the corresponding data sub-tables are different, all the second data to be stored in each second data set are stored in the corresponding data sub-table, and the sub-table identifier of the data sub-table in which the second data to be stored corresponding to each second time identifier to be stored is recorded; if the number of second sets of the second data set is greater than the number of the first sets, create a data sub-table of a first difference number, and assign the newly created data sub-table and the created data sub-table to each second data set, the first difference number is the difference between the first set number and the second data set, the data sub-tables corresponding to each second data set are different, all the second data to be stored in each second data set are stored in the corresponding data sub-table, and the sub-table identifier of the data sub-table in which the second data to be stored corresponding to each second time identifier to be stored is recorded.

[0010] In one embodiment of the present application, after storing all the first data to be stored in each first data set into the corresponding data subtable, the data storage method also includes: obtaining multiple second data to be stored, the second data to be stored including a second time identifier to be stored and a second business data; if the second time identifier to be stored is the same as the first time identifier to be stored, obtaining the number of subtables of the data subtable where the first data to be stored corresponding to the same first time identifier to be stored is located, and obtaining the newly added number of second data to be stored corresponding to the same first time identifier to be stored; determining the second difference number based on the difference between the first set number and the subtable number; if the newly added number is less than or equal to the second difference number, assigning the newly added number of created data subtables that do not include the first data to be stored corresponding to the same first time identifier to be stored to each second data set, and each second The data sub-tables corresponding to the data sets are different, all the second data to be stored in each second data set are stored in the corresponding data sub-table, and the sub-table identifier of the data sub-table where the second data to be stored corresponding to each second time identifier to be stored is recorded; if the newly added quantity is greater than the second difference quantity, the third difference quantity is determined according to the difference between the newly added quantity and the second difference quantity, and a data sub-table with the third difference quantity is created, and the newly created data sub-table and the created data sub-table that does not include the first data to be stored corresponding to the same first time identifier to be stored are assigned to each second data set, and the data sub-tables corresponding to each second data set are different, all the second data to be stored in each second data set are stored in the corresponding data sub-table, and the sub-table identifier of the data sub-table where the second data to be stored corresponding to each second time identifier to be stored is recorded.

[0011] In one embodiment of the present application, all the first data to be stored in each first data set are stored in a corresponding data sub-table, including: obtaining a preset storage data size threshold; if the set data size of all the first data to be stored in the first data set is less than or equal to the preset storage data size threshold, all the first data to be stored in the first data set are stored in the corresponding data sub-table at one time; if the set data size of all the first data to be stored in the first data set is greater than the preset storage data size threshold, all the first data to be stored in the first data set are divided into at least two sets to be stored according to the preset storage data size threshold, the set data size of the first data to be stored in each set to be stored is less than or equal to the preset storage data size threshold, and the first data to be stored in the set to be stored is stored one by one in the corresponding data sub-table.

[0012] An embodiment of the present application also provides a data storage device, which includes: an acquisition module, used to acquire multiple first data to be stored, wherein the first time identifiers to be stored of at least two first data to be stored are the same; a grouping module, used to group the multiple first data to be stored according to the first time identifier to be stored of each first data to be stored to obtain at least two first data sets, wherein the first time identifiers to be stored of the first data to be stored in the same first data set are different; an allocation module, used to create data subtables of the first number of sets of first data sets, and allocate a data subtable to each first data set, wherein the data subtables corresponding to the first data sets are different; a storage module, used to store all the first data to be stored in each first data set into the corresponding data subtable; a recording module, used to record the subtable identifier of the data subtable in which the first data to be stored corresponding to each first time identifier to be stored is stored.

[0013] An embodiment of the present application also provides a data storage system, which includes at least two data storage devices as mentioned in the above embodiment, the first data to be stored acquired by each data storage device is at least partially different, and the data sub-tables stored in each data storage device belong to different super tables.

[0014] An embodiment of the present application further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any one of the above embodiments when executing the computer program.

[0015] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method described in any one of the above embodiments is implemented.

[0016] In the scheme implemented by the data storage method, device, vehicle and storage medium provided above, the method obtains multiple first data to be stored with the same first time to be stored identifier, groups the first data to be stored according to the first time to be stored, obtains at least two first data sets, creates data sub-tables of the number of first data sets, each data sub-table corresponds to a first data set, and then stores the first data to be stored in the first data set in the corresponding data sub-table, and records the sub-table identifier of the data sub-table where the first data to be stored corresponding to each first time to be stored identifier is located. In this way, it can be ensured that the time identifiers of the data stored in the same data sub-table are different, avoiding the problem of data loss caused by the presence of data with the same time identifier in the same sub-table. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0018] Figure 1 A schematic diagram of a flow chart of a data storage method provided in an embodiment of the present application;

[0019] Figure 2 A schematic diagram of a module diagram provided for an embodiment of the present application;

[0020] Figure 3 The following is a schematic diagram of a specific implementation process of a data storage method according to an example;

[0021] Figure 4 A schematic diagram of a time set [ts1, ts7] provided in an embodiment of the present application;

[0022] Figure 5 A schematic diagram of a set identifier corresponding to each time identifier provided in an embodiment of the present application;

[0023] Figure 6 A structural schematic diagram of a data storage device provided in an embodiment of the present application;

[0024] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] See also Figure 1 As shown, Figure 1 A schematic diagram of a data storage method provided in an embodiment of the present application, the method comprising the following steps:

[0027] Step S110, obtaining a plurality of first data to be stored.

[0028] Among them, the first time identifiers of at least two first data to be stored are the same.

[0029] The first data to be stored includes a first time identifier to be stored and first business data. The first time identifier to be stored can be the ts field in the tdengine table, or other time identifier methods set by those skilled in the art. When there are at least two first data to be stored with the same first time identifier to be stored in a batch of data, if a traditional data storage method is used, the previously written data may be overwritten and then lost. Therefore, the data storage method provided in this embodiment can be used to avoid data loss.

[0030] For example, the first data to be stored may be a data field written into the tdengine super table, which includes ts (time, i.e., the first time identifier to be stored) and others (other business data). The data structure of the second data to be stored in the following text is the same as the first data to be stored. Among them, there may be one or more second data to be stored that are the same as the first data to be stored, or they may all be different.

[0031] Step S120: grouping a plurality of first data to be stored according to a first time identifier of each first data to be stored to obtain at least two first data sets.

[0032] The first to-be-stored time identifiers of the first to-be-stored data in the same first data set are different.

[0033] In one embodiment, multiple first data to be stored are grouped according to the first time identifier of each first data to be stored to obtain at least two first data sets, including: repeatedly executing the grouping strategy until the first time identifiers of the first data to be stored that have not yet been added to the first data set are all different, so as to obtain multiple first data sets, wherein the number of first sets of the first data sets is equal to the repetition maximum value, and the repetition maximum value is the maximum value of the number of first data to be stored with the same first time identifier of the first data to be stored in the multiple first data to be stored; wherein the grouping strategy includes sorting the first data to be stored that have not yet been added to the first data set according to the first time identifier to be stored, and selecting one first data to be stored corresponding to each first time identifier to be stored to generate a first data set.

[0034] For example, taking the first time identifier to be stored as field ts as an example, all the first data to be stored are sorted according to field ts, and all ts in the first round cannot be repeated, forming a first data set; the remaining repeated ts data are sorted in the second round, and the data without ts is still selected to form the second first data set; the remaining repeated ts data are sorted in the third round, and the data without ts is still selected to form the third first data set... and so on, and finally a number of sets of data with the largest number of repeated times are formed. The number of first sets of the first data set is the same as the one with the largest number of first data to be stored corresponding to the first time identifier to be stored. For example, in a batch of first data to be stored, the number of first data to be stored corresponding to the first time identifier to be stored A is 10, the number of first data to be stored corresponding to the first time identifier to be stored B is 8, and the number of first data to be stored corresponding to other first time identifiers to be stored is 1, then obviously, the number of first sets of the first data set is 10.

[0035] The grouping method of the second data set described below is similar to the grouping method of the first data set and will not be described in detail.

[0036] Step S130: create a first set number of data sub-tables of the first data set, and allocate a data sub-table to each first data set.

[0037] Among them, each first data set corresponds to a different data subtable. That is, one first data set is stored in one data subtable (i.e., subtable). Since the first data to be stored in different first data sets may have repeated first time identifiers to be stored, by such a grouping method, the first data to be stored in different first data sets are subsequently stored in different subtables, which can avoid the problem of data loss caused by the input written later overwriting the data written earlier due to the conflict of time identifiers.

[0038] Taking TDengineQ and other time series databases as an example, they include ordinary tables, stable tables (super tables) and sub tables. Super tables (called stable tables in TDengine) are a special table structure that is used to define a template for a set of sub tables with the same data structure. Stable tables do not store data themselves, but define the columns (fields) and tags (Tags) of sub tables. Sub tables are specific tables derived from stable tables. They inherit the column structure of stable tables and have their own tag values. Sub tables actually store data, such as a reading of a specific device at a specific time. Using the structure of stable tables and sub tables, it is easy to manage and query a large amount of time series data with similar structures but belonging to different categories (such as different devices, different locations). Sub tables of the same type belong to the same super table; a database corresponds to multiple super tables, and each super table corresponds to multiple sub tables.

[0039] In one embodiment, in an initial state, no sub-table has been created in the super table, so an initial data sub-table may be created based on the first set quantity.

[0040] It can be understood that in the initial state, no sub-table has been created in the super table, that is, there is no pre-created data sub-table at present, so a corresponding number of data sub-tables can be created according to the number of first sets (the number of first data sets), so as to realize that one data sub-table stores the first data to be stored in a first data set.

[0041] As an example, before creating the data sub-tables of the first set number of the first data set, the data storage method further includes: generating a sub-table name corresponding to each first data set according to the set identifier of the first data set; and creating a data sub-table corresponding to each first data set based on the generated sub-table names. The corresponding sub-table name is generated based on the set identifier, so that the sub-table name has a certain association with the first data set in terms of name, which can facilitate subsequent manual viewing, etc. However, in actual use, other sub-table naming rules set by those skilled in the art can also be followed for naming.

[0042] In one embodiment, after storing all the first data to be stored in each first data set into the corresponding data subtable, the data storage method also includes: obtaining multiple second data to be stored, the second data to be stored including a second time identifier to be stored and a second business data; if each second time identifier to be stored is different from each first time identifier to be stored, grouping the multiple second data to be stored according to the second time identifier to be stored of each second data to be stored to obtain one or more second data sets, wherein the second time identifier to be stored of the second data to be stored in the same second data set is different; if the number of second sets of the second data set is less than or equal to the number of first sets, assigning a created data subtable to one second data set, and the number corresponding to each second data set According to the difference in subtables, all the second data to be stored in each second data set are stored in the corresponding data subtable, and the subtable identifier of the data subtable where the second data to be stored corresponding to each second time identifier to be stored is recorded; if the number of second sets of the second data set is greater than the number of first sets, a data subtable of the first difference number is created, and the newly created data subtable and the created data subtable are assigned to each second data set, the first difference number is the difference between the first set number and the second data set, and the data subtables corresponding to each second data set are different, and all the second data to be stored in each second data set are stored in the corresponding data subtable, and the subtable identifier of the data subtable where the second data to be stored corresponding to each second time identifier to be stored is recorded. When there is a new batch of data to be stored, the method in this embodiment can be used, and the first set number is regarded as the number of subtables that have been created, and the first time identifier to be stored is regarded as the time identifier of the stored data that has been stored in each data subtable.

[0043] It is understandable that when a sub-table already exists under the super table, it is necessary to first check whether the previous sub-table has stored the first time identifier to be stored that is the same as the second time identifier to be stored to select a suitable allocation strategy to allocate data to the sub-table.

[0044] When there are already subtables under the super table, the first time mark of the first data to be stored in the created data subtable is different from the second time mark of all the second data to be stored that currently needs to be stored. This means that the second data to be stored that needs to be written later has no time conflict with the stored data that has been written to the created data subtable, and will not cause the loss of the stored data. At this time, two situations need to be discussed. Situation 1.1: The number of each created data subtable (the number of the first set) is more than or equal to the number of the second data sets that currently need to be written (the number of the second set), which means that the created data subtable meets the writing requirements of the current batch of data. At this time, it is only necessary to select the created data subtables of the second set to be assigned to the second data set. As an example, when the number of the first set is greater than the number of the second set, the created data subtables of the second set can be selected according to the priority ranking of the data subtables by those skilled in the art. The created data subtables of the second set can also be randomly selected for allocation. Scenario 1.2: The number of created data sub-tables (the number of first sets) is less than the number of second data sets that need to be written (the number of second sets), which means that the created data sub-tables do not meet the writing requirements of the current batch of data. On the one hand, the existing data sub-tables can meet the requirements of a part of the second data set, and on the other hand, the other part of the second data set cannot be allocated to the data sub-tables. At this time, it is necessary to create a corresponding number of data sub-tables based on the first difference number, and then allocate them to the other part of the first data set. The second data sets in the same batch correspond to the data sub-tables one by one.

[0045] In another embodiment, after storing all the first data to be stored in each first data set into the corresponding data sub-table, the data storage method also includes: obtaining multiple second data to be stored, the second data to be stored including a second time identifier to be stored and a second business data; if the second time identifier to be stored is the same as the first time identifier to be stored, obtaining the number of sub-tables of the data sub-table where the first data to be stored corresponding to the same first time identifier to be stored is located, and obtaining the newly added number of second data to be stored corresponding to the same first time identifier to be stored; determining the second difference number based on the difference between the first set number and the sub-table number; if the newly added number is less than or equal to the second difference number, allocating the newly added number of created data sub-tables that do not include the first data to be stored corresponding to the same first time identifier to be stored to each second data set, and each second number According to the different data sub-tables corresponding to the second data sets, all the second data to be stored in each second data set are stored in the corresponding data sub-table, and the sub-table identifier of the data sub-table in which the second data to be stored corresponding to each second time identifier to be stored is recorded; if the newly added quantity is greater than the second difference quantity, the third difference quantity is determined according to the difference between the newly added quantity and the second difference quantity, and a data sub-table with the third difference quantity is created, and the newly created data sub-table and the created data sub-table that does not include the first data to be stored corresponding to the same first time identifier to be stored are assigned to each second data set. The data sub-tables corresponding to each second data set are different, and all the second data to be stored in each second data set are stored in the corresponding data sub-table, and the sub-table identifier of the data sub-table in which the second data to be stored corresponding to each second time identifier to be stored is recorded.

[0046] Different from the above-mentioned embodiment, in this embodiment, there is data in the second data to be written that needs to be written in a new batch, which has a time conflict with the data already written in the data sub-table. Therefore, it is no longer directly grouped and then written in batches, but the number of data sub-tables with the same time identifier (the number of sub-tables) is obtained, and the number of second data to be stored corresponding to the same time identifier that needs to be newly added (the number of newly added data) is obtained. The second difference number is obtained according to the first set number and the number of sub-tables, which indicates that there are still several idle and available data sub-tables that can be provided for the storage of the new batch of data. If the second difference number is greater than or equal to the number of newly added data, it means that the existing data sub-table can meet the writing requirements of the data corresponding to the time identifier of this batch, and the allocation writing can be directly performed. If the second difference number is less than the number of newly added data, it means that the existing data sub-table cannot meet the writing requirements of the data corresponding to the time identifier of this batch, and it is necessary to newly create a data sub-table with a third difference number, and then allocate the writing. In this way, the writing of data corresponding to the same time identifier in the later batch can be realized, and the data of the original time identifier will not be overwritten.

[0047] Continuing from the above embodiment, if the second time identifier to be stored is the same as the first time identifier to be stored, before obtaining the number of sub-tables of the data sub-table where the first data to be stored corresponding to the same first time identifier to be stored is located, and obtaining the newly added number of second data to be stored corresponding to the same first time identifier to be stored, the data storage method also includes: determining the second data to be stored corresponding to each second time identifier to be stored different from the first time identifier to be stored as batch write data, grouping multiple batch write data according to the second time identifier to be stored of each batch write data to obtain one or more third data sets, wherein the second time identifiers to be stored of the batch write data in the same third data set are different; if the number of third sets of the third data set is less than or equal to the number of the first set, assigning a created third data set to the third data set. A data sub-table is created, the data sub-table corresponding to each third data set is different, all the batch-write data in each third data set is stored in the corresponding data sub-table, and the sub-table identifier of the data sub-table where the batch-write data corresponding to each different second time identifier to be stored is recorded; if the third set quantity of the third data set is greater than the first set quantity, a data sub-table with a fourth difference quantity is created, and the newly created data sub-table and the created data sub-table are assigned to each third data set, the fourth difference quantity is the difference between the first set quantity and the third data set, the data sub-table corresponding to each third data set is different, all the batch-write data in each third data set is stored in the corresponding data sub-table, and the sub-table identifier of the data sub-table where the batch-write data corresponding to each different second time identifier to be stored is recorded.

[0048] It is understandable that the time stamp of a part of the second data to be stored in the later batch may not conflict with the time stamp of the stored data previously stored in the data subtable. In this case, this part of the data can be taken out separately and written in batches with reference to the processing method of the first data to be stored. For the data with time conflicts, that is, the second data to be stored where the time stamp of the second data to be stored is the same as the time stamp of the first data to be stored, the method provided in the above embodiment is used to store it separately. This can effectively improve the writing efficiency without causing data loss.

[0049] Step S140: store all first data to be stored in each first data set into the corresponding data sub-table, and record the sub-table identifier of the data sub-table where the first data to be stored corresponding to each first time identifier to be stored is stored.

[0050] In one embodiment, all first data to be stored in each first data set are stored in a corresponding data sub-table, including: obtaining a preset storage data size threshold; if the set data size of all first data to be stored in the first data set is less than or equal to the preset storage data size threshold, all first data to be stored in the first data set are stored in the corresponding data sub-table at one time; if the set data size of all first data to be stored in the first data set is greater than the preset storage data size threshold, all first data to be stored in the first data set are divided into at least two sets to be stored according to the preset storage data size threshold, the set data size of the first data to be stored in each set to be stored is less than or equal to the preset storage data size threshold, and the first data to be stored in the set to be stored is stored one by one in the corresponding data sub-table.

[0051] It is understandable that there is a certain limit on the amount of data written each time. If the amount of data written at one time is too large, the writing may fail. Therefore, a set of data can be split and then written in batches to ensure successful writing. The preset storage data size threshold is the maximum data size that can be written at one time, which can be set by those skilled in the art according to the writing environment.

[0052] The batch writing of the second data to be stored and the batch writing data may also be implemented by referring to the above embodiment, which will not be elaborated here.

[0053] The data storage method provided in the above embodiment obtains multiple first data to be stored with the same first time to be stored identifier, groups the first data to be stored according to the first time to be stored, obtains at least two first data sets, creates data sub-tables for the number of first data sets, each data sub-table corresponds to a first data set, and then stores the first data to be stored in the first data set in the corresponding data sub-table, and records the sub-table identifier of the data sub-table where the first data to be stored corresponding to each first time to be stored identifier is located, so as to ensure that the time identifiers of the data stored in the same data sub-table are different, thereby avoiding the problem of data loss caused by the presence of data with the same time identifier in the same sub-table.

[0054] The method provided in the above embodiment can disperse all data with the same ts (same time identifier) ​​into different sub-tables for storage, so that data with the same ts time can also be stored in the database and will not be overwritten by subsequent data with the same ts, thereby realizing a solution in which data will not be lost due to the same ts time.

[0055] Assume that multiple processes need to write the data of a certain super table into the tdengine database at the same time. In terms of physical storage, a super table is divided into multiple sub-tables to store data separately. The method provided in the above embodiment takes a certain process as an example, writes data into multiple sub-tables at the same time, and determines the sub-table number of the ts storage through multiple batches of data writing of a process to avoid repeated ts storage in the same sub-table. When a process writes multiple batches of data, it will determine each time whether the currently written ts conflicts with the previously written ts. If there is a conflict, a new sub-table number will be added (a new data sub-table will be created) for writing.

[0056] Since tdengine storage compression ratio is very high, it can save disk space when used to store massive data; since tdengine query method and language are roughly the same as MySQL, the learning cost when querying data is very low and you can get started immediately; when a large number of ts time fields are repeated, but the data are different and need to be stored in the same super table, the data storage method provided in the embodiment of the present application solves the data storage in this scenario without data loss; the data storage method provided in this embodiment solves the problem of massive data storage and query problems, and also solves the problem of no data loss.

[0057] A specific embodiment is provided below to exemplarily illustrate the data storage method provided in the above embodiment.

[0058] Take the example of a super table named stable_name, k processes writing to the stable_name super table, and the data fields (data to be stored) written to the stable_name super table being ts (time) and others (other business data). In order to improve the efficiency of data writing, the data of a super table is stored in multiple processes, and the data of each process is submitted in batches. The name of the sub-table corresponding to the super table stable_name is determined by the subsequent parameters x and y: stable_name_${x}a${y}, and the letter a between the sub-table names ${x} and ${y} means and, and is only a legal character identifier for separating x and y. Among them, the number ${x} created by the sub-table is determined according to the k value. Please refer to the method provided in the above embodiment is executed by the following modules. Figure 2 , Figure 2 A schematic diagram of a module diagram provided in an embodiment of the present application. Figure 2 As shown:

[0059] 1. Sub-table judgment module.

[0060] Module Description: According to the k value, determine the number ${x} value of the subtable.

[0061] Parameter: k.

[0062] Return value: Returns a collection.

[0063] Process: Determine the number of sub-tables created based on the k value and output a data set.

[0064] The output set is: {1,2,3,…,k}.

[0065] The value of subtable number ${x} should be determined for each element in the output set.

[0066] 2. Writer module.

[0067] Module description: According to the batch data obtained by each writer process, the fields ts of all data are sorted. In the first round, all ts cannot be repeated, forming a data set; the remaining repeated ts data are sorted in the second round, and the data with non-repeating ts are still selected to form the second data set; the remaining repeated ts data are sorted in the third round, and the data with non-repeating ts are still selected to form the third data set; ... and so on. Finally, multiple sets of data with the largest number of repeated data are formed. Determine the number ${y} value of the subtable establishment.

[0068] Parameters: The batch data set that each writer process gets.

[0069] Return value: the set with the largest number of repeated ts field data. The ts fields in each set are different.

[0070] 3. Sub-table module.

[0071] Module Description: According to the correspondence between the data set and the sub-table name obtained by the writer module, the corresponding data is inserted into the corresponding sub-table in batches. stable_name_${x}a${y}.

[0072] 4. ts recording module.

[0073] Module description: According to each writer module number, the sub-table information of each ts is recorded separately to avoid writing the same ts data into the same sub-table. Once the ts that appear later are the same as the previous ts, the sub-table information that the subsequent ts should be written into is found based on this module information.

[0074] Parameters: writer number, that is, the natural number of the k value; the value of ts.

[0075] Return value: Returns the corresponding subtable name number ${y} information of the corresponding ts value.

[0076] See also Figure 3 , Figure 3FIG. 1 is a schematic diagram of a specific implementation flow of a data storage method as an example. Figure 3 As shown:

[0077] S1: According to the sub-table judgment module, the sub-table number ${x} data set is obtained: {1,2,3,…,k}.

[0078] S2: Get the data set, data1, data2…data20.

[0079] This can be achieved by executing the writer module. For example, the first writer of k value obtains a batch data set; the data set is as follows:

[0080] data1={ts1,other1};

[0081] data2 = {ts2, other2};

[0082] data3 = {ts3, other3};

[0083] data4 = {ts4, other4};

[0084] data5 = {ts1, other5};

[0085] data6 = {ts2, other6};

[0086] data7 = {ts1, other7};

[0087] data8 = {ts3, other8};

[0088] data9 = {ts5, other9};

[0089] data10 = {ts6, other10};

[0090] data11 = {ts5, other11};

[0091] data12 = {ts1, other12};

[0092] data13 = {ts2, other13};

[0093] data14 = {ts4, other14};

[0094] data15 = {ts1, other15};

[0095] data16 = {ts3, other16};

[0096] data17 = {ts1, other17};

[0097] data18 = {ts7, other18};

[0098] data19 = {ts3, other19};

[0099] data20 = {ts2, other20};

[0100] Note: The batch data set should be appropriately large, covering a sufficient amount of data to maximize the performance of data stored in the database at one time, for example, 2,000 or 5,000 records.

[0101] S3: Sort data1-data20 according to the ts field to obtain the time set [ts1, ts7].

[0102] See also Figure 4 , Figure 4 A schematic diagram of a time set [ts1, ts7] provided in accordance with an embodiment of the present application.

[0103] S4: Get the sets set1-set6 and partition the data according to different ts.

[0104] The data set obtained after 6 rounds of sorting is as follows:

[0105] Set set1={data1:{ts1,other1},data2:{ts2,other2},data3:{ts3,other3},data4:{ts4,other4},data9:{ts5,other9},data10:{ts6,other10},data18:{ts7,other18}};

[0106] Set set2={data5:{ts1,other5},data6:{ts2,other6},data8:{ts3,other8},data14:{ts4,other14},data11:{ts5,other11}};

[0107] Set set3={data7:{ts1,other7},data13:{ts2,other13},data16:{ts3,other16}};

[0108] Set set4={data12:{ts1,other12},data20:{ts2,other20},data19:{ts3,other19}};

[0109] Set set5={data15:{ts1,other15}};

[0110] Set set6={data17:{ts1,other17}}.

[0111] S5: According to the result obtained in S4, the information corresponding to ts (ts data) is stored in the ts recording module.

[0112] See also Figure 5 , Figure 5 A schematic diagram of the set identifiers corresponding to each time identifier provided in an embodiment of the present application, wherein the set below ts represents the subtable ${y} number table in which the corresponding ts value has been stored.

[0113] S6: Determine the batch insert data set and obtain 6 sets of data.

[0114] Connect all the data in each of the six sets obtained by S4 into an insert statement and prepare to insert it into the corresponding subtable. However, it must be noted that the amount of data inserted each time must not exceed the size of the tdengine database (the preset storage data size threshold) in combination with the size of each data entry, to avoid failure of warehousing due to excessive batch data at one time. The size of this data is determined according to the actual situation. Batch insertion into the database is selected to increase the writing speed, otherwise the efficiency of data warehousing is extremely low and loses its practical significance.

[0115] For example, if the set1 data set should be inserted into the database three times, divide the set1 data into three equal parts and use three insert statements for backup.

[0116] For example, if the data set set2 should be inserted into the database twice, divide the data of set2 into two equal parts and use two insert statements for backup.

[0117] The insert data backup data obtained again should be:

[0118] The first group of data: set1-insert1; set1-insert2; set1-insert3;

[0119] The second set of data: set2-insert1; set2-insert2;

[0120] The third set of data: set3-insert1;

[0121] The fourth set of data: set4-insert1;

[0122] The fifth set of data: set5-insert1;

[0123] The sixth set of data: set6-insert1;

[0124] S7: Determine the value of the subtable number ${y}, which is equal to the number of data sets 6.

[0125] Determine that the value of the subtable number ${y} should be the number of sets.

[0126] S8: Determine the subtable number ${x}a${y}.

[0127] For example, 6 is assumed in this case; assuming k = 2; if the above steps S2-S7 are executed by the first process, k is 1, then the subtable name is determined as:

[0128] stable_name_1a1;stable_name_1a2;stable_name_1a3;stable_name_1a4;stable_name_1a5;stable_name_1a6.

[0129] If the second process is executed according to the principle of steps S2-S7, the acquired data finally obtains 3 data sets. At this time, k takes the value of 2, and the corresponding sub-table name can be:

[0130] stable_name_2a1; stable_name_2a2; stable_name_2a3.

[0131] In the above, the naming rule of the subtable is taken as an example, the name suffix is ​​obtained by sequentially arranging and combining the natural number of k and the natural number of set set, and the two values ​​are connected by the letter a. Those skilled in the art can modify and formulate it as needed.

[0132] S9: Determine the insert data set of each subtable.

[0133] Taking the previous stable_name_1a${y} series sub-table names as an example, the data sets they insert are:

[0134] stable_name_1a1=The first set of data: set1-insert1; set1-insert2; set1-insert3;

[0135] stable_name_1a2 = The second set of data: set2-insert1; set2-insert2;

[0136] stable_name_1a3=The third set of data: set3-insert1;

[0137] stable_name_1a4=4th set of data: set4-insert1;

[0138] stable_name_1a5=5th set of data: set5-insert1;

[0139] stable_name_1a6=The sixth set of data: set6-insert1;

[0140] Finally, execute the sub-table module to perform the corresponding data insertion operation to realize data storage.

[0141] S10: The first writer with a k value of 1 obtains new batch data again and sorts the new batch data according to the ts field to obtain a new time set.

[0142] For specific implementation, please refer to step S2 and step S3.

[0143] Determine whether the new time set (this time set) overlaps with the previous one.

[0144] If the minimum time value of the new time set is greater than the maximum time value of the previous time set, step S11 is executed. That is, the new time set does not overlap with the time set obtained in step S3 in the time dimension. This is similar to the case where each second time identifier to be stored is different from each first time identifier to be stored.

[0145] If the minimum time value of this time set is less than or equal to the maximum time value of the previous time set, step S12 is executed. That is, the new time set overlaps with the time set obtained in step S3 in the time dimension. This is similar to the case where the second time identifier to be stored is the same as the first time identifier to be stored.

[0146] S11: If the time set this time is [ts8, ts20], it means that the minimum value of this time set is greater than the maximum value of the previous time set. The new batch data will be executed normally with reference to steps S2-S6, and finally the data will be stored normally.

[0147] S12: If the time set this time is [ts1, ts20], first go to the ts record module to query the subtable number name that has been stored in the database with the same ts.

[0148] If the repeated ts is a value that has not been fully numbered before, write the first unnumbered value as ${y} value and execute step S13.

[0149] If the repeated ts is a value that has been fully numbered before, the ${y} value is determined again by adding one to the number, and step S14 is executed.

[0150] S13: If the repeated ts value is ts4, then {1,2} is found from the ts record module query. The subtable numbers written by ts4 before are 1a1 and 1a2. At this time, the number is not full 6, so the obtained ${y} value should be extended to 3, and the subtable number written is stable_name_1a3. And the corresponding ts record module value is updated to ts4 = {1,2,3}.

[0151] S14: If the repeated ts value is ts1, then the query from the ts record module shows {1,2,3,4,5,6}. The subtable numbers written by ts1 before are 1a1, 1a2, 1a3, 1a4, 1a5, 1a6. At this time, the numbers are full and maximized, so the obtained ${y} value should be added by 1 to get 7, and the subtable number written is stable_name_1a7. And the corresponding ts record module value is updated to ts1 = {1,2,3,4,5,6,7}.

[0152] S15: The determined data is stored in the corresponding sub-table.

[0153] In one embodiment, a data storage device is provided, and the data storage device is used to execute the data storage method provided in any of the above embodiments. Figure 6 , Figure 6 A structural diagram of a data storage device provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, the data storage device 600 includes an acquisition module 601, which is used to acquire multiple first data to be stored, wherein the first time identifiers to be stored of at least two first data to be stored are the same; a grouping module 602, which is used to group the multiple first data to be stored according to the first time identifier to be stored of each first data to be stored to obtain at least two first data sets, wherein the first time identifiers to be stored of the first data to be stored in the same first data set are different; an allocation module 603, which is used to create a data subtable of the first set number of first data sets, and allocate a data subtable to each first data set, wherein the data subtables corresponding to the first data sets are different; a storage module 604, which is used to store all the first data to be stored in each first data set into the corresponding data subtable; a recording module 605, which is used to record the subtable identifier of the data subtable in which the first data to be stored corresponding to each first time identifier to be stored is stored.

[0154] In one embodiment, the grouping module is configured to: repeatedly execute the grouping strategy until the first time identifiers of the first data to be stored that have not yet been added to the first data set are all different, so as to obtain multiple first data sets, wherein the number of first sets of the first data sets is equal to the repetition maximum value, and the repetition maximum value is the maximum value of the number of first data to be stored with the same first time identifier in multiple first data to be stored; wherein the grouping strategy includes sorting the first data to be stored that have not yet been added to the first data set according to the first time identifier to be stored, and selecting a first data to be stored corresponding to each first time identifier to be stored to generate a first data set.

[0155] In one embodiment, the device also includes a data sub-table naming module, which is used to generate a sub-table name corresponding to each first data set according to the set identifier of the first data set before creating a first set number of data sub-tables for the first data set; and create a data sub-table corresponding to each first data set based on the generated sub-table names.

[0156] In one embodiment, the acquisition module is also used to acquire multiple second data to be stored after storing all the first data to be stored in each first data set into the corresponding data sub-table, the second data to be stored including a second time identifier to be stored and a second business data; the storage module is also used to group the multiple second data to be stored according to the second time identifier to be stored of each second data to be stored to obtain one or more second data sets if each second time identifier to be stored is different from each first time identifier to be stored, wherein the second time identifier to be stored of the second data in the same second data set is different; if the number of second sets of the second data set is less than or equal to the number of first sets, assign a created data sub-table to one second data set, and the data sub-table corresponding to each second data set Different, all the second data to be stored in each second data set are stored in the corresponding data sub-table, and the sub-table identifier of the data sub-table in which the second data to be stored corresponding to each second time identifier to be stored is recorded through the recording module; if the number of second sets of the second data set is greater than the number of first sets, a data sub-table of a first difference number is created, and the newly created data sub-table and the created data sub-table are assigned to each second data set, the first difference number is the difference between the number of first sets and the second data set, and the data sub-tables corresponding to each second data set are different, all the second data to be stored in each second data set are stored in the corresponding data sub-table, and the sub-table identifier of the data sub-table in which the second data to be stored corresponding to each second time identifier to be stored is recorded through the recording module.

[0157] In one embodiment, the acquisition module is also used to acquire multiple second data to be stored after storing all the first data to be stored in each first data set into the corresponding data sub-table, and the second data to be stored includes a second time identifier to be stored and a second business data; the storage module is also used to acquire the number of sub-tables of the data sub-table where the first data to be stored corresponding to the same first time identifier to be stored is located, and acquire the newly added number of second data to be stored corresponding to the same first time identifier to be stored if the second time identifier to be stored is the same as the first time identifier to be stored; determine the second difference number based on the difference between the first set number and the sub-table number; if the newly added number is less than or equal to the second difference number, assign the newly added number of created data sub-tables that do not include the first data to be stored corresponding to the same first time identifier to be stored to each second data set, and each second data set corresponds to If the data sub-tables are different, all the second data to be stored in each second data set are stored in the corresponding data sub-table, and the sub-table identifier of the data sub-table in which the second data to be stored corresponding to each second time identifier to be stored is recorded through the recording module; if the newly added quantity is greater than the second difference quantity, the third difference quantity is determined according to the difference between the newly added quantity and the second difference quantity, and a data sub-table with the third difference quantity is created, and the newly created data sub-table and the created data sub-table that does not include the first data to be stored corresponding to the same first time identifier to be stored are assigned to each second data set, and the data sub-tables corresponding to each second data set are different, all the second data to be stored in each second data set are stored in the corresponding data sub-table, and the sub-table identifier of the data sub-table in which the second data to be stored corresponding to each second time identifier to be stored is recorded through the recording module.

[0158] In one embodiment, the storage module is also configured to obtain a preset storage data size threshold; if the collective data size of all the first data to be stored in the first data set is less than or equal to the preset storage data size threshold, all the first data to be stored in the first data set are stored in the corresponding data sub-table at one time; if the collective data size of all the first data to be stored in the first data set is greater than the preset storage data size threshold, all the first data to be stored in the first data set are divided into at least two sets to be stored according to the preset storage data size threshold, the collective data size of the first data to be stored in each set to be stored is less than or equal to the preset storage data size threshold, and the first data to be stored in the set to be stored are stored one by one in the corresponding data sub-table.

[0159] For the specific definition of the data storage device, please refer to the definition of the data storage method above, which will not be repeated here. Each module in the above data storage device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the electronic device in the form of hardware, or can be stored in the memory of the electronic device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0160] In this embodiment, the data storage device is substantially provided with a plurality of modules for executing the data storage method in any of the above embodiments. The specific functions and technical effects may refer to the above embodiments and will not be described in detail here.

[0161] In one embodiment, a data storage system is provided, the data storage system comprising at least two data storage devices as described in the above embodiment, the first data to be stored acquired by each data storage device is at least partially different, and the data sub-tables stored by each data storage device belong to different super tables. The specific functions and technical effects of the data storage system can refer to the above embodiment, and will not be repeated here.

[0162] See also Figure 7 An embodiment of the present invention further provides an electronic device 700, including a processor 701, a memory 702 and a communication bus 703; the communication bus 703 is used to connect the processor 701 and the memory 702; the processor 701 is used to execute a computer program stored in the memory 702 to implement a method as described in any one of the above embodiments.

[0163] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. The computer program is used to enable a computer to execute the method described in any one of the above embodiments.

[0164] The embodiment of the present application also provides a non-volatile readable storage medium, which stores one or more modules (programs). When the one or more modules are applied to a device, the device can execute instructions (instructions) of the steps included in the embodiment 1 of the embodiment of the present application.

[0165] The embodiment of the present application also provides a computer program product, including a computer program, which can implement the steps and corresponding contents of the aforementioned method embodiment when executed by a processor.

[0166] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0167] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0168] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0169] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0170] It should be understood that the terms "first", "second", etc. mentioned in this application are used to distinguish similar objects, and do not necessarily represent that they refer to a specific order or sequence. The technical features in which these terms are located can be interchanged under appropriate circumstances, so that the embodiments of the application described here can be implemented in an order other than the illustrated or written description.

[0171] It should be understood that although the flowcharts provided in the embodiments of the present application indicate the steps by arrows, the order indicated by the arrows does not necessarily limit the order in which the steps are implemented. Those skilled in the art can perform the steps in other orders according to the requirements for different implementation scenarios.

[0172] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A data storage method, characterized in that: The data storage method comprises: Acquire a plurality of first data to be stored, wherein the first time identifiers of at least two of the first data to be stored are the same; Grouping a plurality of first data to be stored according to a first time identifier of each first data to be stored to obtain at least two first data sets, wherein the first time identifiers of the first data to be stored in the same first data set are different; Creating a first set number of data sub-tables of the first data sets, and allocating a data sub-table to each first data set, wherein the data sub-tables corresponding to the first data sets are different; All first data to be stored in each first data set are stored in the corresponding data sub-table, and the sub-table identifier of the data sub-table where the first data to be stored corresponding to each first time identifier to be stored is recorded.

2. The data storage method according to claim 1, characterized in that: The plurality of first data to be stored are grouped according to the first time identifier of each first data to be stored to obtain at least two first data sets, including: Repeat the grouping strategy until the first to-be-stored time identifiers of the first to-be-stored data that have not yet been added to the first data set are all different, thereby obtaining a plurality of first data sets, wherein the number of first sets of the first data sets is equal to a repetition maximum value, and the repetition maximum value is the maximum value of the number of first to-be-stored data with the same first to-be-stored time identifier among the plurality of first to-be-stored data; The grouping strategy includes sorting the first data to be stored that have not been added to the first data set according to the first time identifier to be stored, and selecting one first data to be stored corresponding to each first time identifier to be stored to generate a first data set.

3. The data storage method according to claim 1, characterized in that: Before creating the first set number of data sub-tables of the first data set, the data storage method further includes: Generate a subtable name corresponding to each first data set according to the set identifier of the first data set; A data sub-table corresponding to each first data set is created based on the generated sub-table names.

4. The data storage method according to claim 1, characterized in that: After storing all the first to-be-stored data in each first data set into the corresponding data sub-table, the data storage method further includes: Acquire a plurality of second data to be stored, wherein the second data to be stored includes a second time identifier to be stored and a second business data; If each of the second time identifiers to be stored is different from each of the first time identifiers to be stored, a plurality of second data to be stored are grouped according to the second time identifier to be stored of each second data to be stored to obtain one or more second data sets, wherein the second time identifiers to be stored of the second data to be stored in the same second data set are different; If the number of second sets of second data sets is less than or equal to the number of first sets, a created data sub-table is allocated to each second data set, each second data set corresponds to a different data sub-table, all second data to be stored in each second data set are stored in the corresponding data sub-table, and the sub-table identifier of the data sub-table where the second data to be stored corresponding to each second time identifier to be stored is recorded; If the second set quantity of the second data set is greater than the first set quantity, create a data sub-table of the first difference quantity, and assign the newly created data sub-table and the created data sub-table to each second data set, the first difference quantity is the difference between the first set quantity and the second data set, and the data sub-tables corresponding to each second data set are different. All the second data to be stored in each second data set are stored in the corresponding data sub-table, and the sub-table identifier of the data sub-table where the second data to be stored corresponding to each second time identifier to be stored is recorded.

5. The data storage method according to claim 1, characterized in that: After storing all the first to-be-stored data in each first data set into the corresponding data sub-table, the data storage method further includes: Acquire a plurality of second data to be stored, wherein the second data to be stored includes a second time identifier to be stored and a second business data; If the second time identifier to be stored is the same as the first time identifier to be stored, obtaining the number of sub-tables of the data sub-table where the first data to be stored corresponding to the same first time identifier to be stored is located, and obtaining the number of newly added second data to be stored corresponding to the same first time identifier to be stored; Determine a second difference quantity according to the difference between the first set quantity and the sub-table quantity; If the newly added quantity is less than or equal to the second difference quantity, the newly added quantity of created data sub-tables that do not include the first data to be stored corresponding to the same first time identifier to be stored are allocated to each second data set, the data sub-tables corresponding to each second data set are different, all the second data to be stored in each second data set are stored in the corresponding data sub-table, and the sub-table identifier of the data sub-table in which the second data to be stored corresponding to each second time identifier to be stored is recorded; If the newly added quantity is greater than the second difference quantity, a third difference quantity is determined according to the difference between the newly added quantity and the second difference quantity, and a data sub-table for the third difference quantity is created, and the newly created data sub-table and the created data sub-table that does not include the first data to be stored corresponding to the same first time identifier to be stored are assigned to each second data set, and the data sub-tables corresponding to each second data set are different, and all the second data to be stored in each second data set are stored in the corresponding data sub-table, and the sub-table identifier of the data sub-table where the second data to be stored corresponding to each second time identifier to be stored is recorded.

6. The data storage method according to any one of claims 1 to 5, characterized in that: Storing all first data to be stored in each first data set into the corresponding data sub-table includes: Get the preset storage data size threshold; If the set data size of all the first data to be stored in the first data set is less than or equal to the preset storage data size threshold, all the first data to be stored in the first data set are stored in the corresponding data sub-table at one time; If the set data size of all the first data to be stored in the first data set is greater than the preset storage data size threshold, all the first data to be stored in the first data set are divided into at least two sets to be stored according to the preset storage data size threshold, and the set data size of the first data to be stored in each set to be stored is less than or equal to the preset storage data size threshold, and the first data to be stored in the sets to be stored are stored one by one in the corresponding data subtable.

7. A data storage device, characterized in that: The data storage device comprises: An acquisition module, used for acquiring a plurality of first data to be stored, wherein the first time identifiers of at least two first data to be stored are the same; a grouping module, configured to group a plurality of first data to be stored according to a first time identifier of each first data to be stored to obtain at least two first data sets, wherein the first time identifiers of the first data to be stored in the same first data set are different; an allocation module, configured to create a first set number of data sub-tables of the first data sets, and allocate a data sub-table to each first data set, wherein the data sub-tables corresponding to the first data sets are different; A storage module, used for storing all first data to be stored in each first data set into a corresponding data sub-table; The recording module is used to record the sub-table identifier of the data sub-table where the first data to be stored corresponding to each first time identifier to be stored is stored.

8. A data storage system, characterized in that: The data storage system comprises at least two data storage devices as claimed in claim 7, the first data to be stored acquired by each data storage device is at least partially different, and the data sub-tables stored in each data storage device belong to different super tables.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images