Method and device for optimizing single-point import of time series data in database

Abstract

The invention relates to a method and a device for optimizing single-point import of time series data by a database. The method comprises the following steps: creating a delta table as intermediate storage when a time sequence table is configured; after receiving time sequence data, grouping the time sequence data according to a TSTAG column; setting a preset threshold parameter, and judging each group of data volume: temporarily storing the data volume to a delta table when the data volume is smaller than a threshold, otherwise, directly inserting the data volume into a CU unit; furthermore, when stock data with the same TSTAG value exists in the delta table, a merging process is started, new data and old data are scanned and merged, and if the merged data size exceeds a threshold value, the whole data is transferred to a CU for storage. According to the method, the line memory delta table is introduced to serve as intermediate storage, efficient data routing is achieved through threshold judgment and a dynamic merging mechanism, and the number of generated CUs in a single-point import scene is remarkably reduced, so that the data compression efficiency is improved, the metadata scale is reduced, and the import and query performance of a time sequence table is synchronously improved.

Description

Technical Field

[0001] This application belongs to the field of database data import technology, and specifically relates to a method, apparatus, computer-readable storage medium, and electronic device for optimizing single-point import of time-series data in a database. Background Technology

[0002] In enterprise-level time-series database applications, efficient storage and query performance optimization of time-series data are key technical challenges. Current mainstream enterprise-level relational databases (such as openGauss and Vastbase G100) typically employ a CU (Compression Unit)-based storage architecture when handling the insertion of time-series data into time-series tables. Specifically, time-series data needs to be grouped and aggregated according to the TSTAG (time-series table label column) column, and then sorted according to the TSTIME (time-series table timestamp column) column before being stored in the CU unit. During this process, the database system compresses the data within the CU unit and simultaneously records metadata information related to the CU (such as data range, compression parameters, and statistical information).

[0003] From a performance optimization perspective, the larger the amount of data stored in a single CU unit, the higher the data compression efficiency. Given a constant total data volume, reducing the number of CU units directly reduces the total amount of metadata information. This characteristic brings two advantages: first, reducing the number of CUs reduces the number of disk I / O operations, significantly improving the performance of batch data import; second, reducing the amount of metadata reduces the scanning overhead of metadata during time-series queries, thereby optimizing query response efficiency.

[0004] However, in real-world time-series business scenarios, especially real-time monitoring applications (such as industrial sensors and IoT devices), time-series data is typically generated in TSTAG groups, with each group often containing only a few to a few hundred records. This scenario, characterized by extremely small data insertion volumes, is known as a "single-point import scenario." In this scenario, because each group of time-series data, even after aggregation and sorting, cannot fill a single CU unit, the database system generates a separate CU unit for each group, leading to an exponential increase in the number of CU units. Over time, the storage space of the time-series table will expand rapidly due to the disordered expansion of numerous small CU units, while the scale of metadata information will also continue to climb. This problem ultimately leads to a dual performance bottleneck: on the one hand, the frequent generation and compression operations of CU units during data import exacerbate the IO load, significantly reducing compression efficiency and import performance; on the other hand, the amount of metadata that needs to be scanned during time-series queries surges, significantly extending query response time. Therefore, existing CU-based time-series data storage mechanisms suffer from significant problems of wasted storage resources and performance degradation in single-point import scenarios. Summary of the Invention

[0005] To address the aforementioned problems, this application proposes a novel method for optimizing single-point import of time-series data into a database. The purpose of this invention is to optimize the process of single-point import of time-series data into a database. The core solution involves designing an intermediate storage to store small batches of data, determining whether to insert the data into the delta table or the CU based on the data volume. Simultaneously, an additional merging process is implemented: if data exists in the delta table, the data in the delta table is scanned and then merged and inserted. The main technical strategies of this invention are as follows: 1. For a batch of inserted data, if the amount of data after being grouped by TSTAG is less than a certain preset threshold, it will be stored in the intermediate storage; if the amount of data is not less than this threshold, it will still be inserted into the CU unit. 2. If data exists in the intermediate storage, the newly inserted data needs to be merged with the existing data in the intermediate storage before insertion. The insertion rules for the merged data are the same as in strategy 1.

[0006] Terminology Explanation: TSTAG: Time series table label column, used to identify the classification or grouping of time series data.

[0007] TSTIME: The timestamp column of a time series table, used to record the time points of time series data.

[0008] CU (Compression Unit): A compression unit, the basic unit for storing time-series data.

[0009] delta: Increment.

[0010] delta table: Incremental table.

[0011] Specifically, this application provides the following technical solutions: The first aspect of this application provides a method for optimizing single-point import of time-series data in a database, applied to a time-series database system, wherein the time-series table includes a compression unit (CU) and an intermediate storage unit, and the method includes the following steps: S1. Configure the time series table, wherein the intermediate storage unit is designed as a row-store delta table for temporary storage of small batches of time series data; S2. Receive a batch of time series data to be inserted, wherein the time series data includes a TSTAG column and a TSTIME column; S3. The time series data is grouped according to the TSTAG column to obtain multiple data groups, and each data group corresponds to a unique TSTAG value; S4. Set a preset threshold parameter to determine the storage location of the data group; S5. For each data group, determine whether its data volume is less than the preset threshold; S6. If the amount of data is less than the preset threshold, the data group is stored in the intermediate storage unit; otherwise, the data group is directly stored in the compression unit CU.

[0012] Furthermore, in the method of this application, the preset threshold is set by the parameter vtime_deltarow_threshold, wherein: If the parameter value is -1, then all data groups are stored in the intermediate storage unit; If the parameter value is 0, all data groups are stored in the compression unit CU; If the parameter value is greater than 0, then the parameter value is the preset threshold.

[0013] Furthermore, the method of this application also includes a merge insertion step: Before step S6, if there is existing data in the intermediate storage unit that has the same TSTAG value as the current data group, then the following sub-steps are executed: S6a. Scan the corresponding existing data from the intermediate storage unit according to the TSTAG value of the current data group; S6b: Merge the current data group with the corresponding existing data to obtain the merged data; S6c. Determine whether the amount of data after merging is less than the preset threshold. S6d. If the amount of data after merging is greater than or equal to a preset threshold, the merged data is stored in the compression unit CU, and the existing data in the intermediate storage unit is deleted; if the amount of data after merging is less than the preset threshold, the merged data is stored in the intermediate storage unit.

[0014] Furthermore, in the method of this application, in step S6a, in order to accelerate the scanning of data from the intermediate storage unit, an index based on the TSTAG column is established on the intermediate storage unit.

[0015] Furthermore, the method of this application also maintains a hash table to record the number of data rows corresponding to each TSTAG value in the intermediate storage unit; The hash table is initialized when the intermediate storage unit is first scanned and is updated synchronously when data is inserted or deleted.

[0016] Furthermore, in the method of this application, the time-series database system is an openGauss database or a Vastbase database.

[0017] A second aspect of this application provides a device for optimizing single-point import of time-series data into a database. This device is integrated into a time-series database system, wherein the time-series table of the time-series database system includes a compression unit (CU) and an intermediate storage unit. The device includes: The configuration module is used to configure the time series table, and the intermediate storage unit is designed as a row-store delta table for temporary storage of small batches of time series data. A data receiving module is used to receive time-series data to be inserted, the time-series data including a TSTAG column and a TSTIME column; The grouping module is used to group the time series data to be inserted by the TSTAG column, resulting in multiple data groups, each corresponding to a unique TSTAG value; The threshold setting module is used to set a preset threshold parameter to determine the storage location of the data group; The threshold judgment module is used to determine whether the amount of data in each data group is less than a preset threshold. The storage control module is used to route the data group to the intermediate storage unit or the compression unit CU for storage based on the threshold judgment result; if the data volume is less than the preset threshold, the data group is stored in the intermediate storage unit; otherwise, the data group is directly stored in the compression unit CU.

[0018] The device implements the steps of the aforementioned method for optimizing single-point import of time-series data in a database during operation.

[0019] Furthermore, the device of this application also includes a merging processing module, used for: Detect whether existing data with the same TSTAG value exists in the intermediate storage unit; Perform a data merge operation and re-determine the storage location based on the merged data volume.

[0020] Furthermore, the device of this application also includes: The index management module is used to create and maintain TSTAG column indexes on the intermediate storage unit; The hash table maintenance module is used to dynamically update the mapping relationship between TSTAG values ​​and data row numbers in the intermediate storage unit.

[0021] A third aspect of this application provides an electronic device, including: a memory and a processor; Memory: Used to store computer programs; Processor: Used to execute the computer program to implement the steps of the aforementioned method for optimizing single-point import of time-series data in a database.

[0022] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the steps of the aforementioned method for optimizing single-point import of time-series data in a database.

[0023] In summary, while some existing time-series databases employ intermediate storage mechanisms to optimize time-series data processing, these intermediate storage mechanisms are mostly based on memory caching. Compared to such solutions, this invention innovatively introduces a delta table as intermediate storage, resulting in the following significant advantages: (1) Enhanced transaction consistency and data reliability: This invention uses a delta table as an intermediate storage carrier, which can effectively ensure the atomicity of transactions and data consistency of time-series data during the import process. Compared with the volatile characteristics of memory cache (such as cached data may be lost or inconsistent due to system failure), the structure design of the delta table supports transaction-level persistence and rollback capabilities, ensuring that data remains logically consistent throughout multi-stage processing (such as aggregation, sorting, and compression), and significantly improving the reliability of data storage.

[0024] (2) Reduce memory consumption and improve import efficiency: Compared with memory caching solutions, delta tables, as intermediate storage, consume less memory. Their storage structure, through optimized data organization (such as lightweight data page management and non-contiguous memory allocation), can significantly reduce memory consumption in scenarios involving the import of large volumes of time-series data. This reduction in memory usage directly alleviates the performance bottleneck caused by memory contention during the import process, thereby improving the overall throughput and efficiency of data import.

[0025] In summary, this invention, through the delta table intermediate storage scheme, represents a substantial improvement over existing memory caching schemes in terms of both transaction consistency assurance and efficient utilization of memory resources, providing better technical support for single-point import scenarios of time-series databases.

[0026] Other features and advantages of this application will be set forth in detail in the following description, or will become apparent through the implementation of the relevant technical solutions of this application. The objectives and other advantages of this application can be achieved through the technical features and means explicitly pointed out in the description, claims, and drawings, and will be obtained through the implementation of these technical contents. Attached Figure Description

[0027] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings involved in the description of the embodiments will be briefly introduced below. It should be noted that the accompanying drawings only show some embodiments of this application. For those skilled in the art, other related drawings can be derived from these drawings without creative effort.

[0028] Figure 1 The overall implementation flowchart of the method for optimizing the single-point import of time series data for the database of this application is shown.

[0029] Figure 2 This is a flowchart illustrating the specific implementation process of Strategy 1 in this application.

[0030] Figure 3 This is a flowchart illustrating the specific implementation process of Strategy 2 in this application.

[0031] Figure 4 A structural diagram of the apparatus for optimizing single-point import of time-series data for the database of this application.

[0032] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be noted that the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the protection scope of this application.

[0034] In this document, the term "comprising" and any variations thereof (such as "including," "including," etc.) are open-ended expressions and should be understood as "including but not limited to," meaning that the listed content is not exhaustive and may include other content not explicitly mentioned. The term "based on" should be understood as "at least partially based on," meaning that the basis or condition referred to may not be the only factor and may involve other relevant factors. The term "one embodiment" should be understood as "at least one embodiment," meaning that the described embodiment is not the only possible implementation, and other similar embodiments may exist.

[0035] In this application, the terms "a" and "a plurality of" are used to modify related elements or features, and their expression is illustrative rather than restrictive. Unless otherwise expressly stated in the context, "a" should be understood as "at least one," and "a plurality of" should be understood as "at least two." Those skilled in the art should reasonably interpret these terms based on the semantic and logical relationships of the context to ensure that they cover the possibility of "one or more."

[0036] Figure 1The diagram shows the overall implementation flow of the database optimization method for single-point import of time-series data provided in this application. This method is applied to a time-series database system, whose time-series table includes a compression unit (CU) and an intermediate storage unit. The method includes the following steps: S1. Configure the time series table, wherein the intermediate storage unit is designed as a row-store delta table for temporary storage of small batches of time series data; S2. Receive a batch of time series data to be inserted, wherein the time series data includes a TSTAG column and a TSTIME column; S3. The time series data is grouped according to the TSTAG column to obtain multiple data groups, and each data group corresponds to a unique TSTAG value; S4. Set a preset threshold parameter to determine the storage location of the data group; S5. For each data group, determine whether its data volume is less than the preset threshold; S6. If the amount of data is less than the preset threshold, the data group is stored in the intermediate storage unit; otherwise, the data group is directly stored in the compression unit CU.

[0037] To more clearly illustrate the technical solution of this application, the following will provide further explanation through specific scenario embodiments.

[0038] The purpose of this invention is to optimize the process of importing time-series data from a single point in a database. The core solution involves designing an intermediate storage to store small batches of data, determining whether to insert the data into the delta table or the CU based on the data volume. An additional merging process is also included; if data exists in the delta table, the data in the delta table is scanned and then merged and inserted. The main technical strategies of this invention are as follows: 1. For a batch of inserted data, if the amount of data after being grouped by TSTAG is less than a certain preset threshold, it will be stored in the intermediate storage; if the amount of data is not less than this threshold, it will still be inserted into the CU unit. 2. If data exists in the intermediate storage, the newly inserted data needs to be merged with the existing data in the intermediate storage before insertion. The insertion rules for the merged data are the same as in strategy 1.

[0039] This embodiment uses the openGauss database as an example to describe the specific implementation process of the above strategy: Strategy 1: (1) The intermediate storage is designed as a row-based delta table. When creating the time-series table, a delta table will be created for each partition. (2) Add the table option parameter vtime_deltarow_threshold, which represents the threshold for the number of rows inserted into the delta table in time series. The values ​​and meanings of this parameter are as follows: a value of -1 means that the data is directly inserted into the delta table, a value of 0 means that the data is directly inserted into the CU, and a value greater than 0 means that the actual threshold is used. (3) When vtime_deltarow_threshold is greater than 0, the data needs to be grouped according to TSTAG before insertion, and then the relationship between the size of a single group of data and vtime_deltarow_threshold is judged: if it is less than, it is inserted into the delta table; if it is not less than, it is inserted into the CU. (4) An example of the specific implementation process of this strategy is as follows: Figure 2 As shown in the figure, t is the value of vtime_deltarow_threshold, the skip list is the structure for storing data in memory in groups according to TSTAG, tag_id is the unique ID of each group, and rows is the number of data rows in the group.

[0040] Strategy 2: (1) For the case of vtime_deltarow_threshold>0 in Strategy 1 above, an additional merging process is designed. When it is determined that the number of rows corresponding to a certain tag_id is less than vtime_deltarow_threshold, the data will not be directly inserted into the delta table. Instead, the number of rows corresponding to the tag_id will be scanned from the delta table. If the sum of the two exceeds vtime_deltarow_threshold, this part of the data needs to be scanned from the delta table and then merged with the current data and inserted into the CU. (2) In order to speed up the process of scanning the delta table by tag_id, an index on tag_id is created on the delta table. By scanning the index, the data corresponding to tag_id can be quickly obtained; (3) In order to avoid repeatedly scanning the delta table to obtain the function information corresponding to tag_id, a hash table is established to store tag_id and its corresponding row number in the delta table. This hash table is established when the delta table is scanned for the first time, and the hash table is updated synchronously when inserting or deleting from the delta table. (4) Includes a specific implementation process example of the merger, such as Figure 3 As shown, for the case where tag_id = 2, rows = 50, because there is data in the delta table and the sum of the data in both tables is greater than 100, the merged data will be directly inserted into the CU.

[0041] Figure 4 The diagram shows a device for optimizing single-point import of time-series data according to this application. This device is integrated into a time-series database system. The time-series table of the time-series database system includes a compression unit (CU) and an intermediate storage unit. The device includes: The configuration module is used to configure the time series table, and the intermediate storage unit is designed as a row-store delta table for temporary storage of small batches of time series data. A data receiving module is used to receive time-series data to be inserted, the time-series data including a TSTAG column and a TSTIME column; The grouping module is used to group the time series data to be inserted by the TSTAG column, resulting in multiple data groups, each corresponding to a unique TSTAG value; The threshold setting module is used to set a preset threshold parameter to determine the storage location of the data group; The threshold judgment module is used to determine whether the amount of data in each data group is less than a preset threshold. The storage control module is used to route the data group to the intermediate storage unit or the compression unit CU for storage based on the threshold judgment result; if the data volume is less than the preset threshold, the data group is stored in the intermediate storage unit; otherwise, the data group is directly stored in the compression unit CU.

[0042] The above-mentioned device implements the steps of the database optimization single-point import time series data method disclosed in this application when it is running.

[0043] The flowcharts and block diagrams in the accompanying drawings illustrate possible implementations of apparatus, methods, and computer program products according to various embodiments of this application, including architecture, functionality, and operation. In these figures, each block may represent a module, program segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should be noted that each block in the block diagrams and / or flowcharts, and combinations thereof, can be implemented using either a dedicated hardware-based system or a combination of dedicated hardware and computer instructions to achieve the specified function or operation.

[0044] like Figure 5 As shown in the illustration, an embodiment of this application also discloses an electronic device, including: a processor 310, a communication interface 320, a memory 330 for storing a processor-executable computer program, and a communication bus 340. The processor 310, communication interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 executes the executable computer program to implement the steps of the above-described method for optimizing single-point import of time-series data in a database.

[0045] It is understood that, in addition to memory and a processor, this electronic device may also include input devices (such as a keyboard), output devices (such as a display), and other communication modules. These input devices, output devices, and other communication modules all communicate with the processor through I / O interfaces (i.e., input / output interfaces).

[0046] The operations described in this application can be implemented by writing computer program code using one or more programming languages ​​or a combination thereof. The programming languages ​​include, but are not limited to, the following types: Object-oriented programming languages, such as Java, Smalltalk, C++, etc. Conventional procedural programming languages, such as "C" or similar programming languages.

[0047] The execution methods of program code include, but are not limited to: It runs entirely on the user's computer; Part of it executes on the user's computer, and part of it executes on a remote computer; Execute as a standalone software package; It is executed entirely on a remote computer or server.

[0048] In scenarios involving remote computers, the remote computer can connect to the user's computer via any type of network, including but not limited to local area networks (LANs) or wide area networks (WANs). Furthermore, the remote computer can also connect to external computers through an internet service provider, for example, by utilizing the internet for connection.

[0049] Furthermore, this application also discloses a computer-readable storage medium, wherein when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform the steps of the database optimization single-point import time-series data method disclosed in this application.

[0050] In the context of this application, a computer-readable storage medium refers to a tangible medium capable of storing computer program code and related data. Specific examples include, but are not limited to, the following: (1) Portable computer disk: such as floppy disks and other removable magnetic storage media.

[0051] (2) Hard disk: including mechanical hard disks and solid-state hard disks and other fixed storage devices.

[0052] (3) Random Access Memory (RAM): A volatile storage medium used for temporary storage of data and program code.

[0053] (4) Read-only memory (ROM): a non-volatile storage medium used to store fixed programs and data.

[0054] (5) Erasable programmable read-only memory (EPROM) or flash memory: non-volatile storage media that supports multiple erasures and reprogrammings.

[0055] (6) Fiber optic storage devices: storage media based on fiber optic technology.

[0056] (7) Portable compact disc read-only memory (CD-ROM): a read-only medium that stores data in the form of an optical disc.

[0057] (8) Optical storage devices: such as DVDs, Blu-ray discs and other storage media based on optical principles.

[0058] (9) Magnetic storage devices: such as magnetic tapes, disks and other storage media based on magnetic principles.

[0059] (10) Any suitable combination of the above: for example, combining multiple storage media to meet different storage needs.

[0060] These computer-readable storage media can be used to store the program code and related data described in this application to support program execution and persistent data storage.

[0061] Specifically, according to embodiments of this application, the processes described in the flowcharts can be implemented as computer software programs. For example, embodiments of this application relate to a computer program product comprising a computer program carried on a non-transitory computer-readable medium. This computer program includes program code for performing the database optimization single-point import time-series data method disclosed in this application. When the computer program is executed by a processing device, it can achieve the functions defined in the embodiments of this application.

[0062] While the foregoing discussion contains several specific implementation details, these details should not be construed as limiting the scope of this application. The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this application is not limited to technical solutions formed by specific combinations of the above-described technical features. Furthermore, this application should also cover other technical solutions formed by any combination of the above-described technical features or their equivalents without departing from the foregoing disclosed concept.

[0063] Those skilled in the art should also understand that modifications can be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features, without departing from the spirit and scope of the technical solutions of the embodiments of this application. These modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the core spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for optimizing single-point import of time-series data in a database, applied to a time-series database system, wherein the time-series table includes a compression unit (CU) and an intermediate storage unit, characterized in that... The method includes the following steps: S1. Configure the time series table, wherein the intermediate storage unit is designed as a row-store delta table for temporary storage of small batches of time series data; S2. Receive a batch of time series data to be inserted, wherein the time series data includes a TSTAG column and a TSTIME column; S3. The time series data is grouped according to the TSTAG column to obtain multiple data groups, and each data group corresponds to a unique TSTAG value; S4. Set a preset threshold parameter to determine the storage location of the data group; S5. For each data group, determine whether its data volume is less than the preset threshold; S6. If the amount of data is less than the preset threshold, the data group is stored in the intermediate storage unit; otherwise, the data group is directly stored in the compression unit CU.

2. The method according to claim 1, characterized in that, The preset threshold is set by the parameter vtime_deltarow_threshold, where: If the parameter value is -1, then all data groups are stored in the intermediate storage unit; If the parameter value is 0, all data groups are stored in the compression unit CU; If the parameter value is greater than 0, then the parameter value is the preset threshold.

3. The method according to claim 1, characterized in that, The method also includes a merge insertion step: Before step S6, if there is existing data in the intermediate storage unit that has the same TSTAG value as the current data group, then the following sub-steps are executed: S6a. Scan the corresponding existing data from the intermediate storage unit according to the TSTAG value of the current data group; S6b: Merge the current data group with the corresponding existing data to obtain the merged data; S6c. Determine whether the amount of data after merging is less than the preset threshold. S6d. If the amount of data after merging is greater than or equal to a preset threshold, the merged data is stored in the compression unit CU, and the existing data in the intermediate storage unit is deleted. If the amount of data after merging is less than a preset threshold, the merged data will be stored in the intermediate storage unit.

4. The method according to claim 3, characterized in that, In step S6a, to accelerate the scanning of data from the intermediate storage unit, an index based on the TSTAG column is created on the intermediate storage unit.

5. The method according to claim 4, characterized in that, The method also maintains a hash table to record the number of data rows corresponding to each TSTAG value in the intermediate storage unit; The hash table is initialized when the intermediate storage unit is first scanned and is updated synchronously when data is inserted or deleted.

6. The method according to claim 1, characterized in that, The time-series database system is either the openGauss database or the Vastbase database.

7. A device for optimizing single-point import of time-series data into a database, integrated into a time-series database system, wherein the time-series table of the time-series database system includes a compression unit (CU) and an intermediate storage unit, characterized in that... The device includes: The configuration module is used to configure the time series table, and the intermediate storage unit is designed as a row-store delta table for temporary storage of small batches of time series data. A data receiving module is used to receive time-series data to be inserted, the time-series data including a TSTAG column and a TSTIME column; The grouping module is used to group the time series data to be inserted by the TSTAG column, resulting in multiple data groups, each corresponding to a unique TSTAG value; The threshold setting module is used to set a preset threshold parameter to determine the storage location of the data group; The threshold judgment module is used to determine whether the amount of data in each data group is less than a preset threshold. The storage control module is used to route the data group to the intermediate storage unit or the compression unit CU for storage based on the threshold judgment result; if the data volume is less than the preset threshold, the data group is stored in the intermediate storage unit; otherwise, the data group is directly stored in the compression unit CU.

8. The apparatus according to claim 7, characterized in that, The device also includes a merging processing module, used for: Detect whether existing data with the same TSTAG value exists in the intermediate storage unit; Perform a data merge operation and re-determine the storage location based on the merged data volume.

9. The apparatus according to claim 7, characterized in that, The device also includes: The index management module is used to create and maintain TSTAG column indexes on the intermediate storage unit; The hash table maintenance module is used to dynamically update the mapping relationship between TSTAG values ​​and data row numbers in the intermediate storage unit.

10. An electronic device, characterized in that, include: Memory and processor; Memory: Used to store computer programs; Processor: for executing the computer program to implement the steps of the method for database optimization of single-point import of time-series data as described in any one of claims 1-6.

Images