Data storage method and device based on relational database, equipment and medium

Abstract

The application discloses a data storage method and device based on a relational database, equipment and medium, and relates to the technical field of databases.The method comprises the following steps: obtaining to-be-compressed data of the relational database; using a string table compression algorithm to compress the to-be-compressed data according to preset initial dictionaries and suffix strings corresponding to each target mark stored in the relational database, to obtain encoded data; wherein the target mark is a mark added to the preset initial dictionaries in the compression process, and the suffix string comprises characters behind each target mark when the target mark is added to the preset initial dictionaries; and storing the encoded data in the relational database.The application uses the string table compression algorithm to compress to-be-compressed data in the relational database, which can improve the data compression efficiency, save storage space and improve the storage efficiency; and the calling of the suffix string corresponding to each target mark can improve the configuration efficiency of the marks in the dictionaries and further improve the compression efficiency of repeated word groups.

Description

Technical Field

[0001] This invention relates to the field of database technology, and in particular to a data storage method, apparatus, device, and medium based on a relational database. Background Technology

[0002] With the continuous development of the internet and information technology, the importance of databases is becoming increasingly prominent. Databases are a technology used to store and manage data, applicable to various scenarios such as business, government, and personal applications. Currently, databases are handling increasingly large volumes of data, requiring the storage of massive amounts of data. For example, the daily storage volume of raw data can reach hundreds or even thousands of gigabytes. Given the vast amount of data and limited storage resources, it is necessary to compress the raw data to significantly reduce its size and save storage space.

[0003] Currently, most cluster systems use row-based methods for real-time data acquisition and caching, such as... Figure 1 The compression of massive amounts of data typically employs methods such as row compression, page compression, and dictionary compression to save disk space and improve storage efficiency. One approach involves the following steps: 1. In a cluster system, a non-relational database is deployed on each node to record data. 2. Row and page compression are implemented in the non-relational database through table design and indexes, while dictionary compression is achieved using specific data types. 3. Data from each node is recorded and written to system memory, with a maximum recording time of half an hour. This process is repeated, overwriting existing data, and a database plugin is used to call the non-relational database and record the data in the database tables. However, since row compression, page compression, and dictionary compression are not highly efficient and consume significant storage resources, improving data compression efficiency, saving storage space, and enhancing storage efficiency are urgent problems that need to be addressed. Summary of the Invention

[0004] The purpose of this invention is to provide a data storage method, apparatus, device, and computer-readable storage medium based on a relational database, so as to improve data compression efficiency, save storage space, and enhance storage efficiency.

[0005] To address the aforementioned technical problems, this invention provides a data storage method based on a relational database, comprising:

[0006] Retrieve data to be compressed from a relational database;

[0007] Based on the preset initial dictionary and the suffix strings corresponding to each target tag stored therein, the data to be compressed is compressed using a string list compression algorithm to obtain encoded data; wherein, the target tag is a tag added to the preset initial dictionary during the compression process, and the suffix string includes the characters following each target tag when it is added to the preset initial dictionary;

[0008] The encoded data is stored in the relational database.

[0009] On the other hand, the step of compressing the data to be compressed using a string table compression algorithm based on a preset initial dictionary and the suffix strings corresponding to each target tag to obtain encoded data includes:

[0010] The preset initial dictionary is updated based on the suffix matching length corresponding to the current target tag matched by the current string; wherein, the suffix matching length is the number of characters in the current string that have the same position content as the suffix string corresponding to the current target tag after the current string.

[0011] On the other hand, updating the preset initial dictionary based on the suffix matching length corresponding to the current target tag matched by the current string includes:

[0012] If the current string matches the current target tag, then a new character is added to the current string to obtain the updated current string;

[0013] Determine whether the updated current string matches a marker in the preset initial dictionary;

[0014] If the updated current string does not match the marker in the preset initial dictionary, then the suffix string corresponding to the stored current target marker is used to match the current string and obtain the suffix matching result;

[0015] If the suffix matching result is a match, then a new character is added to the current string to obtain the updated current string, and the step of matching the current string with the suffix string corresponding to the stored current target tag to obtain the suffix matching result is executed;

[0016] If the suffix matching result is not matched, then the suffix matching length of the current target tag corresponding to the current string is determined, the first tag is output in the encoding stream, and the second tag corresponding to the suffix matching length of the current string is added to the preset initial dictionary; wherein, the prefix of the first tag is the current target tag, and the suffix is ​​the suffix matching length; the second tag is the tag of the current target tag and the string corresponding to each number of matched suffix characters.

[0017] On the other hand, after compressing the data to be compressed using a string table compression algorithm based on a preset initial dictionary and the suffix strings corresponding to each target tag stored in the database to obtain encoded data, the process further includes:

[0018] The newly added tags in the preset initial dictionary are adjusted according to their usage frequency.

[0019] On the other hand, the suffix string includes a preset number of characters following each of the target tags when they are added to the preset initial dictionary.

[0020] On the other hand, the process of obtaining the data to be compressed from the relational database includes:

[0021] Based on the storage time of the data stored in the database tables of the relational database, the stored data is divided into their respective corresponding partition tables;

[0022] The stored data in each target partition table is used as the data to be compressed; wherein the storage time of the stored data in the target partition table is later than the storage time of the stored data in the non-target partition table; the partition table includes the target partition table and the non-target partition table.

[0023] On the other hand, the step of using the stored data in each target partition table as the data to be compressed includes:

[0024] The target partition table is determined by merging the partition tables; wherein the storage time of the data in the merged partition tables is earlier than the storage time of the data in the non-merged partition tables; the target partition table includes the merged partition tables and the non-merged partition tables.

[0025] The target multi-row data in the merged partition table is converted into their respective single-row compressed data to obtain the converted merged partition table.

[0026] The stored data in each of the non-merged partition tables and the converted merged partition tables are used as the data to be compressed.

[0027] The present invention also provides a data storage device based on a relational database, comprising:

[0028] The acquisition module is used to acquire the data to be compressed from the relational database.

[0029] The compression module is used to compress the data to be compressed using a string table compression algorithm based on a preset initial dictionary and the suffix characters corresponding to each target tag stored therein, to obtain encoded data; wherein, the target tag is a tag added to the preset initial dictionary during the compression process, and the suffix characters include the characters following each target tag when it is added to the preset initial dictionary;

[0030] A storage module is used to store the encoded data into the relational database.

[0031] The present invention also provides a data storage device based on a relational database, comprising:

[0032] Memory, used to store computer programs;

[0033] A processor is configured to implement the steps of the data storage method based on a relational database as described above when executing the computer program.

[0034] In addition, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the data storage method based on a relational database as described above.

[0035] The present invention provides a data storage method based on a relational database, comprising: acquiring data to be compressed from the relational database; compressing the data to be compressed using a string table compression algorithm according to a preset initial dictionary and the suffix strings corresponding to each target tag stored therein, to obtain encoded data; wherein, the target tag is a tag added to the preset initial dictionary during the compression process, and the suffix string includes the characters following each target tag when it is added to the preset initial dictionary; and storing the encoded data in the relational database.

[0036] As can be seen, this invention uses a string table compression algorithm in a relational database to compress the data to be compressed, which can improve data compression efficiency, save storage space, and improve storage efficiency. Furthermore, by calling the suffix strings corresponding to each target tag during the compression process, the configuration efficiency of tags in the dictionary can be improved, further enhancing the compression efficiency of repeated phrases. In addition, this invention also provides a data storage device, apparatus, and computer-readable storage medium based on a relational database, which also possesses the aforementioned beneficial effects. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0038] Figure 1 This is a schematic diagram of a conventional data compression and storage method for a cluster system;

[0039] Figure 2 A flowchart illustrating a data storage method based on a relational database provided in an embodiment of the present invention;

[0040] Figure 3 This is a flowchart illustrating another data storage method based on a relational database provided in an embodiment of the present invention.

[0041] Figure 4 A flowchart illustrating the dictionary update process of another data storage method based on a relational database provided in an embodiment of the present invention;

[0042] Figure 5 This is a structural block diagram of a data storage device based on a relational database provided in an embodiment of the present invention;

[0043] Figure 6 This is a simplified structural diagram of a data storage device based on a relational database, provided in an embodiment of the present invention.

[0044] Figure 7 This is a schematic diagram of the specific structure of a data storage device based on a relational database, provided in an embodiment of the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Please refer to Figure 2 , Figure 2 A flowchart illustrating a data storage method based on a relational database, provided as an embodiment of the present invention. The method may include:

[0047] Step 101: Obtain the data to be compressed from the relational database.

[0048] In this embodiment, the relational database can be a database that uses a relational model for data organization and management. Data in the relational database is represented in the form of a two-dimensional table, where each row represents a data record and each column represents a data attribute. This embodiment does not limit the specific type of relational database; for example, it can use PostgreSQL (an object-relational database management system) or other types of relational databases.

[0049] Correspondingly, the data to be compressed in this embodiment can be data that needs to be compressed and stored in a relational database, such as data on nodes (e.g., servers) of a cluster system. Figure 3 As shown, in this implementation example, a PostgreSQL database (i.e., a relational database) can be deployed and installed on the nodes of the cluster system. The data generation module can periodically collect and generate data, writing it into memory, such as recording data from half an hour in memory and overwriting it cyclically. Through a PostgreSQL database plugin, the deployed PostgreSQL database is called to record the data into database tables, thus compressing and storing the data in the database tables (i.e., the stored data). In other words, this step could potentially precede the deployment and installation of a relational database on the nodes of the cluster system.

[0050] Correspondingly, the specific method by which the processor obtains the data to be compressed from the relational database in this step can be set by the designer according to the usage scenario and user needs. For example, the processor can directly use the data stored in the database tables of the relational database as the data to be compressed. Since the computation time required for any database query or update is related to the size of the data, the smaller the data size, the less computation time is required. For massive amounts of data, this embodiment can adopt the divide-and-conquer approach, dividing the data into smaller and larger tables. Therefore, in this embodiment, the data in the database tables can be stored in the form of partitioned tables. That is, by splitting a large database table into multiple smaller partitioned tables, the original database table containing all time data is divided into several database tables (i.e., partitioned tables) according to time periods. Each partitioned table contains only data for a fixed time period, and the data items are arranged in the original time order. This improves query performance and data management efficiency, and different partitioned tables can be compressed independently to further achieve data compression.

[0051] For example, in this step, the processor can divide the stored data into corresponding partition tables based on the storage time of the data stored in the database tables of the relational database; the stored data in each target partition table is used as the data to be compressed; wherein, the storage time of the stored data in the target partition table is later than the storage time of the stored data in the non-target partition table; the partition table includes the target partition table and the non-target partition table; that is, in this embodiment, the stored data in the target partition table can be compressed using the String-Zoo Compression (LZW) algorithm, and the stored data in the non-target partition table can be compressed and stored using a lossless compression algorithm or a lossy compression algorithm with a lower compression ratio to reduce storage space due to its longer storage time.

[0052] To further reduce the compression ratio, this embodiment can merge multiple rows of stored data in a database table into a single row of stored data. For example, the process of using the stored data in each target partition table as the data to be compressed can include: determining the merged partition table in the target partition table; wherein the storage time of the stored data in the merged partition table is earlier than the storage time of the stored data in the non-merged partition table; the target partition table includes the merged partition table and the non-merged partition table; converting each target multi-row data in the merged partition table into its corresponding single-row compressed data to obtain the converted merged partition table; and using the stored data in each non-merged partition table and the converted merged partition table as the data to be compressed. In other words, this embodiment can divide the target partition table that needs to be compressed using the LZW algorithm into a non-merged partition table that can be directly compressed and a merged partition table that needs to be merged before compression, so as to convert multiple rows (e.g., 1000 rows) of data in the merged partition table with a longer storage time into single-row data (i.e., single-row compressed data). Correspondingly, this embodiment can also segment the single-row compressed data according to specific columns, so that each compressed row corresponds to the data of a single business item.

[0053] Correspondingly, the specific method for converting each target multi-row data in the merged partition table into its corresponding single-row compressed data to obtain the converted merged partition table, i.e. the specific method for determining the target multi-row data, can be set by the designer. For example, the multi-row data of the same node can be determined as the target multi-row data, as shown in Tables 1 and 2 below, and the multi-row data corresponding to node 1 can be converted into single-row data.

[0054] Table 1. Merged Partition Table Before Merging

[0055] Table 2 Merged Partition Table

[0056] like Figure 3As shown, the table partitioning module and data merging module can be used to periodically partition historical data and merge multiple rows of data, further reducing the data compression rate and improving processing efficiency.

[0057] Step 102: Based on the preset initial dictionary and the suffix strings corresponding to each target tag stored in the database, the data to be compressed is compressed using a string table compression algorithm to obtain encoded data; wherein, the target tag is the tag added to the preset initial dictionary during the compression process, and the suffix string includes the characters following each target tag when it is added to the preset initial dictionary.

[0058] It is understood that the String List Compression (LZW) algorithm in this embodiment achieves lossless data compression by establishing a string table and using shorter codes to represent longer strings. The LZW algorithm has three important objects: the data stream, the encoding stream, and the compilation table (i.e., the string table). During encoding, the data stream is the input object, and the encoding stream (the encoded data after compression) is the output object. During decoding, the encoding stream is the input object, and the data stream is the output object. The compilation table is an object that is needed during both encoding and decoding. In this embodiment, the preset initial dictionary can be the compilation table in the initial state during encoding and decoding, and the preset initial dictionary can include all possible roots.

[0059] Correspondingly, to increase the data compression rate to save disk space and improve data storage efficiency, this embodiment uses the extended functions of relational databases (such as user-defined functions) to implement the string table compression algorithm. That is, it uses the external table function of the relational database combined with a program that supports LZW algorithm compression to compress and process data, thereby achieving lossless compression and decompression. The LZW algorithm compression process can be as follows: ① Initially, the initial dictionary is preset to contain all possible roots. At this time, the prefix string P and the current character C are both empty. ② Read in the new character C and merge it with P to form the string P+C. ③ Determine if P+C is in the dictionary; if "yes", then P=P+C, return to step ②; if "no", then output the mapping of P, P=P+C, add the marker of the prefix string P to the dictionary, establish the mapping, and let P=C / / (the current P only contains one character C). ④ Determine if there are any more codewords to be decoded in the codeword stream; if "yes", then return to step ②; if "no", then output the codeword representing the current prefix P to the codeword stream; the process ends.

[0060] In this embodiment, to improve the configuration efficiency of markers in the dictionary and further enhance the compression efficiency of repeated phrases, the matching rules of the LZW algorithm can be optimized using the stored suffix strings corresponding to each target marker. This improves the configuration efficiency of markers in the dictionary by calling and matching the suffix strings corresponding to each target marker during the compression process. For example, in this step, the preset initial dictionary can be updated based on the suffix matching length corresponding to the current target marker matched by the current string; where the suffix matching length is the number of characters in the current string that have the same position content as the suffix string corresponding to the current target marker.

[0061] For example, in some embodiments, after a marker match is found, the processor may not immediately end the matching process. Instead, it may search in a sliding window for a better match in the stored suffix string. If a better match is found, it is written to the output stream (such as the encoded stream) in the form of {position, suffix match length}, and each matched string is added as a marker. For example, if the beginning of the data to be compressed contains the character "vdisk", the first time the following marker flag1 will be generated: {v,d}. If "vdisk" is matched again during the matching process, the conventional LZW algorithm will only match flag1: {v,d}, and add a marker flagX: {vd,i}, i.e., vdi; thus, "vdisk" must appear multiple times to put the entire word into a single marker. To optimize this issue, when "vdisk" is first encountered and flag1 ({v,d}) is generated, the corresponding suffix string (such as isk, or the complete vdisk) can be stored. After "vdisk" is encountered a second time, if the newly added "i" character in the vdi string does not have a matching tag in the dictionary after matching flag1{v,d} (the current target tag), flag1 is not directly output. Instead, the search continues along the suffix string corresponding to flag1 to see if there are more matches. If more matches are found, a tag {flag1, suffix match length} is output to the encoding stream. Simultaneously, the three tags corresponding to "vdisk" (i.e., the suffix match length) – {vd,i}, {vdi,s}, and {vdis,k} – are added to the dictionary as new tags. This way, when "vdisk" is matched again, the corresponding tag can be output directly.

[0062] It should be noted that the target marker in this embodiment can be all or part of the markers added to the preset initial dictionary during the compression process (i.e., newly added markers). For example, if the target marker can be a partially added marker, such as if the current string has only matched the initial markers in the preset initial dictionary (such as all possible roots) during the compression process, but has not matched any newly added markers, then when the current string cannot match the markers in the preset initial dictionary, the markers of the current string (i.e., newly added markers) can be added to the preset initial dictionary, and the marker can be used as the target marker to store the suffix string corresponding to the current string.

[0063] Correspondingly, the specific content of the suffix strings corresponding to each target tag stored in this embodiment can be set by the designer. For example, the suffix string corresponding to each target tag may include a preset number of characters following each target tag when it is added to the preset initial dictionary, that is, a preset number of characters following its position; or it may include the latter part of the characters of the word in which each target tag is added to the preset initial dictionary. Accordingly, the suffix string corresponding to each target tag stored may also include the character content of each target tag itself. This embodiment does not impose any restrictions on this.

[0064] In this step, pre-stored words or phrases (i.e., preset strings) can be further utilized to improve the configuration efficiency of markers in the dictionary and further improve the compression efficiency of repeated phrases. For example, in this embodiment, the processor can read the preset string corresponding to the obtained string configuration instruction into memory, thereby enabling it to compress the data to be compressed using a string table compression algorithm based on the preset initial dictionary, the preset strings in memory, and the suffix strings corresponding to each target marker, to obtain encoded data. In other words, users can configure the corresponding preset strings through string configuration instructions according to the business scenario of the data to be compressed and stored in the relational database, in order to further improve the data compression efficiency. Correspondingly, the processor can also update the preset initial dictionary according to the preset string matching length matched by the current string; where the matching length is the number of characters after the current string that have the same content as the content in the matched preset string.

[0065] Furthermore, in this embodiment, after step 102, the newly added tags in the preset initial dictionary can be adjusted according to their usage frequency. That is, in this embodiment, after compression, instead of directly deleting the entire dictionary (i.e., the preset initial dictionary with added tags), the newly added tags with lower usage frequency are deleted according to their usage frequency, while the newly added tags with higher usage frequency are retained, in order to increase the efficiency of subsequent data compression and decompression.

[0066] Step 103: Store the coded data in a relational database.

[0067] It is understood that in this embodiment, the processor can store the encoded stream (i.e., encoded data) compressed using the LZW algorithm in a relational database to improve the storage performance of the relational database.

[0068] Accordingly, in this implementation, after the processor stores the encoded data in a relational database, it can also organize the encoded data (such as index organization) to facilitate subsequent retrieval. For example... Figure 3 As shown, the processor can organize the data files corresponding to each of the partition tables to ensure data consistency.

[0069] In this embodiment, the present invention uses a string table compression algorithm to compress the data to be compressed in a relational database, which can improve data compression efficiency, save storage space and improve storage efficiency; and by calling the suffix strings corresponding to the target tags during the compression process, the configuration efficiency of the tags in the dictionary can be improved, further improving the compression efficiency of repeated phrases.

[0070] Based on the above method embodiments, the specific process of updating the preset initial dictionary according to the suffix matching length corresponding to the current target tag matched by the current string in the above embodiments can be as follows: Figure 4 As shown, it includes:

[0071] Step 1021: If the current string matches the current target token, then add a new character to the current string to obtain the updated current string.

[0072] Step 1022: Determine whether the updated current string matches the marker in the preset initial dictionary; if not, proceed to step 1023.

[0073] Step 1023: Use the suffix string corresponding to the current target tag stored to match the current string and obtain the suffix matching result; if yes, proceed to step 1024; if no, proceed to step 1025.

[0074] Step 1024: If the suffix matching result is a match, then add a new character to the current string to obtain the updated current string, and proceed to step 1023;

[0075] Step 1025: If the suffix matching result is no match, determine the suffix matching length of the current target tag corresponding to the current string, output the first tag in the encoding stream, and add the second tag corresponding to the suffix matching length of the current string to the preset initial dictionary.

[0076] The first marker has a prefix of the current target marker and a suffix of the suffix matching length; the second marker is a marker for the string corresponding to the current target marker and the matched suffix characters.

[0077] It is understood that in this embodiment, when the current string (such as the current character C mentioned above) does not match a marker in the preset initial dictionary, and the target marker in the preset initial dictionary (i.e. the current target marker) was matched last time, the suffix string corresponding to the current target marker stored in memory is used to match the current string and subsequent characters in sequence, so as to improve the configuration efficiency of the markers in the dictionary.

[0078] Corresponding to the above method embodiments, this invention also provides a data storage device based on a relational database. The data storage device based on a relational database described below and the data storage method based on a relational database described above can be referred to and corresponded to each other.

[0079] Please refer to Figure 5 , Figure 5 This is a structural block diagram of a data storage device based on a relational database, provided in an embodiment of the present invention. The device may include:

[0080] Module 10 is used to acquire the data to be compressed from the relational database;

[0081] Compression module 20 is used to compress the data to be compressed using a string table compression algorithm based on a preset initial dictionary and the suffix characters corresponding to each target tag stored in the database to obtain encoded data; wherein, the target tag is the tag added to the preset initial dictionary during the compression process, and the suffix characters include the characters following each target tag when it is added to the preset initial dictionary;

[0082] Storage module 30 is used to store coded data in a relational database.

[0083] In some embodiments, the compression module 20 may include:

[0084] The dictionary update submodule is used to update the preset initial dictionary based on the suffix matching length corresponding to the current target tag matched by the current string; where the suffix matching length is the number of characters in the current string that have the same content at the same position as the suffix string corresponding to the current target tag.

[0085] In some embodiments, the dictionary update submodule may include:

[0086] The first update unit is used to add a new character to the current string if the current string matches the current target tag, so as to obtain the updated current string;

[0087] The judgment unit is used to determine whether the updated current string matches a marker in the preset initial dictionary;

[0088] The suffix matching unit is used to match the current string with the suffix string corresponding to the current target tag if the updated current string does not match the tag in the preset initial dictionary, and obtain the suffix matching result.

[0089] The second update unit is used to add a new character to the current string if the suffix matching result is a match, so as to obtain the updated current string and send a start signal to the suffix matching unit.

[0090] The dictionary update unit is used to determine the suffix matching length of the current target token corresponding to the current string if the suffix matching result is not matched, output the first token in the encoded stream, and add the second token corresponding to the suffix matching length of the current string to the preset initial dictionary; wherein, the prefix of the first token is the current target token, and the suffix is ​​the suffix matching length; the second token is the token of the string corresponding to the current target token and the matched suffix characters of each number.

[0091] In some embodiments, the device may further include:

[0092] The adjustment module is used to adjust the newly added tags in the preset initial dictionary according to the usage frequency of each newly added tag in the preset initial dictionary.

[0093] In some embodiments, the suffix string includes a preset number of characters following each target tag when it is added to a preset initial dictionary.

[0094] In some embodiments, the acquisition module 10 includes:

[0095] The partitioning submodule is used to divide the stored data into corresponding partition tables based on the storage time of the data stored in the database tables of the relational database.

[0096] The determination submodule is used to select the stored data in each target partition table as the data to be compressed; wherein the storage time of the stored data in the target partition table is later than the storage time of the stored data in the non-target partition table; the partition table includes the target partition table and the non-target partition table.

[0097] In some embodiments, the stored data in each target partition table is used as the data to be compressed, including:

[0098] The merge determination unit is used to determine the merged partition table in the target partition table; wherein the storage time of the stored data in the merged partition table is earlier than the storage time of the stored data in the non-merged partition table; the target partition table includes the merged partition table and the non-merged partition table.

[0099] The data merging unit is used to convert the target multi-row data in the merged partition table into their respective single-row compressed data to obtain the converted merged partition table.

[0100] The data determination unit is used to take the stored data in each non-merged partition table and the converted merged partition table as the data to be compressed.

[0101] In this embodiment, the present invention uses a string table compression algorithm to compress the data to be compressed in a relational database, which can improve data compression efficiency, save storage space and improve storage efficiency; and by calling the suffix strings corresponding to the target tags during the compression process, the configuration efficiency of the tags in the dictionary can be improved, further improving the compression efficiency of repeated phrases.

[0102] Corresponding to the above method embodiments, this invention also provides a data storage device based on a relational database. The data storage method and device based on a relational database described below can be referred to in correspondence with the data storage method based on a relational database described above.

[0103] Please refer to Figure 6 , Figure 6 This is a simplified structural diagram of a data storage device based on a relational database, provided as an embodiment of the present invention. The device may include:

[0104] Memory D1 is used to store computer programs;

[0105] Processor D2 is used to implement the steps of the data storage method based on a relational database provided in the above method embodiments when executing a computer program.

[0106] Accordingly, please refer to Figure 7 , Figure 7This is a schematic diagram illustrating the specific structure of a data storage device based on a relational database, as provided in an embodiment of the present invention. The data storage device 310 can vary significantly due to different configurations or performance characteristics. It may include one or more central processing units (CPUs) 322 (e.g., one or more processors) and a memory 332, and one or more storage media 330 (e.g., one or more mass storage devices) storing application programs 342 or data 344. The memory 332 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more units (not shown in the diagram), each unit including a series of instruction operations on the host. Furthermore, the central processing unit 322 may be configured to communicate with the storage media 330 and execute the series of instruction operations in the storage media 330 on the data storage device 310.

[0107] The data storage device 310 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input / output interfaces 358, and / or one or more operating systems 341, such as Linux systems.

[0108] In this embodiment, the data storage device 310 can specifically be a node device, such as a server, that has a relational database deployed in the cluster system.

[0109] The steps in the relational database-based data storage method described above can be implemented by the structure of a relational database-based data storage device.

[0110] Corresponding to the above method embodiments, this invention also provides a computer program product. The computer program product described below and the data storage method based on a relational database described above can be referred to and correspond to each other.

[0111] A computer program product includes a computer program / instructions that, when executed by a processor, implement the steps of the data storage method based on a relational database provided in the above-described method embodiments.

[0112] Corresponding to the above method embodiments, this invention also provides a computer-readable storage medium. The computer-readable storage medium described below corresponds to and can be referred to in relation to the data storage method based on a relational database described above.

[0113] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the data storage method based on a relational database as described in the above method embodiments.

[0114] The computer-readable storage medium can specifically be a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or any other readable storage medium capable of storing program code.

[0115] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatuses, devices, computer program products, and computer-readable storage media disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant details can be found in the method section.

[0116] The foregoing has provided a detailed description of a data storage method, apparatus, device, and computer-readable storage medium based on a relational database provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the present invention.

Claims

1. A data storage method based on a relational database, characterized in that, include: Retrieve data to be compressed from a relational database; Based on the preset initial dictionary and the suffix strings corresponding to each target tag stored therein, the data to be compressed is compressed using a string table compression algorithm to obtain encoded data; wherein, the target tag is a tag added to the preset initial dictionary during the compression process, and the suffix string includes a preset number of characters following each target tag when it is added to the preset initial dictionary; The encoded data is stored in the relational database; The step involves compressing the data to be compressed using a string table compression algorithm based on a preset initial dictionary and the suffix strings corresponding to each target tag, to obtain encoded data, including: The preset initial dictionary is updated based on the suffix matching length corresponding to the current target tag matched by the current string; wherein, the suffix matching length is the number of characters in the current string that have the same position content as the suffix string corresponding to the current target tag after the current string.

2. The data storage method based on a relational database according to claim 1, characterized in that, The step of updating the preset initial dictionary based on the suffix matching length corresponding to the current target tag matched in the current string includes: If the current string matches the current target tag, then a new character is added to the current string to obtain the updated current string; Determine whether the updated current string matches a marker in the preset initial dictionary; If the updated current string does not match the marker in the preset initial dictionary, then the suffix string corresponding to the stored current target marker is used to match the current string and obtain the suffix matching result; If the suffix matching result is a match, then a new character is added to the current string to obtain the updated current string, and the step of matching the current string with the suffix string corresponding to the stored current target tag to obtain the suffix matching result is executed; If the suffix matching result is not matched, then the suffix matching length of the current target tag corresponding to the current string is determined, the first tag is output in the encoding stream, and the second tag corresponding to the suffix matching length of the current string is added to the preset initial dictionary; wherein, the prefix of the first tag is the current target tag, and the suffix is ​​the suffix matching length; the second tag is the tag of the current target tag and the string corresponding to each number of matched suffix characters.

3. The data storage method based on a relational database according to claim 1, characterized in that, After compressing the data to be compressed using a string table compression algorithm based on a preset initial dictionary and the suffix strings corresponding to each target tag to obtain encoded data, the process further includes: The newly added tags in the preset initial dictionary are adjusted according to their usage frequency.

4. The data storage method based on a relational database according to any one of claims 1 to 3, characterized in that, The process of obtaining the data to be compressed from the relational database includes: Based on the storage time of the data stored in the database tables of the relational database, the stored data is divided into their respective corresponding partition tables; The stored data in each target partition table is used as the data to be compressed; wherein the storage time of the stored data in the target partition table is later than the storage time of the stored data in the non-target partition table; the partition table includes the target partition table and the non-target partition table.

5. The data storage method based on a relational database according to claim 4, characterized in that, The step of using the stored data in each target partition table as the data to be compressed includes: The target partition table is determined by merging the partition tables; wherein the storage time of the data in the merged partition tables is earlier than the storage time of the data in the non-merged partition tables; the target partition table includes the merged partition tables and the non-merged partition tables. The target multi-row data in the merged partition table is converted into their respective single-row compressed data to obtain the converted merged partition table. The stored data in each of the non-merged partition tables and the converted merged partition tables are used as the data to be compressed.

6. A data storage device based on a relational database, characterized in that, include: The acquisition module is used to acquire the data to be compressed from the relational database. The compression module is used to compress the data to be compressed using a string table compression algorithm based on a preset initial dictionary and the suffix strings corresponding to each target tag, to obtain encoded data; wherein, the target tag is a tag added to the preset initial dictionary during the compression process, the suffix character includes the character following each target tag when it is added to the preset initial dictionary; the suffix string includes a preset number of characters following each target tag when it is added to the preset initial dictionary; A storage module is used to store the coded data into the relational database; The compression module includes: The dictionary update submodule is used to update the preset initial dictionary based on the suffix matching length corresponding to the current target tag matched by the current string; wherein, the suffix matching length is the number of characters in the current string that have the same position content as the suffix string corresponding to the current target tag after the current string.

7. A data storage device based on a relational database, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the data storage method based on a relational database as described in any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the data storage method based on a relational database as described in any one of claims 1 to 5.

Images