Storage method and decompression method of structured database and electronic device

By identifying data with hierarchical characteristics in a structured database and performing hierarchical compression, the problem of low compression rate of small blocks of data is solved, and more efficient storage performance is achieved.

CN114817181BActive Publication Date: 2025-04-04深圳市汉云科技有限公司
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202210404388.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-18
Publication Date
2025-04-04
Estimated Expiration
2042-04-18

AI Technical Summary

Technical Problem

In structured databases, due to the low compression rate of small blocks of data, the storage performance loss is large.

Method used

By identifying the data to be stored with hierarchical characteristics, the data layer to be compressed is determined based on the preset number of compressed layers, and the data on the data layer is compressed, and the compressed data is managed in conjunction with a dictionary table to avoid repeated compression.

Benefits of technology

Improves data compression rate, saves storage space, and reduces the loss of storage performance of structured databases.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114817181B_ABST
    Figure CN114817181B_ABST
Patent Text Reader

Abstract

The present application is applicable to the technical field of structured databases, and provides a storage method, a decompression method and an electronic device for a structured database, including: obtaining data to be stored; if the data to be stored is data with layered characteristics, determining the data layer to be compressed in the data to be stored according to a preset number of compression layers; compressing the data on the data layer to be compressed to obtain compressed data; storing the compressed data and the data on the data layer that does not need to be compressed in the data to be stored in the structured database. Through the above method, the compression rate of the data to be stored can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the technical field of structured databases, and in particular, relates to a storage method, a decompression method, an electronic device, and a computer-readable storage medium of a structured database. Background Art

[0002] Currently, in order to save disk space, data usually needs to be compressed before storage.

[0003] Most existing compression technologies are used to compress and decompress "large blocks of data", but the compression ratio for "small blocks of data" is not high, or even cannot be compressed. In structured databases, data is stored and queried by rows and columns (cells). Since the data in each row and column (cell) is usually small, the existing compression technology will not compress this data, which leads to a large loss of storage performance in structured databases. Summary of the invention

[0004] The embodiments of the present application provide a storage method, a decompression method and an electronic device for a structured database, which can solve the problem that the compression rate of the data to be stored is low, thereby causing a large loss in storage performance of the structured database.

[0005] In a first aspect, an embodiment of the present application provides a method for storing a structured database, comprising:

[0006] Get the data to be stored;

[0007] If the data to be stored is data with layered characteristics, determining the data layer to be compressed in the data to be stored according to a preset number of compression layers;

[0008] Compressing the data on the data layer to be compressed to obtain compressed data;

[0009] The compressed data and the data on the data layer that does not need to be compressed among the data to be stored are stored in a structured database.

[0010] In a second aspect, an embodiment of the present application provides a method for decompressing a structured database, comprising:

[0011] Acquire data to be decompressed, wherein the data to be decompressed includes uncompressed data and compressed data obtained by using the storage method of the structured database according to claim 1;

[0012] Extracting the compressed data from the data to be decompressed;

[0013] Decompressing the compressed data to obtain decompressed data corresponding to the compressed data;

[0014] Decompressed data corresponding to the data to be decompressed is determined according to the decompressed data corresponding to the compressed data and the uncompressed data.

[0015] In a third aspect, an embodiment of the present application provides a storage device for a structured database, including:

[0016] A module for acquiring data to be stored, used for acquiring data to be stored;

[0017] A data layer determination module to be compressed, used for determining the data layer to be compressed in the data to be stored according to a preset number of compression layers if the data to be stored is data with layered characteristics;

[0018] A compressed data determination module, used for compressing the data on the data layer to be compressed to obtain compressed data;

[0019] The data storage module is used to store the compressed data and the data on the data layer that does not need to be compressed in the data to be stored in a structured database.

[0020] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect or the second aspect when executing the computer program.

[0021] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect or the second aspect is implemented.

[0022] In a sixth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device executes the method described in the first aspect or the second aspect above.

[0023] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0024] In the embodiment of the present application, since the data to be stored is data with hierarchical characteristics, it usually has a large degree of duplication with other data with hierarchical characteristics. Therefore, the data layer to be compressed in the data to be stored is determined according to a preset number of compression layers, and the data on the data layer to be compressed is compressed. This can improve the compression rate of the data to be stored, thereby saving the disk occupied by the data stored in the structured database, and reducing the loss of storage performance of the structured database. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0026] Figure 1 It is a flowchart of a storage method of a structured database provided in one embodiment of the present application;

[0027] Figure 2 is a flowchart of another structured database storage method provided by an embodiment of the present application;

[0028] Figure 3 It is a schematic diagram of the architecture of a dictionary table recording compressed data and compressed data provided by an embodiment of the present application;

[0029] Figure 4 It is a flowchart of a method for decompressing a structured database provided in one embodiment of the present application;

[0030] Figure 5 is a flowchart of another method for decompressing a structured database provided in another embodiment of the present application;

[0031] Figure 6 It is a schematic diagram of the structure of a storage device for a structured database provided in one embodiment of the present application;

[0032] Figure 7 It is a structural schematic diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0033] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0034] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0035] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0036] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0037] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0038] Embodiment 1:

[0039] In a structured database, since directly storing pictures, large files, etc. will reduce the storage and query efficiency of the structured database, in a production environment, pictures, large files, etc. are generally stored in a dedicated file storage system (such as hdfs, ftp, http), and then the access address of the file (the access address is in the form of: http: / / 192.168.0.11:88 / biz / a.jpg, ftp: / / 192.168.0.100 / 2021 / 01 / 01 / 1.txt, ftp: / / 192.168.0.100 / 2021 / 01 / 01 / 2.txt, etc.) is stored in the structured database.

[0040] According to the access address format above, the Uniform Resource Locator (URL) is data with hierarchical characteristics. If there are many files in a subdirectory, the address data prefix will have a high degree of duplication (that is, the directory path equivalent to the absolute path of multiple files is repeated). For example, suppose an FTP server stores daily data files by "day", and there are 10,000 files per day, then the addresses of these 10,000 files may be stored in the database as:

[0041] ftp: / / 192.168.0.100 / 2021 / 01 / 01 / 1.txt

[0042] ftp: / / 192.168.0.100 / 2021 / 01 / 01 / 2.txt

[0043] ftp: / / 192.168.0.100 / 2021 / 01 / 01 / 3.txt

[0044] ftp: / / 192.168.0.100 / 2021 / 01 / 01 / 4.txt

[0045] ftp: / / ....

[0046] ftp: / / 192.168.0.100 / 2021 / 01 / 01 / 10000.txt

[0047] From this example, we can see that the prefix part of the 10,000 file addresses ftp: / / 192.168.0.100 / 2021 / 01 / 01 / is exactly the same, only the final file names 1.txt, 2.txt, and 3.txt are different.

[0048] For this scenario, we propose a storage method for a structured database, in which data with hierarchical characteristics are partially compressed, and the partially compressed data and the uncompressed data are stored together.

[0049] Since data with hierarchical characteristics usually has a high degree of repetition, using the above method for compression can improve the compression efficiency of the data, thereby saving disk space.

[0050] The following describes the storage method of the structured database provided in the embodiment of the present application in conjunction with the accompanying drawings.

[0051] Figure 1 A flowchart of a storage method of a structured database provided in an embodiment of the present application is shown, and the details are as follows:

[0052] Step S11, obtaining data to be stored.

[0053] In this embodiment, the data to be stored is data to be stored in a structured database.

[0054] Step S12: If the data to be stored is data with layered characteristics, then the data layer to be compressed in the data to be stored is determined according to a preset number of compression layers.

[0055] In this embodiment, when a data to be stored is distinguished by a certain character, the data to be stored is data with hierarchical characteristics. For example, assuming that the data to be stored is "ftp: / / 192.168.0.100 / 2021 / 01 / 01 / 1.txt", the character " / " is used to distinguish the data to be stored into four layers, the data on the first layer is "ftp: / / 192.168.0.100", the data on the second layer is "2021", the data on the third layer is "01", the data on the fourth layer is "01", and the last "1.txt" is the file name.

[0056] Of course, in addition to the hierarchical characteristics of URL addresses, standard house addresses such as ID card addresses and household registration addresses also have hierarchical characteristics. For example, if the house address is "Xili Street, Nanshan District, Shenzhen City, Guangdong Province", the house address can be divided as follows: the first layer is "Guangdong Province", the second layer is "Shenzhen City", the third layer is "Nanshan District", and the fourth layer is "Xili Street".

[0057] In this embodiment, the number of data layers to be compressed in the data to be stored is equal to the preset number of compression layers. For example, assuming that the preset number of compression layers is 3, the number of data layers to be compressed in the data to be stored is also 3.

[0058] In some embodiments, considering that for different data to be stored, the probability that the data on the front data layer is the same is greater, therefore, the first N data layers in the data to be stored can be determined as the data layers to be compressed in the data to be stored, where N is the preset number of compression layers. For example, assuming that N is 3, and the data to be stored is "ftp: / / 192.168.0.100 / 2021 / 01 / 01 / 1.txt", then "ftp: / / 192.168.0.100 / 2021 / 01" is determined as the data on the data layer to be compressed.

[0059] It should be noted that when the preset number of compression layers is greater than the maximum number of layers corresponding to the data to be stored, the maximum number of layers corresponding to the data to be stored is used as the data layer to be compressed in the data to be stored, or the user is prompted to change the preset number of compression layers.

[0060] Step S13, compressing the data on the data layer to be compressed to obtain compressed data.

[0061] Step S14, storing the compressed data and the data on the data layer that does not need to be compressed among the data to be stored in the structured database.

[0062] Specifically, the compressed data and the data on the data layer that does not need to be compressed are spliced ​​according to their positions in the data to be stored, and the spliced ​​data is stored in the structured database.

[0063] In the embodiment of the present application, since the data to be stored is data with hierarchical characteristics, it usually has a large degree of duplication with other data with hierarchical characteristics. Therefore, the data layer to be compressed in the data to be stored is determined according to a preset number of compression layers, and the data on the data layer to be compressed is compressed. This can improve the compression rate of the data to be stored, thereby saving the disk occupied by the data stored in the structured database, and reducing the loss of storage performance of the structured database.

[0064] In some embodiments, in order to effectively manage the compressed data, a dictionary table is used to record each compressed data and the compressed data corresponding to each compressed data. In this case, the above step S13 includes:

[0065] A1. Determine whether there is compressed data identical to the data on the data layer to be compressed in the compressed data recorded in the dictionary table, wherein the dictionary table is used to record each compressed data and to record compressed data corresponding to each compressed data.

[0066] In this embodiment, when the same data is compressed, it will be recorded in the dictionary table, and the dictionary table also records the compressed data corresponding to the compressed data. After the data layer to be compressed of the data to be stored is determined, the data on the data layer to be compressed is extracted, and the extracted data is compared with the compressed data recorded in the dictionary table. If there is compressed data identical to the extracted data in each compressed data recorded in the dictionary table, it indicates that the data on the data layer to be compressed has been compressed. Otherwise, it indicates that the data on the data layer to be compressed has not been compressed.

[0067] A2. If the compressed data recorded in the dictionary table contains compressed data identical to the data on the data layer to be compressed, then obtain information pointing to the compressed data in the dictionary table as the corresponding compressed data.

[0068] Since the compressed data is the information in the dictionary table pointing to the compressed data, the corresponding compressed data can be found based on the compressed data, so that the corresponding compressed data can be successfully found based on the compressed data, thereby realizing the decompression of the compressed data.

[0069] A3. If the dictionary table does not record compressed data identical to the data on the data layer to be compressed, then the data on the data layer to be compressed is recorded in the dictionary table as compressed data, and the information pointing to the data on the data layer to be compressed recorded in the dictionary table is obtained as the corresponding compressed data.

[0070] In the embodiment of the present application, since a dictionary table is used to record the compressed data and its corresponding compressed data, before the data on the data layer to be compressed needs to be compressed, it is possible to select whether to compress the data on the data layer to be compressed based on the compressed data recorded in the dictionary table, thereby avoiding repeated compression of the data on the data layer to be compressed.

[0071] In some embodiments, the dictionary table includes at least one first array and a second array of elements, the elements of the first array are pointers to the second array, and the elements of the second array are recorded compressed data, and the step A1 includes:

[0072] A11. Calculate the hash value corresponding to the data on the data layer to be compressed.

[0073] In this embodiment, a hash function may be used to calculate the hash value corresponding to the data on the data layer to be compressed.

[0074] A12. A value determined according to the hash value and the first preset value is used as a target value.

[0075] The first preset value may be a value set when the table is created. For example, the first preset value may be set to 1024.

[0076] In this embodiment, the hash value and the first preset value can be modulo operation, and the remainder obtained can be used as the target value. For example, assuming that the data on the data layer to be compressed is "ftp: / / 192.168.0.100 / 2021 / 01", the target value can be determined in the following way: Hash(ftp: / / 192.168.0.100 / 2021 / 01)%1024. Among them, "Hash" represents the hash function, "%" represents the modulo operation, and "1024" is the above-mentioned first preset value.

[0077] A13. Determine whether there is an element at the position in the first array with the same position number as the target value.

[0078] Since the data on the data layer to be compressed are the same, their corresponding hash values ​​must be the same, and the target value determined according to the hash value and the first preset value must also be the same, and the elements in the first array are pointers to the second array, and the elements of the second array are the recorded compressed data, therefore, if there is no element at the position in the first array with the same position number as the target value, it means that the second array does not record the same data as the data on the data layer to be compressed, that is, it means that the data on the data layer to be compressed has not been compressed. That is, through the above method, it is possible to quickly determine whether the data on the data layer to be compressed has been compressed.

[0079] Correspondingly, the above step A3 includes:

[0080] A31. If there is no element at the position with the same position number as the target value in the first array, a new second array is created, and the data on the data layer to be compressed is used as the first element in the new second array, and the element at the position with the same position number as the target value in the first array is pointed to the new second array, and then the compressed data corresponding to the data on the data layer to be compressed is determined according to the target value, the first preset value and the position number of the first element in the new second array.

[0081] For example, assuming that the target value is "1022" and the data on the data layer to be compressed is "ftp: / / 192.168.0.100 / 2021 / 01". If it is determined that the position "1022" in the first array is empty (NULL), that is, there is no information, it means that the data identical to the data on the data layer to be compressed has not been compressed before. At this time, a new second array will be created, and "ftp: / / 192.168.0.100 / 2021 / 01" will be recorded in the first position of the second array. Assuming that the position number of the elements of the second array starts from 0, the compressed data can be determined according to the following formula: N*HASH_ARRAY+H, where "N" is the position number of the element in the second array that is the data on the data layer to be compressed. In this example, "N" is "0", "HASH_ARRAY" is the first preset value (assuming it is 1024), and "H" is the target value. In this example, "H" is "1022", and the compressed data obtained is "1022".

[0082] It should be pointed out that other formulas can also be used to calculate the compressed data, as long as they are related to the target value, the first preset value and the position sequence number of the elements of the data on the data layer to be compressed in the second array, and are not limited here.

[0083] A32. If there is an element at the position with the same position number as the target value in the first array, determine the position number corresponding to the data on the data layer to be compressed in the second array pointed to by the element, and determine the compressed data corresponding to the data on the data layer to be compressed based on the target value, the first preset value and the position number determined in the second array.

[0084] In this embodiment, if it is determined that there is an element at the position in the first array whose position number is the same as the above-mentioned target value, that is, there is a pointer to the second array at the same position as the target value, it indicates that the data identical to the data on the data layer to be compressed may have been compressed. At this time, the compressed data corresponding to the data on the data layer to be compressed can be directly determined according to the target value, the first preset value and the position number determined in the second array.

[0085] In some embodiments, if there is an element at a position in the first array whose position number is the same as the target value in step A32, determining the position number corresponding to the data on the data layer to be compressed in the second array pointed to by the element includes:

[0086] A321. If there is an element at the position in the first array whose position number is the same as the target value, the data on the data layer to be compressed are compared one by one with the elements in the second array pointed to by the elements in the first array.

[0087] A322. If the data on the data layer to be compressed is the same as the target element, the position number of the target element is used as the position number corresponding to the data on the data layer to be compressed, wherein the target element is an element in the second array.

[0088] A323. If the data on the data layer to be compressed is different from the elements in the second array, the data on the data layer to be compressed is used as the latest element in the second array, and the position number of the latest element is the position number corresponding to the data on the data layer to be compressed.

[0089] In the embodiment of the present application, if the second array has 3 elements, the data on the data layer to be compressed can be compared with the first element in the second array in order from front to back. If the two are different, the data on the data layer to be compressed can be compared with the second element in the second array. If they are the same, it is determined that the position number corresponding to the second element in the second array is the position number corresponding to the data on the data layer to be compressed. Of course, if the data on the data layer to be compressed is different from the 3 elements of the second array, the data on the data layer to be compressed is recorded after the 3 elements of the second array. At this time, the data on the data layer to be compressed is used as the 4th element of the second array.

[0090] In some embodiments, the preset number of compression layers may be a value specified by the user, or a value automatically determined according to the data to be stored.

[0091] In an embodiment of the present application, when the preset number of compression layers is a value automatically determined according to the data to be stored, it is determined according to the type of the data to be stored and / or according to the number of layers included in the data to be stored itself.

[0092] When determining the number of compression layers according to the type of the data to be stored, the corresponding number of compression layers is set in advance for different types of data to be stored. In this way, after the data to be stored with hierarchical characteristics is further classified and the type to which it belongs is obtained, the number of compression layers corresponding to the data to be stored can be automatically determined. For example, assuming that the data to be stored with hierarchical characteristics is "ftp: / / 192.168.0.100 / 2021 / 01 / 01 / 1.txt" and "Xili Street, Nanshan District, Shenzhen City, Guangdong Province", the first data to be stored is classified as a URL address class, and the second data to be stored is classified as a house address class. Assuming that the number of compression layers pre-set for the URL address class is 3, and the number of compression layers pre-set for the house address class is 4, it is determined that the number of compression layers corresponding to the above "ftp: / / 192.168.0.100 / 2021 / 01 / 01 / 1.txt" is 3, and the number of compression layers corresponding to "Xili Street, Nanshan District, Shenzhen City, Guangdong Province" is 4.

[0093] When determining the number of compression layers according to the type of the data to be stored and the number of layers included in the data to be stored, first determine the type of the data to be stored with classification characteristics. If it is a preset first type, then determine the number of compression layers corresponding to the data to be stored with classification characteristics according to the number of compression layers corresponding to the first type. If it is a preset second type, then determine the corresponding number of compression layers according to the number of layers included in the data to be stored.

[0094] For example, assuming that the first type is a house address type, the second type is a URL address type, and the number of compression layers corresponding to the house address type is 4, then when it is determined that the data to be stored is a house address type, the number of compression layers corresponding to the data to be stored is directly determined to be 4. When it is determined that the data to be stored is a URL address type, the default number of compression layers is the total number of layers corresponding to all layers of the URL address except the last file name. Since the number of layers of some types of data to be stored is usually fixed, for example, for the data to be stored of the house address type, it is usually composed of fixed layers such as provinces, cities, districts, and streets, while the number of layers of some types of data to be stored is usually not fixed, for example, for the data to be stored of the URL address type, it is related to the stored address. Therefore, determining the number of compression layers according to the type of the data to be stored and the number of layers included in the data to be stored itself can greatly improve the accuracy of the determined number of compression layers, thereby greatly improving the compression efficiency of the subsequent data compression.

[0095] In some embodiments, if the number of compression layers is a value automatically determined according to the data to be stored, the storage method of the structured database further includes:

[0096] B1. Count the average number of data to be stored corresponding to each compressed data after the data to be stored is compressed with the above compression layers.

[0097] B2. If the above number is less than a preset number threshold, reduce the above number of compression layers, and use the reduced number of compression layers as the new number of compression layers.

[0098] In order to more clearly describe the process of how to update the number of compression layers, a specific example is used below. Assuming that the data to be stored is a URL address type, and the user has not set a fixed number of compression layers, the program's initial default number of compression layers is all the layers after the URL address except the last file name. During the program's running, the average number of data to be stored compressed under each URL prefix at this number of layers is dynamically counted. If this number is less than the preset number threshold (assuming it is 100), the number of compression layers is decreased.

[0099] Take the data to be stored in the form of ftp: / / 192.168.0.100 / 2021 / 12 / 01 / 1.txt and ftp: / / 192.168.0.100 / 2022 / 02 / 01 / 1.txt as examples:

[0100] a. Initially, the preset number of compression layers is the number of layers corresponding to all layers except the 1.txt file name, that is, 4 layers (if the data to be stored is "ftp: / / 192.168.0.100 / 2021 / 12 / 01 / 1.txt", the data on the data layer to be compressed determined according to the preset number of compression layers is "ftp: / / 192.168.0.100 / ", "2021 / ", "12 / ", "01 / ").

[0101] b. If during the program running, the dynamic statistical sampling evaluation finds that the number of data compressed using the prefix "ftp: / / 192.168.0.100 / 2021 / 12 / 01 / " is 90 (the actual meaning is: there are 90 files in this directory), and the number of data compressed using the prefix "ftp: / / 192.168.0.100 / 2022 / 02 / 01 / " is 96 (the actual meaning is: there are 96 files in this directory), then (90+96) / 2=93, and 93 is less than 100, therefore, the preset number of compression layers is reduced from 4 to 3, that is, the data layers to be compressed become all layers except the file name and the previous layer of the file name (if the data to be stored is "ftp: / / 192.168.0.100 / 2021 / 12 / 01 / 1.txt", the data on the data layer to be compressed determined according to the new number of compression layers is "ftp: / / 192.168.0.100 / ", "2021 / ", "12 / ").

[0102] Of course, if after the data to be stored is compressed with the number of compression layers, the average number of data to be stored corresponding to each compressed data is not less than the preset number threshold, the number of compression layers remains unchanged.

[0103] In B1 to B2 above, since the number of compression layers is dynamically adjusted according to the number of data to be stored corresponding to each compressed data, a balance can be achieved between the dictionary table size and the compression rate. This is because when there are too many data layers to be compressed, the dictionary table will become larger, but the corresponding compression rate will become higher; and when there are too few data layers to be compressed, the dictionary table will become smaller, but at the same time the compression rate will also become lower. A large dictionary table requires more memory and storage; a high compression rate can save the storage of real data.

[0104] In order to more clearly describe the storage method of the structured database provided in the embodiment of the present application, the following description is made taking the data to be stored as a URL address as an example.

[0105] First, when creating a table, specify that a field stores URL address data, so that the database can use the corresponding compression and decompression methods to handle the storage and query of the data. The syntax is as follows:

[0106] CREATE TABLE test_table (

[0108] XH INT PRIMARY KEY COMMENT'serial number', ...

[0109] DZ VARCHAR(256)USING URLCODE LEVEL 3HASH_ARRAY 1024COMMENT'File address', ... )

[0111] In the above table creation statement, the DZ field contains the "USRING URLCODE" clause, indicating that the field is used to store URL address data. Since URL address data is hierarchical data, the database will use corresponding compression and decompression methods to handle the data storage and query of this field.

[0112] The above DZ field also contains a "LEVEL 3" clause to prompt the database to extract only the first three layers of the URL address for compression encoding when compressing the URL address data in this field, that is, the preset number of compression layers is 3. The number of compression encoding layers of the address is limited here mainly to prevent the number of layers from being too deep, resulting in the final dictionary table being too large, affecting system resource usage and performance.

[0113] The above DZ field also contains a "HASH_ARRAY 1024" clause to indicate that the first preset value is 1024, and to indicate that when the database creates the corresponding dictionary table, the appropriate size of its "Hash array (i.e., the first array)" is 1024. If the position numbers corresponding to each element in the Hash array start from 0, the position numbers corresponding to each element in the Hash array are from 0 to 1023.

[0114] After the table is created, the database will compress and store the URL address stored in the "USRING URLCODE" clause. For the specific process, see Figure 2 .

[0115] Assume that the data to be stored are ftp: / / 192.168.0.100 / 2021 / 01 / 01 / 1.txt, ftp: / / 192.168.0.100 / 2022 / 02 / 01 / 1.txt, and ftp: / / 192.168.0.100 / 2022 / 02 / 02 / 1.txt.

[0116] (I) Reference Figure 2 , insert ftp: / / 192.168.0.100 / 2021 / 01 / 01 / 1.txt:

[0117] 1) Extract the specified layer 3 data, that is, P is ftp: / / 192.168.0.100 / 2021 / 01. Corresponding Figure 2 Follow step 1 in the

[0118] 2) Calculate Hash(ftp: / / 192.168.0.100 / 2021 / 01)%1024. Assuming the target value is 1022, this step corresponds to Figure 2 Step 2 in . There are many choices of Hash functions, which will not be described here.

[0119] 3) If the 1022nd position of the Hash array (i.e. the first array mentioned above) is NULL, then create a new element array (i.e. the second array mentioned above) and insert the first data ftp: / / 192.168.0.100 / 2021 / 01. At the same time, let the element at position 1022 of the Hash array point to the new array. This step corresponds to Figure 2 Follow steps 3 and 4 in .

[0120] 4) 1022 is the compressed data corresponding to ftp: / / 192.168.0.100 / 2021 / 01. This step corresponds to Figure 2 Follow step 5 in the

[0121] 5) 1022 + tail data / 01 / 1.txt, that is, "1022 / 01 / 1.txt" is the last data written into the target database. Figure 2 Follow step 6 in the

[0122] (II) Reference Figure 2 , continue to insert ftp: / / 192.168.0.100 / 2022 / 02 / 01 / 1.txt, the process is as follows:

[0123] 1) Extract the specified 3-layer data, that is, P is ftp: / / 192.168.0.100 / 2022 / 02, corresponding to Figure 2 Follow step 1 in the

[0124] 2) Calculate Hash (ftp: / / 192.168.0.100 / 2022 / 02) % 1024. Assuming the target value is 1022, the corresponding Figure 2 Follow step 2 in the

[0125] 3) If the 1022th position of the current hash array points to an element array, ftp: / / 192.168.0.100 / 2022 / 02 is compared with the elements in the element array one by one. No identical elements are found, so new data ftp: / / 192.168.0.100 / 2022 / 02 is inserted at the end of the element array, and the position number of the new data in the element array is 1. Figure 2 Follow steps 3 / 7 / 8 / 9 / 10.

[0126] 4) 1*1024+1022=2046 is the compressed data of ftp: / / 192.168.0.100 / 2022 / 02, corresponding to Figure 2 Follow step 11 in the guide.

[0127] 5) 2046 + tail data / 01 / 1.txt, that is, 2046 / 01 / 1.txt is the last data written to the target database, corresponding to Figure 2 Follow step 6 in the

[0128] (III) Reference Figure 2 , continue to insert ftp: / / 192.168.0.100 / 2022 / 02 / 02 / 1.txt, the process is as follows:

[0129] 1) Extract the specified 3-layer data, that is, P is ftp: / / 192.168.0.100 / 2022 / 02, corresponding to Figure 2 Follow step 1 in .

[0130] 2) Calculate Hash (ftp: / / 192.168.0.100 / 2022 / 02) % 1024. As above, assume that the target value is 1022, corresponding to Figure 2 Follow step 2 in the

[0131] 3) If the 1022th position of the current hash array points to an element array, then ftp: / / 192.168.0.100 / 2022 / 02 is compared with the elements in the element array one by one, and it is found that the same element already exists, and its position number is 1, corresponding to Figure 2 Follow steps 3 / 7 / 8 / 10.

[0132] 4) 1*1024+1022=2046 is the compressed data of ftp: / / 192.168.0.100 / 2022 / 02, corresponding to Figure 2 Follow step 11 in the guide.

[0133] 5) 2046 + tail data / 01 / 1.txt, that is, 2046 / 02 / 1.txt is the last data written to the target database, corresponding to Figure 2 Follow step 6 in the

[0134] The final dictionary table is as follows Figure 3 shown.

[0135] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0136] Embodiment 2:

[0137] Figure 4 A flowchart of a method for decompressing a structured database provided in an embodiment of the present application is shown, and is described in detail as follows:

[0138] Step S41, obtaining data to be decompressed, wherein the data to be decompressed includes uncompressed data and compressed data obtained by using the storage method of the structured database in the first embodiment.

[0139] In this embodiment, the data to be decompressed is obtained from a structured database. For example, if the data under the "USRINGURLCODE" field is specified as partially compressed data when creating a table, the data extracted from the field is the data to be decompressed in this embodiment.

[0140] In this embodiment, the splicing relationship between the compressed data and the uncompressed data in the data to be decompressed is the same as the splicing relationship between the compressed data and the data on the data layer that does not need to be compressed in the first embodiment.

[0141] Step S42: extracting the compressed data from the data to be decompressed.

[0142] In this embodiment, the corresponding extraction action is performed according to the bytes occupied by the compressed data obtained in Embodiment 1 and the position thereof. For example, assuming that the bytes occupied by the compressed data obtained in Embodiment 1 are 4 bytes and are located in the first 4 bytes of the data to be decompressed, the first 4 bytes of the data to be decompressed are extracted as the compressed data.

[0143] Step S43, decompressing the compressed data to obtain decompressed data corresponding to the compressed data.

[0144] In this embodiment, the decompression process corresponds to the compression process in the first embodiment.

[0145] Step S44: Determine the decompressed data corresponding to the data to be decompressed according to the decompressed data corresponding to the compressed data and the uncompressed data.

[0146] In the embodiment of the present application, since only part of the data to be decompressed needs to be decompressed, after decompressing the part of the data, the obtained decompressed data is spliced ​​with the uncompressed data in the data to be decompressed to obtain the final decompressed data.

[0147] In some embodiments, step S43 includes:

[0148] The position serial numbers of the elements of the first array in the dictionary table and the position serial numbers of the elements of the second array in the dictionary table are determined according to the compressed data and the first preset value.

[0149] According to the determined position serial numbers of the elements of the first array and the position serial numbers of the elements of the second array, corresponding data are read out from the dictionary table as decompressed data corresponding to the compressed data.

[0150] refer to Figure 5 The decompression flow chart below uses "2046 / 02 / 1.txt" to illustrate the process of obtaining the decompressed data corresponding to the data to be decompressed:

[0151] 1) Read the first 4 bytes and calculate the corresponding compression code. The compression code calculated here is 2046. Corresponding to step 1 of the flowchart.

[0152] 2) Calculate 2046%1024=1022. Corresponding to step 2 of the flowchart.

[0153] 3) Calculate 2046 / 1024=1. “ / ” means taking the quotient.

[0154] 4) According to the above steps 2) and 3), the decompressed data is the data recorded at position 1 of the element array at position 1022 of the Hash array, that is, ftp: / / 192.168.0.100 / 2022 / 02.

[0155] 5) Add the data found in step 4) to the data after the first 4 bytes of 2046 / 02 / 1.txt (i.e. / 02 / 1.txt), and the resulting ftp: / / 192.168.0.100 / 2022 / 02 / 02 / 1.txt is the original actual data after decompression.

[0156] Embodiment three:

[0157] Corresponding to the storage method of the structured database in the first embodiment above, Figure 6 A structural block diagram of a storage device for a structured database provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0158] Reference Figure 6 The storage device 6 of the structured database includes: a data acquisition module 61 to be stored, a data layer determination module 62 to be compressed, a compressed data determination module 63, and a data storage module 64. Among them:

[0159] The to-be-stored data acquisition module 61 is used to acquire the to-be-stored data.

[0160] The data layer determination module 62 to be compressed is used to determine the data layer to be compressed in the data to be stored according to a preset number of compression layers if the data to be stored is data with layered characteristics.

[0161] The compressed data determination module 63 is used to compress the data on the data layer to be compressed to obtain compressed data.

[0162] The data storage module 64 is used to store the compressed data and the data on the data layer that does not need to be compressed in the data to be stored into a structured database.

[0163] In the embodiment of the present application, since the data to be stored is data with hierarchical characteristics, it usually has a large degree of duplication with other data with hierarchical characteristics. Therefore, the data layer to be compressed in the data to be stored is determined according to a preset number of compression layers, and the data on the data layer to be compressed is compressed. This can improve the compression rate of the data to be stored, thereby saving the disk occupied by the data stored in the structured database.

[0164] In some embodiments, the compressed data determination module 63 includes:

[0165] The data storage judgment unit is used to judge whether there is compressed data identical to the data on the data layer to be compressed in the compressed data recorded in the dictionary table, wherein the dictionary table is used to record each compressed data and record the compressed data corresponding to each compressed data.

[0166] The data storage processing unit is used to obtain the information pointing to the compressed data in the dictionary table as the corresponding compressed data if there is compressed data identical to the data on the data layer to be compressed in the compressed data recorded in the dictionary table.

[0167] The data unstored processing unit is used to record the data on the data layer to be compressed as compressed data in the dictionary table if the dictionary table does not record the compressed data identical to the data on the data layer to be compressed, and obtain the information in the dictionary table pointing to the data on the data layer to be compressed as the corresponding compressed data.

[0168] In some embodiments, the dictionary table includes a first array and a second array of at least one element, the element of the first array is a pointer to the second array, the element of the second array is the recorded compressed data, and the data storage determination unit includes:

[0169] The hash value calculation unit is used to calculate the hash value corresponding to the data on the data layer to be compressed.

[0170] The target value determination unit is used to determine a value according to the hash value and the first preset value as the target value.

[0171] The element existence judgment unit is used to judge whether there is an element at a position in the first array with the same position number as the target value.

[0172] Correspondingly, the data storage processing unit includes:

[0173] An array creation unit is used to create a new second array if there is no element at the position with the same position number as the target value in the first array, and use the data on the data layer to be compressed as the first element in the newly created second array, and make the element at the position with the same position number as the target value in the first array point to the newly created second array, and then determine the compressed data corresponding to the data on the data layer to be compressed according to the target value, the first preset value and the position number of the first element in the newly created second array.

[0174] The position determination unit is used to determine the position number corresponding to the data on the data layer to be compressed in the second array pointed to by the above element if there is an element at the position whose position number is the same as the above target value in the above first array, and determine the compressed data corresponding to the data on the data layer to be compressed according to the above target value, the above first preset value and the position number determined in the above second array.

[0175] In some embodiments, when the position determination unit determines that there is an element at the position with the same position number as the target value in the first array, determining the position number corresponding to the data on the data layer to be compressed in the second array pointed to by the element specifically includes:

[0176] If there is an element at the position with the same position number as the target value in the first array, the data on the data layer to be compressed will be compared one by one with the elements in the second array pointed to by the elements in the first array; if the data on the data layer to be compressed is the same as the target element, the position number of the target element will be used as the position number corresponding to the data on the data layer to be compressed, wherein the target element is an element in the second array; if the data on the data layer to be compressed is different from all the elements in the second array, the data on the data layer to be compressed will be used as the latest element in the second array, and the position number of the latest element will be the position number corresponding to the data on the data layer to be compressed.

[0177] In some embodiments, the number of compression layers is a value specified by the user, or a value automatically determined based on the data to be stored.

[0178] In some embodiments, if the number of compression layers is a value automatically determined according to the data to be stored, the storage device 6 of the structured database further includes:

[0179] The quantity statistics module is used to count the average quantity of the to-be-stored data corresponding to each compressed data after the to-be-stored data is compressed with the above-mentioned number of compression layers.

[0180] The compression layer number updating module is used to reduce the above-mentioned compression layer number if the above-mentioned average value is less than a preset number threshold, and use the reduced compression layer number as the new compression layer number.

[0181] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0182] Embodiment 4:

[0183] Figure 7 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 7 As shown, the electronic device 7 of this embodiment includes: at least one processor 70 ( Figure 7 Only one processor is shown in the figure), a memory 71, and a computer program 72 stored in the memory 71 and executable on the at least one processor 70, wherein the processor 70 implements the steps of any of the above-mentioned method embodiments when executing the computer program 72.

[0184] The electronic device 7 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The electronic device may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will appreciate that Figure 7 It is only an example of the electronic device 7 and does not constitute a limitation on the electronic device 7. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0185] The processor 70 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0186] In some embodiments, the memory 71 may be an internal storage unit of the electronic device 7, such as a hard disk or memory of the electronic device 7. In other embodiments, the memory 71 may also be an external storage device of the electronic device 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 7. Further, the memory 71 may also include both an internal storage unit and an external storage device of the electronic device 7. The memory 71 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 71 may also be used to temporarily store data that has been output or is to be output.

[0187] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0188] An embodiment of the present application also provides a network device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps in any of the above-mentioned method embodiments when executing the computer program.

[0189] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0190] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0191] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / electronic device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0192] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0193] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0194] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0195] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0196] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A storage method for a structured database, characterized in that: Applied to the storage of URL data, the method comprises: Acquire data to be stored, where the data to be stored is URL data; If the data to be stored is data with layered characteristics, determining the data layer to be compressed in the data to be stored according to a preset number of compression layers; Compressing the data on the data layer to be compressed to obtain compressed data; Storing the compressed data and the data on the data layer that does not need to be compressed among the data to be stored in a structured database; The step of compressing the data on the data layer to be compressed to obtain compressed data includes: Determine whether there is compressed data identical to the data on the data layer to be compressed in the compressed data recorded in the dictionary table, wherein the dictionary table is used to record each compressed data and compressed data corresponding to each compressed data; If there is compressed data identical to the data on the data layer to be compressed in the compressed data recorded in the dictionary table, obtaining information pointing to the compressed data in the dictionary table as the corresponding compressed data; If the dictionary table does not record the compressed data identical to the data on the data layer to be compressed, then the data on the data layer to be compressed is recorded in the dictionary table as the compressed data, and the information pointing to the data on the data layer to be compressed recorded in the dictionary table is obtained as the corresponding compressed data; The dictionary table includes a first array and a second array of at least one element, the elements of the first array are pointers to the second array, and the elements of the second array are recorded compressed data, and the determining whether there is compressed data identical to the data on the data layer to be compressed in the compressed data recorded in the dictionary table includes: Calculate the hash value corresponding to the data on the data layer to be compressed; A value determined according to the hash value and the first preset value is used as a target value; Determine whether there is an element at a position in the first array with the same position number as the target value; Correspondingly, if the dictionary table does not record the compressed data identical to the data on the data layer to be compressed, then the data on the data layer to be compressed is recorded in the dictionary table as the compressed data, and the information pointing to the compressed data in the dictionary table is obtained as the corresponding compressed data, including: If there is no element at the position with the same position number as the target value in the first array, a new second array is created, and the data on the data layer to be compressed is used as the first element in the newly created second array, and the element at the position with the same position number as the target value in the first array is made to point to the newly created second array, and then the compressed data corresponding to the data on the data layer to be compressed is determined according to the target value, the first preset value and the position number of the first element in the newly created second array; If there is an element at the position with the same position number as the target value in the first array, the position number corresponding to the data on the data layer to be compressed is determined in the second array pointed to by the element, and the compressed data corresponding to the data on the data layer to be compressed is determined based on the target value, the first preset value and the position number determined in the second array.

2. The storage method of a structured database according to claim 1, characterized in that: If there is an element at a position in the first array whose position number is the same as the target value, determining the position number corresponding to the data on the data layer to be compressed in the second array pointed to by the element, including: If there is an element at the position in the first array whose position number is the same as the target value, then the data on the data layer to be compressed are compared one by one with the elements in the second array pointed to by the elements in the first array; If the data on the data layer to be compressed is the same as the target element, the position number of the target element is used as the position number corresponding to the data on the data layer to be compressed, wherein the target element is an element in the second array; If the data on the data layer to be compressed is different from all the elements in the second array, the data on the data layer to be compressed is taken as the latest element in the second array, and the position number of the latest element is the position number corresponding to the data on the data layer to be compressed.

3. The method for storing a structured database according to any one of claims 1 to 2, characterized in that: The number of compression layers is a value specified by the user, or a value automatically determined according to the data to be stored.

4. The storage method of a structured database according to claim 3, characterized in that: If the number of compression layers is a value automatically determined according to the data to be stored, the storage method of the structured database further includes: Counting the average number of data to be stored corresponding to each compressed data after the data to be stored is compressed with the number of compression layers; If the average value is less than a preset number threshold, the number of compression layers is reduced, and the reduced number of compression layers is used as the new number of compression layers.

5. A method for decompressing a structured database, characterized in that: include: Acquire data to be decompressed, wherein the data to be decompressed includes uncompressed data and compressed data obtained by using the storage method of the structured database according to claim 1; Extracting the compressed data from the data to be decompressed; Decompressing the compressed data to obtain decompressed data corresponding to the compressed data; Decompressed data corresponding to the data to be decompressed is determined according to the decompressed data corresponding to the compressed data and the uncompressed data.

6. A storage device for a structured database, characterized in that: Applied to the storage of URL data, the device comprises: A module for acquiring data to be stored, used for acquiring data to be stored, wherein the data to be stored is URL data; A data layer determination module to be compressed, used for determining the data layer to be compressed in the data to be stored according to a preset number of compression layers if the data to be stored is data with layered characteristics; A compressed data determination module, used for compressing the data on the data layer to be compressed to obtain compressed data; A data storage module, used for storing the compressed data and the data on the data layer that does not need to be compressed in the data to be stored in a structured database; The step of compressing the data on the data layer to be compressed to obtain compressed data includes: Determine whether there is compressed data identical to the data on the data layer to be compressed in the compressed data recorded in the dictionary table, wherein the dictionary table is used to record each compressed data and compressed data corresponding to each compressed data; If there is compressed data identical to the data on the data layer to be compressed in the compressed data recorded in the dictionary table, obtaining information pointing to the compressed data in the dictionary table as the corresponding compressed data; If the dictionary table does not record the compressed data identical to the data on the data layer to be compressed, then the data on the data layer to be compressed is recorded in the dictionary table as the compressed data, and the information pointing to the data on the data layer to be compressed recorded in the dictionary table is obtained as the corresponding compressed data; The dictionary table includes a first array and a second array of at least one element, the elements of the first array are pointers to the second array, and the elements of the second array are recorded compressed data, and the determining whether there is compressed data identical to the data on the data layer to be compressed in the compressed data recorded in the dictionary table includes: Calculate the hash value corresponding to the data on the data layer to be compressed; A value determined according to the hash value and the first preset value is used as a target value; Determine whether there is an element at a position in the first array with the same position number as the target value; Correspondingly, if the dictionary table does not record the compressed data identical to the data on the data layer to be compressed, then the data on the data layer to be compressed is recorded in the dictionary table as the compressed data, and the information pointing to the compressed data in the dictionary table is obtained as the corresponding compressed data, including: If there is no element at the position with the same position number as the target value in the first array, a new second array is created, and the data on the data layer to be compressed is used as the first element in the newly created second array, and the element at the position with the same position number as the target value in the first array is made to point to the newly created second array, and then the compressed data corresponding to the data on the data layer to be compressed is determined according to the target value, the first preset value and the position number of the first element in the newly created second array; If there is an element at the position with the same position number as the target value in the first array, the position number corresponding to the data on the data layer to be compressed is determined in the second array pointed to by the element, and the compressed data corresponding to the data on the data layer to be compressed is determined based on the target value, the first preset value and the position number determined in the second array.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Citation Information

Patent Citations

  • Data processing method and device and electronic equipment

    CN112668033A