Data storage method, data reading method, device, equipment and storage medium

Abstract

The present application relates to a data storage method, a data reading method, an apparatus, a computer device, a computer-readable storage medium and a computer program product. The data storage method comprises: obtaining data to be stored, and compressing the data to be stored to obtain a compressed data block; determining free compression header information from the target compression header information corresponding to the disk, the free compression header information is used to indicate the free storage space in the compressed data block corresponding to the target compression header information; according to the free compression header information, the compressed data block is stored in the free storage space. The method is used to improve the utilization rate of disk space when storing compressed data.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data storage method, a data reading method, an apparatus, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Typically, in a storage system or file system, a disk is divided into multiple sectors of a fixed size (such as 512 bytes), and multiple sectors are organized into disk blocks. For example, 16 512-byte sectors can form an 8K disk block.

[0003] When storing data, the data can be compressed first to reduce redundant data and optimize the encoding method, thereby reducing the storage space occupied by the data. However, when storing compressed data, traditional technologies have the problem of low disk space utilization. Summary of the invention

[0004] Based on this, it is necessary to provide a data storage method, data reading method, device, computer equipment, computer-readable storage medium and computer program product that can improve disk space utilization when storing compressed data in response to the above technical problems.

[0005] In a first aspect, the present application provides a data storage method. The method comprises:

[0006] Acquire data to be stored, and compress the data to be stored to obtain a compressed data block;

[0007] Determine free compression header information from the target compression header information corresponding to the disk, wherein the free compression header information is used to indicate free storage space in the compression data block corresponding to the target compression header information;

[0008] According to the free compression header information, the compressed data block is stored in the free storage space.

[0009] In one embodiment, the target compression header information includes a plurality of compression header information, and storing the compressed data block in the free storage space according to the free compression header information includes:

[0010] Detecting whether there is non-idle compressed header information in each of the compressed header information, and obtaining a detection result;

[0011] According to the detection result and the free compression header information, the compressed data block is stored in the free storage space.

[0012] In one embodiment, storing the compressed data block in the free storage space according to the detection result and the free compression header information includes:

[0013] When the detection result is that the non-idle compressed header information exists in the target compressed header information, obtaining an offset address and a compressed data length from the non-idle compressed header information adjacent to the idle compressed header information;

[0014] Determine a target offset address corresponding to the free storage space according to the offset address and the compressed data length;

[0015] The compressed data block is stored in the free storage space according to the target offset address.

[0016] In one embodiment, storing the compressed data block in the free storage space according to the detection result and the free compression header information includes:

[0017] When the detection result is that the non-free compressed header information does not exist in the target compressed header information, the first storage space in the compressed data block is determined as the free storage space, and the compressed data block is stored in the free storage space.

[0018] In one of the embodiments, before storing the compressed data block in the free storage space according to the target offset address, the method further includes:

[0019] According to the compression header magic number included in the target compression header information, the target compression header information is legally verified to obtain a verification result;

[0020] The step of storing the compressed data block in the free storage space according to the target offset address comprises:

[0021] When the verification result shows that the target compression header information is legal, the compressed data block is stored in the free storage space according to the target offset address.

[0022] In a second aspect, the present application also provides a data reading method. The method comprises:

[0023] Receiving a data reading request, wherein the data reading request includes a file identifier corresponding to the data to be read;

[0024] Determine, according to the file identifier, target compression header information corresponding to the file identifier, wherein the compressed data stored in the compression data block corresponding to the target compression header information is stored by the method described in any one of the first aspects above;

[0025] The to-be-read data is read from the compressed data block according to the target compressed header information.

[0026] In one embodiment, reading the to-be-read data from the compressed data block according to the target compressed header information includes:

[0027] According to the file identifier, determining the target compression header information corresponding to the file identifier from the target compression header information;

[0028] The data to be read is obtained according to the target offset address included in the target compression header information.

[0029] In one embodiment, acquiring the to-be-read data according to the target offset address included in the target compression header information includes:

[0030] Reading a compressed data block from the compressed data block according to the target offset address, and reading a compressed data length and a first check value of the compressed data block from the target compression header information;

[0031] Calculate a second check value according to the compressed data length;

[0032] If the first check value is the same as the second check value, the compressed data block is decompressed to obtain the data to be read.

[0033] In a third aspect, the present application further provides a data storage device. The device comprises:

[0034] A compression module, used to obtain data to be stored, and compress the data to be stored to obtain a compressed data block;

[0035] A first determining module, configured to determine free compression header information from target compression header information corresponding to the disk, wherein the free compression header information is used to indicate free storage space in a compression data block corresponding to the target compression header information;

[0036] A storage module is used to store the compressed data block in the free storage space according to the free compression header information.

[0037] In a fourth aspect, the present application further provides a data reading device. The device comprises:

[0038] A receiving module, used for receiving a data reading request, wherein the data reading request includes a file identifier corresponding to the data to be read;

[0039] a second determining module, configured to determine, according to the file identifier, target compression header information corresponding to the file identifier, wherein the compressed data stored in the compression data block corresponding to the target compression header information is stored according to the method described in any one of the first aspects above;

[0040] A reading module is used to read the to-be-read data from the compressed data block according to the target compressed header information.

[0041] In a fifth aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method described in the first aspect and the second aspect are implemented.

[0042] In a sixth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described in the first aspect and the second aspect are implemented.

[0043] In a seventh aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect and the second aspect are implemented.

[0044] The above-mentioned data storage method, data reading method, device, computer equipment, computer-readable storage medium and computer program product, the server obtains the data to be stored, and compresses the data to be stored to obtain a compressed data block, then determines the free compression header information from the target compression header information corresponding to the disk, the free compression header information is used to indicate the free storage space in the compressed data block corresponding to the target compression header information, and then, according to the free compression header information, the compressed data block is stored in the free storage space. When storing the compressed data block, it is only necessary to store the compressed data block in the free storage space according to the free compression header information, and there is no need to align the compressed data block with the sector size in the disk, thereby avoiding the problem of space fragmentation inside the sector due to the inconsistency between the size of the compressed data block and the sector size, thereby improving the effective data density of the disk and improving the utilization rate of the disk space. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 An application environment diagram of a data storage method in one embodiment;

[0047] Figure 2 A schematic diagram of a flow chart of a data storage method in one embodiment;

[0048] Figure 3 is a schematic diagram of a compressed data block structure in one embodiment;

[0049] Figure 4 is a flow chart of step 203 in one embodiment;

[0050] Figure 5 is a flow chart of step 402 in one embodiment;

[0051] Figure 6 A schematic diagram of a flow chart of a data reading method in an embodiment;

[0052] Figure 7 is a flow chart of step 603 in one embodiment;

[0053] Figure 8 is a flow chart of step 702 in one embodiment;

[0054] Fig. 9 A schematic diagram of a flow chart of a data storage method in another embodiment;

[0055] Fig.10 is a flow chart of a data reading method in another embodiment;

[0056] Fig.11 is a structural block diagram of a data storage device in one embodiment;

[0057] Fig.12 is a structural block diagram of a data reading device in one embodiment;

[0058] Fig.13 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] Data compression is a technology that reduces the storage space occupied by data by reducing redundant data and optimizing the encoding method. In information systems, there is a large amount of redundant data. Data compression technology can re-encode these redundant data, eliminate redundancy without changing the original information, and increase the effective data density. In the era of big data, the explosive growth of data has prompted the wider application of data compression technology, especially in storage systems. If the data to be stored is compressed to 1 / n of its original size, it is equivalent to increasing the capacity of the storage medium by n times.

[0061] Data compression technology usually includes two subtasks: compression and decompression. The compression task usually occurs before writing to the disk, and the decompression task usually occurs after reading the disk. Depending on whether the decompressed data is completely consistent with the original data before compression, data compression algorithms are usually divided into lossy compression and lossless compression. Lossy compression reduces the size of data by deleting unnecessary information in the original data and reducing the complexity of information. Its advantage is a high compression rate, but the price paid is that the compressed data cannot be accurately restored or reconstructed. Lossless compression reduces the size of data by locating and deleting redundant bits. Its advantage is that the original data can be completely restored from the compressed data, but the disadvantage is that the compression rate is usually low. Lossy compression is often used for images and sounds, and lossless compression is often used for text.

[0062] In a storage system or file system, a disk can be divided into multiple sectors according to a fixed size (such as 512 bytes), and multiple sectors are organized into disk blocks (such as 16 512-byte sectors can form an 8K disk block). Although data compression technology can reduce the size of data, the length of most compressed data after compression is not aligned according to the disk sector or disk block size. Therefore, storing such compressed data on the disk will cause internal fragmentation in the sector. For example, if the sector size is 512B, taking a 1.8K data compression block as an example, the compressed block itself is only 1.8K long. After alignment according to the sector size, it needs to occupy 4 sectors, and there is 0.2K internal fragmentation in the last sector. Although a small amount of internal fragmentation does not have a great impact on the effective data density of the disk, in the big data scenario, a large amount of internal fragmentation will inevitably lead to a significant reduction in the effective data density of the disk.

[0063] In view of this, in the data compression method provided in the embodiment of the present application, the server obtains the data to be stored, and compresses the data to be stored to obtain a compressed data block. Then, the free compression header information is determined from the target compression header information corresponding to the disk, and the free compression header information is used to indicate the free storage space in the compressed data block corresponding to the target compression header information. Then, according to the free compression header information, the compressed data block is stored in the free storage space. When storing the compressed data block, it is only necessary to store the compressed data block in the free storage space according to the free compression header information, and there is no need to align the compressed data block with the sector size in the disk, thereby avoiding the problem of space fragmentation inside the sector due to the inconsistency between the size of the compressed data block and the sector size, thereby improving the effective data density of the disk and improving the utilization rate of the disk space.

[0064] The data storage method provided in the embodiment of the present application can be applied to Figure 1 In the implementation environment shown, the implementation environment includes a server, which can be implemented as an independent server or a server cluster composed of multiple servers, and a data storage system can store data that the server needs to process. The data storage system can be integrated on the server, or placed on the cloud or other network servers. Among them, the server.

[0065] In other possible implementations, the data storage method provided in the embodiments of the present application may also be applied to terminals, which may be, but are not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc.

[0066] In an exemplary embodiment, Figure 2 As shown, a data storage method is provided, which is applied to Figure 1 The server in the example is used to illustrate, including the following steps 201-203:

[0067] Step 201, obtaining data to be stored, and compressing the data to be stored to obtain a compressed data block.

[0068] The data to be stored refers to the data to be stored in the disk, for example, the data to be stored may be voice data, text data, image data, numerical data, etc. Optionally, the data to be stored may be data sent to the server via a network from other devices, or the data to be stored may be data stored in a cache of the server. It is understandable that when storing data, in order to save storage space occupied by the data, the data to be stored may be compressed to obtain compressed data corresponding to the data to be stored.

[0069] In this embodiment, the server can obtain the data to be stored from other devices or its own storage space, and then compress the data to be stored using a data compression algorithm, and determine the compressed data as a compressed data block. Optionally, the data compression algorithm can be a transform coding algorithm, or can also be an incremental coding algorithm, which is not limited in this embodiment.

[0070] Step 202: determine free compression header information from the target compression header information corresponding to the disk, where the free compression header information is used to indicate free storage space in the compressed data block corresponding to the target compression header information.

[0071] In the prior art, when storing data in sectors on a disk, a sector alignment mechanism is used for storage, that is, the starting position corresponding to the data block needs to be aligned with the edge of the physical sector. Therefore, when storing compressed data blocks, the size of the compressed data blocks after compression may be inconsistent with the size of the sectors on the disk, which may cause space fragmentation inside the sectors. In order to avoid the problem of low disk space utilization caused by space fragmentation inside the sectors, data compression technology can be used to arrange and combine multiple compressed data blocks after compression according to certain rules, so as to align the data size obtained after arrangement according to the sector size as much as possible, thereby reducing space fragmentation inside the sectors.

[0072] It should be noted that after multiple compressed data blocks are compressed, a compressed data block can be obtained, and the disk may include multiple compressed data blocks. Each compressed data block includes compression header information and multiple compressed data blocks, wherein the compression header information includes a compression header magic number, a compression data length, a compression header number, and multiple compression header information corresponding to the compression header number. Among them, the compression header magic number is used to determine whether the compression header information is valid and legal compression header information, the compression data length refers to the length of the compressed data contained in the compressed data block, and the compression header number refers to the number of compressed header information included in the compression header information. When storing compressed data blocks, the compressed data blocks can be stored in compressed data blocks with free storage space in the disk, and the compression header information of the compressed data block is determined as the target compression header information.

[0073] There may be multiple compressed header information in the compressed header information. Before storing the compressed data block, the compressed header information is free compressed header information. After storing the compressed data block in the sector corresponding to the compressed data block, the information of the compressed data block may be updated to the free compressed header information. The compressed header information is non-free compressed header information. At this time, the compressed header information includes an offset address, a check value, and a compressed data length. The offset address is the offset of the storage space storing the compressed data block relative to the starting position of the compressed data block, the check value is the check value corresponding to the compressed data, and the compressed data length is the length of the compressed data in the compressed data block. Exemplarily, the structure of the compressed data block may be as follows: Figure 3 shown. Figure 3 In the compressed data block, the compressed header information includes the compressed header magic number, the number of compressed headers, the compressed data length, 16 compressed headers, and 16 compressed data blocks corresponding to the number of 16 compressed headers. Optionally, the number of free compressed header information in the target compressed header information may be one, or the number of free compressed header information in the target compressed header information may be multiple.

[0074] In this embodiment, the server can determine the compressed data block with idle compression header information from the metadata corresponding to each compressed data block, and determine the compression header information in the compressed data block as the target compression header information, thereby determining the compressed header information with no blank value in the compression header information as the idle compression header information from each compressed header information included in the target compression header information.

[0075] In order to further explain the compression header information and compression header information in the compressed data block, the relevant information of the compression header information is described in detail below. It can be understood that since the atomic unit for input and output to the disk is the sector, in order to reduce the probability of errors in the compression header information during access, the size of the compression header information can be a sector size, where the size of the compression header information = the number of bytes occupied by the compression header magic number + the number of bytes occupied by the number of compressed headers + the number of bytes occupied by the compressed data length + the compressed header data length × the number of compressed headers. For example, the number of compressed headers in a compression header information can be 10, 20, 30, etc. Preferably, the number of compressed headers can be [16-50]. In the disk, the compression header information of each compressed data block can occupy a sector, and multiple compressed data blocks in the compressed data block can be stored in sequence in one or more sectors adjacent to the sector. As a possible implementation method, when all the compressed header information in the compressed header information are non-free compressed header information, it can be determined that the compressed data block is full. At this time, if a new compressed data block needs to be stored, the server can re-establish a new compressed data block and generate the compressed header information corresponding to the compressed data block.

[0076] Optionally, the size of the compressed header in the compressed header information can be one or more. The setting of the number of compressed headers in the compressed header information is described in detail below. It can be understood that if the header storage space is reserved according to the number of actual compressed headers in the compressed header information, when new compressed header information is added, it is necessary to make room for the compressed header size for the newly added compressed header, then the positions of all compressed data blocks need to be moved and the relevant offset address information needs to be updated. This process will involve multiple disk IO and memory IO, which will have a greater impact on the overall performance of the compressed data blocks. Therefore, in order to avoid performance problems caused by adding compressed headers, in an embodiment of the present application, the header storage space can be reserved according to the maximum number of compressed headers. It can be understood that even if the number of compressed data blocks contained in the compression header is less than the maximum value, the compression header itself is aligned according to the sector size, and appending a new compression header does not require moving the existing compressed blocks on the disk. The entire append process only involves one disk read IO, i.e. reading the compression header, and two disk write IOs, i.e. writing the compression header and appending the new compressed block. Moreover, the compression header information and the compressed data block are stored in different sectors. Even if a disk error or silent problem occurs in the compressed data block during the disk IO process, the relevant information in the compression header information is still correct. Therefore, as long as the correctness of the compression header information is guaranteed, the consistency check can be performed through the checksum values ​​in the compression header information, thereby narrowing the scope of problem location and enhancing data security.

[0077] On the basis of the above-mentioned maximum reserved header storage space, although for each compressed data block, a sector is required to store the compressed header information, resulting in a waste of a sector size, however, in a big data scenario, a compressed header information containing the maximum number of compressed header information can be realized through greedy thinking. Therefore, even if the last sector occupied by the compressed data block only uses one byte, resulting in a waste of 511 bytes, plus the 512 bytes occupied by the compressed header, N compressed blocks need to waste 1023 bytes, and each compressed block only wastes 1023 / N bytes on average. However, if data compression technology is adopted, when the number of compressed data blocks reaches the order of one hundred thousand to ten million, each compressed block will cause an average waste of 256 bytes of disk space. Therefore, the data compression technology provided by the embodiment of the present application can significantly save disk space.

[0078] Step 203: store the compressed data block in the free storage space according to the free compression header information.

[0079] Optionally, when there is one free compression header information in the target compression header information, the free storage space in the compression data block corresponding to the target compression header information is the storage space corresponding to the free compression header information; when there are multiple free compression header information in the target compression header information, the free storage space in the compression data block corresponding to the target compression header information is the storage space corresponding to multiple free compression header information.

[0080] In this embodiment, the server can determine the number of free compressed header information in the target compacting header according to the free compressed header information, and then determine the storage space for storing the compressed data block according to the number of free compressed header information, and then store the compressed data block in the free storage space. Exemplarily, if the number of free compressed header information is one, the compressed data block can be stored in the free storage space; if the number of free compressed header information is multiple, the free storage space can be divided into two sub-free storage spaces according to the data length of the compressed data block, and the compressed data block is stored in the first sub-free storage space.

[0081] In the above data storage method, the server obtains the data to be stored and compresses the data to be stored to obtain a compressed data block. Then, the free compression header information is determined from the target compression header information corresponding to the disk. The free compression header information is used to indicate the free storage space in the compressed data block corresponding to the target compression header information. Then, according to the free compression header information, the compressed data block is stored in the free storage space. When storing the compressed data block, it is only necessary to store the compressed data block in the free storage space according to the free compression header information, and there is no need to align the compressed data block with the sector size in the disk, thereby avoiding the problem of space fragmentation inside the sector due to the inconsistency between the size of the compressed data block and the sector size, thereby improving the effective data density of the disk and improving the utilization rate of the disk space.

[0082] based on Figure 2 In the embodiment shown, the target compression header information includes multiple compression header information, see Figure 4 This embodiment relates to the process of how the server stores the compressed data block in the free storage space according to the free compression header information. Figure 4 As shown, in this embodiment, step 203 includes Figure 4 Step 401 and step 402 are shown.

[0083] Step 401, detect whether there is non-idle compressed header information in each compressed header information, and obtain a detection result.

[0084] It should be noted that the target compression header information may include multiple compression header information, and the compression header information may be idle compression header information, or the compression header information may be non-idle compression header information, wherein the non-idle compression header information refers to the compression header including the offset address, check value, and compressed data length in the compression header information. The detection result refers to the detection result of whether each compression header information is a null value. Optionally, the detection result may be a null value, or the detection result may be a non-null value.

[0085] In this embodiment, the server can obtain each compressed header information, and then perform a non-empty detection on each compressed header information to obtain a detection result corresponding to each compressed header information. Optionally, when the detection result indicates that a certain compressed header information is a non-empty value, the compressed header can be determined as non-idle compressed header information, so that it can be determined that non-idle compressed header information exists in each compressed header information; when the detection result indicates that all compressed header information are empty values, it can be determined that non-empty compressed header information does not exist in each compressed header information.

[0086] Step 402: store the compressed data block in the free storage space according to the detection result and the free compression header information.

[0087] It can be understood that if the detection result indicates that there is non-free compressed header information in each compression header information, the offset address of the free storage space can be determined based on the offset address in the non-free compressed header information, so that the compressed data block is stored in the free storage space. If the detection result indicates that there is no non-free compressed header information in each compression header information, the offset address of the adjacent sector can be determined based on the storage position of the sector storing the target compression header information, so that the compressed data block is stored in the free storage space.

[0088] In this embodiment, the server may determine the offset address corresponding to the free compressed header information according to the detection result, and thus store the compressed data block in the free storage space according to the offset address.

[0089] In this embodiment, the server obtains a detection result by detecting whether there is non-free compressed header information in each compressed header information. Since the offset address corresponding to the free compressed header information can be determined based on whether there is non-free compressed header information, the compressed data block can be accurately stored in the free storage space based on the detection result and the free compressed header information, thereby improving the accuracy of storing compressed data blocks.

[0090] based on Figure 4 The embodiment shown, see Figure 5 This embodiment relates to the process of how the server stores the compressed data block in the free storage space according to the detection result and the free compression header information. Figure 5 As shown, in this embodiment, step 402 includes Figure 5 Steps 501 to 503 are shown.

[0091] Step 501: when the detection result shows that non-idle compressed header information exists in the target compressed header information, the offset address and the compressed data length are obtained from the non-idle compressed header information adjacent to the idle compressed header information.

[0092] It is understandable that in a compressed data block, multiple compressed data blocks need to be stored continuously. Therefore, when there is non-idle compressed header information in the target compressed header information, the offset address of the free storage space for storing the compressed data block can be determined according to the offset address in the non-idle compressed header information adjacent to the free compressed header information. Therefore, the offset address and the compressed data length can be obtained from the non-idle compressed header information adjacent to the free compressed header information to determine the offset address of the free storage space for storing the compressed data block.

[0093] The offset address refers to the offset address stored in the non-idle compressed header information, and the compressed data length refers to the compressed data length stored in the non-idle compressed header information. In this embodiment, the server can determine that the compressed data block needs to be stored after the compressed data block corresponding to the last non-idle compressed header information when the detection result is that the target compressed header information has non-idle compressed header information, and then obtain the offset address and compressed data length from the non-idle compressed header information adjacent to the idle compressed header information.

[0094] Step 502: Determine the target offset address corresponding to the free storage space according to the offset address and the compressed data length.

[0095] The target offset address refers to the offset address corresponding to the free storage space. According to the target offset address and the starting position of the sector storing the target compression header information, the storage position of the free storage space for storing the compressed data block can be determined.

[0096] In this embodiment, the server may calculate the sum of the offset address and the compressed data length, and then use the sum of the combined value plus 1 as the offset address of the free storage space for storing the compressed data block.

[0097] Step 503: store the compressed data block in the free storage space according to the target offset address.

[0098] In this embodiment, the server can obtain the starting position of the sector according to the storage target compression header information, and then combine the target offset address with the starting position to determine the storage position of the free storage space in the disk, and then store the compressed data block in the storage position.

[0099] In this embodiment, when the server detects that non-free compressed header information exists in the target compression header information, it can determine the target offset address corresponding to the free storage space based on the offset address and the compressed data length by obtaining the offset address and the compressed data length from the non-free compressed header information adjacent to the free compressed header information. Since the non-free compressed header information includes the offset address and the compressed data length, the offset address and the compressed data length can be obtained quickly and accurately, and thus the target offset address corresponding to the free storage space can be quickly and accurately determined based on the offset address and the compressed data length, and then the compressed data block can be stored in the free storage space based on the target offset address, which can improve the efficiency and reliability of storing compressed data blocks.

[0100] based on Figure 4 The embodiment shown in the figure is related to the process of how the server stores the compressed data block in the free storage space according to the detection result and the free compression header information. The server can perform the following steps to implement the process:

[0101] When the detection result is that there is no non-free compressed header information in the target compressed header information, the first storage space in the compressed data block is determined as the free storage space, and the compressed data block is stored in the free storage space.

[0102] It should be noted that, in the embodiment of the present application, a sector can be used to store the compressed header information, and the compressed data block can be stored in one or more free sectors adjacent to the sector. Therefore, if there is no non-free compressed header information in the target compressed header information, the compressed data block can be stored starting from the starting position of the free storage space. Among them, the first storage space in the compressed data block refers to the hollow storage space corresponding to the starting position of the free storage space.

[0103] In this embodiment, when the detection result is that there is no non-free compression header information in the target compression header information, the server can obtain the storage location of the sector storing the target compression header information from the metadata information in the disk, and then obtain the storage location of the free sector adjacent to the sector, and then determine the first storage space in the storage location as the free storage space, and then store the compressed data block in the free storage space.

[0104] In this embodiment, when the server detects that there is no non-free compression header information in the target compression header information, it can determine the first storage space in the compressed data block as free storage space, so that the compressed data block can be directly stored in the free storage space, thereby improving the efficiency of storing compressed data blocks.

[0105] Based on the above embodiment, this embodiment involves a process in which the server performs legal verification on target compression header information. The server can perform the following steps to implement the process:

[0106] According to the compression header magic number included in the target compression header information, the target compression header information is legally verified to obtain a verification result.

[0107] The compression header magic number is used to identify whether the type of the compression header information corresponds to legal compression header information. In this embodiment, the server can obtain the corresponding valid type field from the metadata, and verify the legality of the target compression header information by comparing the valid type field with the compression header magic number included in the target compression header information to obtain a verification result. Exemplarily, if the valid type field is consistent with the compression header magic number, it can be determined that the verification result can be that the target compression header information is legal; if the valid type field is inconsistent with the compression header magic number, it can be determined that the verification result can be that the target compression header information is illegal.

[0108] Correspondingly, the above step 503 includes: when the verification result shows that the target compression header information is legal, storing the compressed data block in the free storage space according to the target offset address.

[0109] It is understandable that if the target compression header information is illegal, it means that the information included in the target compression header information may contain erroneous information, or the type of information included in the target compression header information is wrong. Therefore, it should be prohibited to store the compressed data block in the free storage space.

[0110] In this embodiment, when the target compaction header information is legal, the server can determine that the information included in the target compaction header information is correct information, and thus store the compressed data block in the free storage space according to the target offset address.

[0111] In this embodiment, the server performs legal verification on the target compression header information based on the compression header magic number included in the target compression header information, and can obtain a verification result. Therefore, when the verification result indicates that the target compression header information is legal, the server can store the compressed data block in the free storage space according to the target offset address, thereby improving the accuracy and reliability of data storage.

[0112] In one embodiment, based on the above embodiment, see Figure 6 , this embodiment involves the process of how the server Figure 6 As shown, step 203 includes Figure 6 Steps 601 to 603 shown:

[0113] In an exemplary embodiment, Figure 6 As shown, a data reading method is provided, which is applied to Figure 1 The server in the example is used to illustrate, including the following steps 601-603:

[0114] Step 601: receiving a data read request, wherein the data read request includes a file identifier corresponding to the data to be read.

[0115] The data read request refers to a request for reading stored data from a server. Optionally, the source of the data read request may be other devices, or the source of the data read request may be triggered by a user through a server. It is understood that when the data to be read is stored, the storage information of the data to be read in the file system may generate a file identifier after being stored on the disk.

[0116] In this embodiment, the server may receive a data reading request through a network, and parse the data reading request to obtain a file identifier corresponding to the data to be read included in the data reading request.

[0117] Step 602: Determine target compression header information corresponding to the file identifier according to the file identifier, and the compressed data stored in the compression data block corresponding to the target compression header information is stored according to the data storage method.

[0118] The target compression header information is the compression header information of the compressed data block storing the compressed data block corresponding to the data to be read. It can be understood that the data to be read is stored in the disk through the data compression technology. Therefore, in the process of reading data, the corresponding target compression header information identifier can be obtained from the metadata according to the file identifier, and the target compression header information can be obtained from the disk according to the target compression header information identifier.

[0119] In this embodiment, the server can obtain the target compression header information identifier corresponding to the file identifier from the metadata corresponding to the file system based on the file identifier, and then determine the sector storing the target compression header information based on the target compression header information identifier, and then obtain the target compression header information from the sector.

[0120] Step 603: Read the data to be read from the compressed data block according to the target compressed header information.

[0121] The target compaction header information includes compression header information. In this embodiment, the server can determine the storage location of the compressed data block corresponding to the data to be read in the compaction data block according to the target compaction header information, and read the data to be read from the storage space corresponding to the storage location.

[0122] In this embodiment, the server receives a data reading request, which includes a file identifier corresponding to the data to be read, and then can determine the target compression header information corresponding to the file identifier based on the file identifier. Since the compressed data stored in the compressed data block corresponding to the target compression header information is stored according to the data storage method, the storage location of the compressed data block corresponding to the data to be read can be determined based on the target compression header information, so that the data to be read can be read from the compressed data block according to the storage space corresponding to the storage location, and there is no need to read all the compressed data blocks corresponding to the compressed data block from the disk, thereby reducing the time overhead and space overhead in the data reading process and improving the efficiency of data reading.

[0123] based on Figure 6 The embodiment shown, see Figure 7 This embodiment relates to a process of how the server reads the data to be read from the compressed data block according to the target compressed header information. Figure 7 As shown, in this embodiment, step 603 includes Figure 7 Step 701 and step 702 are shown.

[0124] Step 701: Determine target compression header information corresponding to the file identifier from the target compression header information according to the file identifier.

[0125] The target compression header information is the relevant information of the compressed data block corresponding to the data to be read, and the target compression header information may include the offset address of the compressed data block, the compressed data length corresponding to the compressed data block, and the check value corresponding to the compressed data block.

[0126] In this embodiment, the server can determine the identifier of the target compression header information corresponding to the data to be read based on the file identifier, and then, can determine the subscript of the target compression header information in the target compression header information based on the identifier of the target compression header information, and then, can determine the target compression header information from the multiple compression header information included in the target compression header information based on the subscript.

[0127] Step 702: Obtain the data to be read according to the target offset address included in the target compression header information.

[0128] The target offset address refers to the offset of the storage space storing the compressed data block relative to the starting position of the sector storing the target compression header information.

[0129] In this embodiment, the server can obtain the target offset address from the target compression header information, and then determine the storage location of the compressed data block corresponding to the data to be read based on the target offset address and the starting position of the sector storing the target compression header information, and then obtain the data to be read from the storage space corresponding to the storage location.

[0130] In this embodiment, the server can accurately and quickly determine the target compression header information corresponding to the file identifier from the target compression header information based on the file identifier, so that according to the target offset address included in the target compression header information, it can quickly and accurately obtain the data to be read, thereby improving the efficiency and reliability of data reading.

[0131] based on Figure 7 The embodiment shown, see Figure 8 This embodiment relates to a process of how the server obtains the data to be read according to the target offset address included in the target compression header information. Figure 8 As shown, in this embodiment, step 702 includes Figure 8 Steps 801 to 803 are shown.

[0132] Step 801: read a compressed data block from a compressed data block according to a target offset address, and read a compressed data length and a first check value of the compressed data block from target compression header information.

[0133] Among them, the compressed data block is a compressed data block corresponding to the data to be read, and the first check value refers to the check value calculated according to the data length of the compressed data block during the process of storing the compressed data block. Optionally, the first check value can be a hash value, or the first check value can also be a cyclic redundancy check code.

[0134] In this embodiment, the server can determine the storage space for storing the compressed data block in the compressed data block according to the target offset address, and then read the compressed data block from the storage space, and can read the compressed data length and the first check value of the compressed data block from the target compression header information.

[0135] Step 802: Calculate a second check value according to the compressed data length.

[0136] In this embodiment, the server may calculate the second check value according to the length of the compressed data using the same algorithm as that used to generate the first check value. Optionally, the second check value and the first check value are of the same type.

[0137] As a possible implementation, the first check value in the compression header information is calculated based on the data length of the compressed data when storing the compressed data block. The server can also determine the actual data length based on the obtained compressed data block, and then calculate the second check value based on the actual data length.

[0138] Step 803: If the first check value is the same as the second check value, decompress the compressed data block to obtain the data to be read.

[0139] In this embodiment, the server may compare the first check value with the second check value, and then use a data decompression algorithm to decompress the compressed data block when the first check value is the same as the second check value, and use the decompression result as the data to be read.

[0140] In this embodiment, the server reads the compressed data block from the compressed data block according to the target offset address, and reads the compressed data length and the first check value of the compressed data block from the target compression header information, and then calculates the second check value according to the compressed data length. Then, when the first check value is the same as the second check value, the compressed data block is decompressed to obtain the data to be read. By making the first check value and the second check value the same, it can be ensured that there is no data change in the compressed data block after it is stored on the disk, thereby ensuring the reliability and accuracy of data reading.

[0141] In one embodiment, a data storage method is provided for use in a server, such as Fig. 9 As shown, the method comprises the following steps:

[0142] Step 901: obtain data to be stored, and compress the data to be stored to obtain a compressed data block.

[0143] Step 902: determine the free compression header information from the target compression header information corresponding to the disk.

[0144] Step 903: Detect whether there is non-idle compressed header information in each compressed header information, and obtain a detection result.

[0145] Step 904: when the detection result shows that non-idle compressed header information exists in the target compressed header information, the offset address and the compressed data length are obtained from the non-idle compressed header information adjacent to the idle compressed header information.

[0146] Step 905: Determine the target offset address corresponding to the free storage space according to the offset address and the compressed data length.

[0147] Step 906: perform legal verification on the target compaction header information according to the compaction header magic number included in the target compaction header information to obtain a verification result.

[0148] Step 907: When the verification result shows that the target compression header information is legal, the compressed data block is stored in the free storage space according to the target offset address.

[0149] Step 908: When the detection result shows that there is no non-free compressed header information in the target compressed header information, the first storage space in the compressed data block is determined as free storage space, and the compressed data block is stored in the free storage space.

[0150] In one embodiment, a data reading method is provided for use in a server, such as Fig.10 As shown, the method comprises the following steps:

[0151] Step 1001, receiving a data read request.

[0152] Step 1002: Determine target compression header information corresponding to the file identifier from the target compression header information according to the file identifier.

[0153] Step 1003: Read the compressed data block from the compressed data block according to the target offset address.

[0154] Step 1004: read the compressed data length and the first check value of the compressed data block from the target compression header information.

[0155] Step 1005: Calculate a second check value according to the compressed data length.

[0156] Step 1006: If the first check value is the same as the second check value, decompress the compressed data block to obtain the data to be read.

[0157] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0158] Based on the same inventive concept, the embodiment of the present application also provides a data storage device for implementing the data storage method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more data storage device embodiments provided below can refer to the limitations on the data storage method above, and will not be repeated here.

[0159] In an exemplary embodiment, Fig.11 As shown, a data storage device is provided, comprising:

[0160] The compression module 1101 is used to obtain the data to be stored and compress the data to be stored to obtain a compressed data block;

[0161] A first determining module 1102 is used to determine free compression header information from target compression header information corresponding to the disk, where the free compression header information is used to indicate free storage space in a compressed data block corresponding to the target compression header information;

[0162] The storage module 1103 is used to store the compressed data block in the free storage space according to the free compression header information.

[0163] In one embodiment, the target compressed header information includes a plurality of compressed header information, and the storage module 1103 includes:

[0164] A detection unit, used to detect whether there is non-idle compressed header information in each compressed header information, and obtain a detection result;

[0165] The storage unit is used to store the compressed data block in the free storage space according to the detection result and the free compression header information.

[0166] In one embodiment, the storage unit is specifically used for:

[0167] When the detection result shows that non-idle compressed header information exists in the target compressed header information, an offset address and a compressed data length are obtained from the non-idle compressed header information adjacent to the idle compressed header information;

[0168] Determine the target offset address corresponding to the free storage space according to the offset address and the compressed data length;

[0169] The compressed data block is stored in the free storage space according to the target offset address.

[0170] In one embodiment, the storage unit is specifically used for:

[0171] When the detection result is that there is no non-free compressed header information in the target compressed header information, the first storage space in the compressed data block is determined as the free storage space, and the compressed data block is stored in the free storage space.

[0172] In one embodiment, the above device further comprises:

[0173] A verification module, used to verify the legality of the target compression header information according to the compression header magic number included in the target compression header information, and obtain a verification result;

[0174] The storage unit is specifically used to: when the verification result shows that the target compression header information is legal, store the compressed data block in the free storage space according to the target offset address.

[0175] Each module in the above data reading and storage device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a server in the form of hardware, or can be stored in a memory in a server in the form of software, so that the processor can call and execute operations corresponding to each module.

[0176] Based on the same inventive concept, the embodiment of the present application also provides a data reading device for implementing the data reading method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data reading device embodiments provided below can refer to the limitations on the data reading method above, and will not be repeated here.

[0177] In an exemplary embodiment, Fig.12 As shown, a reading device is provided, comprising:

[0178] The receiving module 1201 is used to receive a data reading request, where the data reading request includes a file identifier corresponding to the data to be read;

[0179] The second determining module 1202 is used to determine the target compression header information corresponding to the file identifier according to the file identifier, and the compressed data stored in the compressed data block corresponding to the target compression header information is stored according to the above data storage method;

[0180] The reading module 1203 is used to read the to-be-read data from the compressed data block according to the target compressed header information.

[0181] In one embodiment, the reading module 1203 includes:

[0182] A determination unit, configured to determine target compression header information corresponding to the file identifier from the target compression header information according to the file identifier;

[0183] The acquisition unit is used to acquire the data to be read according to the target offset address included in the target compression header information.

[0184] In one embodiment, the acquisition unit is specifically used to:

[0185] Reading a compressed data block from the compressed data block according to the target offset address, and reading a compressed data length and a first check value of the compressed data block from the target compression header information;

[0186] Calculate a second check value according to the compressed data length;

[0187] If the first check value is the same as the second check value, the compressed data block is decompressed to obtain the data to be read.

[0188] Each module in the above data reading device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a server in the form of hardware, or can be stored in a memory in a server in the form of software, so that the processor can call and execute operations corresponding to each module.

[0189] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.13 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data of compressed data blocks. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data storage method is implemented.

[0190] Those skilled in the art will understand that Fig.13 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0191] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0192] Acquire the data to be stored, and compress the data to be stored to obtain a compressed data block;

[0193] Determine free compression header information from the target compression header information corresponding to the disk, where the free compression header information is used to indicate free storage space in the compression data block corresponding to the target compression header information;

[0194] According to the free compression header information, the compressed data block is stored in the free storage space.

[0195] In one embodiment, the processor implements the following steps when executing the computer program:

[0196] Detect whether there is non-idle compressed header information in each compressed header information, and obtain a detection result;

[0197] According to the detection result and the free compression header information, the compressed data block is stored in the free storage space.

[0198] In one embodiment, the processor implements the following steps when executing the computer program:

[0199] When the detection result shows that non-idle compressed header information exists in the target compressed header information, an offset address and a compressed data length are obtained from the non-idle compressed header information adjacent to the idle compressed header information;

[0200] Determine the target offset address corresponding to the free storage space according to the offset address and the compressed data length;

[0201] The compressed data block is stored in the free storage space according to the target offset address.

[0202] In one embodiment, the processor implements the following steps when executing the computer program:

[0203] When the detection result is that there is no non-free compressed header information in the target compressed header information, the first storage space in the compressed data block is determined as the free storage space, and the compressed data block is stored in the free storage space.

[0204] In one embodiment, the processor implements the following steps when executing the computer program:

[0205] According to the compression header magic number included in the target compression header information, the target compression header information is legally verified to obtain a verification result;

[0206] When the verification result shows that the target compression header information is legal, the compressed data block is stored in the free storage space according to the target offset address.

[0207] In an exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0208] receiving a data reading request, wherein the data reading request includes a file identifier corresponding to the data to be read;

[0209] Determine, according to the file identifier, target compression header information corresponding to the file identifier, wherein the compressed data stored in the compression data block corresponding to the target compression header information is stored according to the method of any one of claims 1 to 5;

[0210] Read the data to be read from the compressed data block according to the target compressed header information.

[0211] In one embodiment, the processor implements the following steps when executing the computer program:

[0212] According to the file identifier, determining the target compression header information corresponding to the file identifier from the target compression header information;

[0213] The data to be read is obtained according to the target offset address included in the target compression header information.

[0214] In one embodiment, the processor implements the following steps when executing the computer program:

[0215] Reading a compressed data block from the compressed data block according to the target offset address, and reading a compressed data length and a first check value of the compressed data block from the target compression header information;

[0216] Calculate a second check value according to the compressed data length;

[0217] If the first check value is the same as the second check value, the compressed data block is decompressed to obtain the data to be read.

[0218] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0219] Acquire the data to be stored, and compress the data to be stored to obtain a compressed data block;

[0220] Determine free compression header information from the target compression header information corresponding to the disk, where the free compression header information is used to indicate free storage space in the compression data block corresponding to the target compression header information;

[0221] According to the free compression header information, the compressed data block is stored in the free storage space.

[0222] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0223] Detect whether there is non-idle compressed header information in each compressed header information, and obtain a detection result;

[0224] According to the detection result and the free compression header information, the compressed data block is stored in the free storage space.

[0225] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0226] When the detection result shows that non-idle compressed header information exists in the target compressed header information, an offset address and a compressed data length are obtained from the non-idle compressed header information adjacent to the idle compressed header information;

[0227] Determine the target offset address corresponding to the free storage space according to the offset address and the compressed data length;

[0228] The compressed data block is stored in the free storage space according to the target offset address.

[0229] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0230] When the detection result is that there is no non-free compressed header information in the target compressed header information, the first storage space in the compressed data block is determined as the free storage space, and the compressed data block is stored in the free storage space.

[0231] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0232] According to the compression header magic number included in the target compression header information, the target compression header information is legally verified to obtain a verification result;

[0233] When the verification result shows that the target compression header information is legal, the compressed data block is stored in the free storage space according to the target offset address.

[0234] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0235] receiving a data reading request, wherein the data reading request includes a file identifier corresponding to the data to be read;

[0236] Determine, according to the file identifier, target compression header information corresponding to the file identifier, wherein the compressed data stored in the compression data block corresponding to the target compression header information is stored according to the method of any one of claims 1 to 5;

[0237] Read the data to be read from the compressed data block according to the target compressed header information.

[0238] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0239] According to the file identifier, determining the target compression header information corresponding to the file identifier from the target compression header information;

[0240] The data to be read is obtained according to the target offset address included in the target compression header information.

[0241] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0242] Reading a compressed data block from the compressed data block according to the target offset address, and reading a compressed data length and a first check value of the compressed data block from the target compression header information;

[0243] Calculate a second check value according to the compressed data length;

[0244] If the first check value is the same as the second check value, the compressed data block is decompressed to obtain the data to be read.

[0245] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0246] Acquire the data to be stored, and compress the data to be stored to obtain a compressed data block;

[0247] Determine free compression header information from the target compression header information corresponding to the disk, where the free compression header information is used to indicate free storage space in the compression data block corresponding to the target compression header information;

[0248] According to the free compression header information, the compressed data block is stored in the free storage space.

[0249] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0250] Detect whether there is non-idle compressed header information in each compressed header information, and obtain a detection result;

[0251] According to the detection result and the free compression header information, the compressed data block is stored in the free storage space.

[0252] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0253] When the detection result shows that non-idle compressed header information exists in the target compressed header information, an offset address and a compressed data length are obtained from the non-idle compressed header information adjacent to the idle compressed header information;

[0254] Determine the target offset address corresponding to the free storage space according to the offset address and the compressed data length;

[0255] The compressed data block is stored in the free storage space according to the target offset address.

[0256] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0257] When the detection result is that there is no non-free compressed header information in the target compressed header information, the first storage space in the compressed data block is determined as the free storage space, and the compressed data block is stored in the free storage space.

[0258] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0259] According to the compression header magic number included in the target compression header information, the target compression header information is legally verified to obtain a verification result;

[0260] When the verification result shows that the target compression header information is legal, the compressed data block is stored in the free storage space according to the target offset address.

[0261] In one embodiment, a computer program product is also provided, including a computer program, which, when executed by a processor, implements the following steps:

[0262] receiving a data reading request, wherein the data reading request includes a file identifier corresponding to the data to be read;

[0263] Determine, according to the file identifier, target compression header information corresponding to the file identifier, wherein the compressed data stored in the compression data block corresponding to the target compression header information is stored according to the method of any one of claims 1 to 5;

[0264] Read the data to be read from the compressed data block according to the target compressed header information.

[0265] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0266] According to the file identifier, determining the target compression header information corresponding to the file identifier from the target compression header information;

[0267] The data to be read is obtained according to the target offset address included in the target compression header information.

[0268] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0269] Reading a compressed data block from the compressed data block according to the target offset address, and reading a compressed data length and a first check value of the compressed data block from the target compression header information;

[0270] Calculate a second check value according to the compressed data length;

[0271] If the first check value is the same as the second check value, the compressed data block is decompressed to obtain the data to be read.

[0272] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0273] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0274] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data storage method, characterized in that: The method comprises: Acquire data to be stored, and compress the data to be stored to obtain a compressed data block; Determine free compression header information from the target compression header information corresponding to the disk, wherein the free compression header information is used to indicate free storage space in the compression data block corresponding to the target compression header information; According to the free compression header information, the compressed data block is stored in the free storage space.

2. The method according to claim 1, characterized in that The target compression header information includes a plurality of compression header information, and storing the compressed data block in the free storage space according to the free compression header information includes: Detecting whether there is non-idle compressed header information in each of the compressed header information, and obtaining a detection result; According to the detection result and the free compression header information, the compressed data block is stored in the free storage space.

3. The method according to claim 2, characterized in that The step of storing the compressed data block in the free storage space according to the detection result and the free compression header information includes: When the detection result is that the non-idle compressed header information exists in the target compressed header information, obtaining an offset address and a compressed data length from the non-idle compressed header information adjacent to the idle compressed header information; Determine a target offset address corresponding to the free storage space according to the offset address and the compressed data length; The compressed data block is stored in the free storage space according to the target offset address.

4. The method according to claim 2, characterized in that: The step of storing the compressed data block in the free storage space according to the detection result and the free compression header information includes: When the detection result is that the non-free compressed header information does not exist in the target compressed header information, the first storage space in the compressed data block is determined as the free storage space, and the compressed data block is stored in the free storage space.

5. The method according to claim 3, characterized in that: Before storing the compressed data block in the free storage space according to the target offset address, the method further includes: According to the compression header magic number included in the target compression header information, the target compression header information is legally verified to obtain a verification result; The step of storing the compressed data block in the free storage space according to the target offset address comprises: When the verification result shows that the target compression header information is legal, the compressed data block is stored in the free storage space according to the target offset address.

6. A data reading method, characterized in that: The method comprises: Receiving a data reading request, wherein the data reading request includes a file identifier corresponding to the data to be read; Determining target compression header information corresponding to the file identifier according to the file identifier, wherein the compressed data stored in the compression data block corresponding to the target compression header information is stored according to the method described in any one of claims 1 to 5; The to-be-read data is read from the compressed data block according to the target compressed header information.

7. The method according to claim 6, characterized in that The step of reading the to-be-read data from the compressed data block according to the target compressed header information comprises: According to the file identifier, determining the target compression header information corresponding to the file identifier from the target compression header information; The data to be read is obtained according to the target offset address included in the target compression header information.

8. The method according to claim 7, characterized in that The acquiring the to-be-read data according to the target offset address included in the target compression header information comprises: Reading a compressed data block from the compressed data block according to the target offset address, and reading a compressed data length and a first check value of the compressed data block from the target compression header information; Calculate a second check value according to the compressed data length; If the first check value is the same as the second check value, the compressed data block is decompressed to obtain the data to be read.

9. A data storage device, characterized in that: The device comprises: A compression module, used to obtain data to be stored, and compress the data to be stored to obtain a compressed data block; A first determining module, configured to determine free compression header information from target compression header information corresponding to the disk, wherein the free compression header information is used to indicate free storage space in a compression data block corresponding to the target compression header information; A storage module is used to store the compressed data block in the free storage space according to the free compression header information.

10. A data reading device, characterized in that: The device comprises: A receiving module, used for receiving a data reading request, wherein the data reading request includes a file identifier corresponding to the data to be read; a second determining module, configured to determine, according to the file identifier, target compression header information corresponding to the file identifier, wherein the compressed data stored in the compression data block corresponding to the target compression header information is stored according to the method described in any one of claims 1 to 5; A reading module is used to read the to-be-read data from the compressed data block according to the target compressed header information.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images