A ZSTD data packet retrieval method, device, equipment and medium

Abstract

The application discloses a ZSTD data packet retrieval method and device, equipment and medium, relates to the technical field of data retrieval, and comprises the following steps: querying a preset index table, obtaining retrieval related information of a target data packet, determining a first data length to be solved, estimating a first compressed frame length, reading first compressed data from a ZSTD compressed file, and decompressing to obtain first decompressed data and a first actual data length to be solved; if it is determined that the first decompressed data meets a preset complete packet header condition, the first decompressed data is determined as packet header data; a second data length to be solved is determined according to the packet header data and the retrieval related information, a second compressed data is read by appending the second data length to be solved, and second decompressed data and a second actual data length to be solved are obtained after decompression; and if it is determined that the second decompressed data meets a preset complete data packet condition, the second decompressed data is determined as the target data packet.

Description

Technical Field

[0001] This invention relates to the field of data retrieval technology, and in particular to a ZSTD data packet retrieval method, apparatus, device, and medium. Background Technology

[0002] In the field of data storage and retrieval, ZSTD (Zstandard, a lossless data compression algorithm) compression algorithm is widely used due to its high compression performance. A large amount of business data (such as logs, backup files, and database snapshots) is stored in ZSTD compressed format. When retrieving specific data packets from a ZSTD compressed file, the common approach is to pre-set the initial read length based on experience, combined with dynamically appending the data based on block information from streaming decompression. However, current technology requires parsing the ZSTD compressed file frame by frame to obtain the compressed frame offset. This process traverses the frame header information of multiple compressed frames, which is time-consuming and increases pre-retrieval overhead. Furthermore, the initial read length is based on experience, which may result in reading unnecessary compressed data, requiring disk I / O (Input / Output) transfers and consuming memory. The proportion of invalid I / O transfers is high, increasing I / O overhead. When decompressed data is incomplete, appending with a fixed block length (e.g., 64KB) increases the number of I / O operations, significantly prolonging retrieval time.

[0003] In summary, how to reduce the IO overhead and number of IO operations in the ZSTD packet retrieval process while ensuring data integrity, thereby improving packet retrieval efficiency, is a pressing technical problem that needs to be solved. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a ZSTD data packet retrieval method, apparatus, device, and medium that can reduce IO overhead and IO counts in the ZSTD data packet retrieval process while ensuring data integrity, thereby improving data packet retrieval efficiency. The specific solution is as follows:

[0005] Firstly, this application provides a ZSTD data packet retrieval method, including:

[0006] The preset index table of the ZSTD compressed file is queried to obtain the retrieval information of the target data packet, so as to determine the first decompressed data length to be used to obtain the header data of the target data packet. Based on the retrieval information and the first decompressed data length, the length of the first compressed frame to be read for the first time is estimated, so as to read the first compressed data from the ZSTD compressed file, decompress it to obtain the first decompressed data and the first actual decompressed data length after decompression.

[0007] If, based on the first decompressed data length to be decompressed and the first actual decompressed data length, it is determined that the first decompressed data meets the preset complete header condition, then the first decompressed data is determined as the header data of the target data packet;

[0008] The second decompressed data length to be obtained for acquiring the target data packet is determined based on the packet header data and the retrieval related information, so as to estimate the length of the second compressed frame to be read in advance. The second compressed data is read in advance from the ZSTD compressed file, and the second decompressed data and the second actual decompressed data length are obtained after decompression.

[0009] If, based on the second actual decompressed data length and the second expected decompressed data length, it is determined that the second decompressed data meets the preset complete data packet condition, then the second decompressed data is identified as the target data packet.

[0010] Optionally, before querying the preset index table of ZSTD compressed files, the method further includes:

[0011] The ZSTD algorithm is used to compress the file to be compressed, so as to obtain the ZSTD compressed file containing several ZSTD compressed frames;

[0012] During the generation of the aforementioned ZSTD compressed frames, the corresponding compressed frame offset, intra-frame offset, and file compression ratio of the ZSTD compressed frames are calculated and stored in the preset index table.

[0013] Wherein, the compressed frame offset is the offset of the starting position of the pre-compression header of the compressed frame relative to the starting address of the ZSTD compressed file; the intra-frame offset is the offset of the starting position of the pre-compression header of the target data packet relative to the starting address of the corresponding compressed frame.

[0014] Optionally, determining the first decompressed data length for obtaining the header data of the target data packet, and estimating the length of the first compressed frame to be read initially based on the retrieval information and the first decompressed data length, to read the first compressed data from the ZSTD compressed file, and decompressing to obtain the first decompressed data and the first actual decompressed data length, includes:

[0015] The sum of the intra-frame offset and the actual header length recorded in the retrieved information is determined as the first decipherable data length for obtaining the header data of the target data packet;

[0016] Determine the quotient of the first data length to be decrypted and the file compression ratio recorded in the relevant retrieval information, and round the quotient up to determine the result as the length of the first compressed frame read for the first time.

[0017] Based on the first compressed frame length and the compressed frame offset in the retrieved related information, a corresponding first read address range is determined so as to read the first compressed data from the ZSTD compressed file according to the first read address range;

[0018] The first compressed data is input into the target streaming decompression interface to obtain the corresponding first decompressed data and the length of the first actual decompressed data.

[0019] Optionally, the step of determining the first decompressed data as the header data of the target data packet based on the first expected decompressed data length and the first actual decompressed data length includes:

[0020] The first actual solved data length is compared with the first data length to be solved to obtain the corresponding first comparison result;

[0021] If the first comparison result indicates that the length of the first actual decompressed data is not less than the length of the first data to be decompressed, then the first decompressed data is determined to meet the preset complete header condition, so that the first decompressed data is determined as the header data of the target data packet;

[0022] If the first comparison result indicates that the actual decompressed data length is less than the expected decompressed data length, then it is determined that the current first decompressed data does not meet the preset complete packet header condition. The first undecompressed length of the current data block is obtained through the target streaming decompression interface, and the first sum of the first undecompressed length and the data block header length is determined.

[0023] Determine the first difference between the first data length to be decoded and the first actual data length, and determine the quotient between the first difference and the file compression ratio as the first theoretical append length;

[0024] The maximum value between the first sum and the first theoretical append length is determined as the length of the third compressed frame to be appended and read, and the third compressed data of the third compressed frame length is appended and read from the current end address of the ZSTD compressed file.

[0025] The third compressed data is decompressed through the target streaming decompression interface to update the length of the first decompressed data and the first actual decompressed data, and then the process jumps to the step of comparing the length of the first actual decompressed data with the length of the first data to be decompressed.

[0026] Optionally, determining the second decoded data length for obtaining the target data packet based on the packet header data and the retrieval-related information includes:

[0027] Read the header data to obtain the length of the target data packet;

[0028] The sum of the data packet length and the intra-frame offset recorded in the retrieval-related information is determined as the second decrypted data length used to obtain the target data packet.

[0029] Optionally, estimating the length of the second compressed frame to be read, appending the second compressed data to the ZSTD compressed file, and obtaining the second decompressed data and the length of the second actual decompressed data after decompression, includes:

[0030] Determine the second difference between the second data length to be decoded and the first actual data length, and determine the quotient between the second difference and the file compression ratio in the retrieved relevant information as the second compressed frame length;

[0031] The second compressed data, the second compressed frame length, is appended to the current end address of the ZSTD compressed file.

[0032] The second compressed data is decompressed through the target streaming decompression interface to obtain the second decompressed data and the corresponding newly added decompressed length, and the current decompressed second actual decompressed data length is obtained based on the first actual decompressed data length and the newly added decompressed length.

[0033] Optionally, the step of determining the second decompressed data as the target data packet based on the second actual decompressed data length and the second expected decompressed data length includes:

[0034] The second actual solved data length is compared with the second data length to be solved to obtain the corresponding second comparison result;

[0035] If the second comparison result indicates that the actual decompressed data length is not less than the expected decompressed data length, then the second decompressed data is determined to meet the preset complete data packet condition, so that the second decompressed data is identified as the target data packet;

[0036] If the second comparison result indicates that the actual decompressed data length is less than the expected decompressed data length, then it is determined that the current second decompressed data does not meet the preset complete data packet condition. The second undecompressed length of the current data block is obtained through the target streaming decompression interface, and the second sum of the second undecompressed length and the data block header length is determined.

[0037] Determine the second difference between the second expected data length and the second actual data length, and determine the quotient between the second difference and the file compression ratio as the second theoretical append length;

[0038] The maximum value between the second sum and the second theoretical append length is determined as the length of the fourth compressed frame to be appended and read, and the fourth compressed data of the fourth compressed frame length is appended and read from the current end address of the ZSTD compressed file.

[0039] The fourth compressed data is decompressed through the target streaming decompression interface to update the second decompressed data and the second actual decompressed data length, and then the process jumps to the step of comparing the second actual decompressed data length with the second data length to be decompressed.

[0040] Secondly, this application provides a ZSTD data packet retrieval device, comprising:

[0041] The data reading module is used to query the preset index table of the ZSTD compressed file to obtain the retrieval information of the target data packet, so as to determine the first decompressed data length to be used to obtain the header data of the target data packet, and to estimate the length of the first compressed frame to be read for the first time based on the retrieval information and the first decompressed data length, so as to read the first compressed data from the ZSTD compressed file, decompress it to obtain the first decompressed data and the first actual decompressed data length after decompression;

[0042] The data determination module is used to determine the first decompressed data as the header data of the target data packet if, based on the first decompressed data length to be decompressed and the first actual decompressed data length, it is determined that the first decompressed data satisfies the preset complete header condition.

[0043] The length determination module is used to determine the second decompressed data length for obtaining the target data packet based on the packet header data and the retrieval related information, so as to estimate the length of the second compressed frame to be read, to read the second compressed data from the ZSTD compressed file, and to obtain the second decompressed data and the second actual decompressed data length after decompression.

[0044] The data packet determination module is used to determine the second decompressed data as the target data packet if, based on the second actual decompressed data length and the second expected decompressed data length, it is determined that the second decompressed data satisfies the preset complete data packet condition.

[0045] Thirdly, this application provides an electronic device, comprising:

[0046] Memory, used to store computer programs;

[0047] A processor is used to execute the computer program to implement the aforementioned ZSTD data packet retrieval method.

[0048] Fourthly, this application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned ZSTD data packet retrieval method.

[0049] In this application, a preset index table of the ZSTD compressed file is queried to obtain retrieval information related to the target data packet, thereby determining the first decompressed data length to be used to obtain the header data of the target data packet. Based on the retrieval information and the first decompressed data length, the length of the first compressed frame to be read initially is estimated, so as to read the first compressed data from the ZSTD compressed file, decompress it to obtain the first decompressed data and the first actual decompressed data length. If, based on the first decompressed data length and the first actual decompressed data length, it is determined that the first decompressed data meets a preset complete header condition, then the first decompressed data is determined as the header data of the target data packet. Based on the header data and the retrieval information, a second decompressed data length to be used to obtain the target data packet is determined, thereby estimating the length of the second compressed frame to be read additionally. The second compressed data is then read additionally from the ZSTD compressed file, decompressed to obtain the second decompressed data and the second actual decompressed data length. If, based on the second actual decompressed data length and the second decompressed data length, it is determined that the second decompressed data meets a preset complete data packet condition, then the second decompressed data is determined as the target data packet. As can be seen from the above, this application queries the preset index table of the ZSTD compressed file to obtain the retrieval information of the target data packet, and determines the first decompressed data length to be used to obtain the header data of the target data packet. Then, it estimates the length of the first compressed frame to be read for the first time by combining the retrieval information and the first decompressed data length. The first compressed data is then read from the ZSTD compressed file, and after decompression, the first decompressed data and the first actual decompressed data length are obtained. If the first decompressed data meets the preset complete header condition based on the first decompressed data length and the first actual decompressed data length, the first decompressed data is designated as the header data of the target data packet. Subsequently, the second decompressed data length to be used to obtain the target data packet is determined based on the header data and the retrieval information, and the length of the second compressed frame to be read is estimated. The second compressed data is then read from the ZSTD compressed file and decompressed to obtain the second decompressed data and the second actual decompressed data length. If the second decompressed data meets the preset complete data packet condition based on the second actual decompressed data length and the second decompressed data length, the second decompressed data is designated as the target data packet.In this way, through the process described above in this application, based on the index table for precise positioning, staged decompression, and conditional append reading, the relevant compressed data of the target data packet can be quickly located by relying on the index table. The integrity of the decompressed data is ensured by verifying the target length and the actual length twice, avoiding the loss of data packets due to the compressed frame structure or data offset. This improves the efficiency and reliability of accurately extracting the target data packet from the ZSTD compressed file, ensures the integrity and efficiency of data reading, and further reduces the IO overhead and IO count in the ZSTD data packet retrieval process to improve data packet retrieval efficiency while ensuring data integrity. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0051] Figure 1 This is a flowchart of a ZSTD data packet retrieval method disclosed in this application;

[0052] Figure 2 This is a flowchart and timing diagram of a ZSTD data packet retrieval method disclosed in this application;

[0053] Figure 3 This is a timing diagram illustrating the process of extracting packet header data from a data packet, as disclosed in this application.

[0054] Figure 4 This is a schematic diagram of the process for extracting target data packets as disclosed in this application;

[0055] Figure 5 This is a schematic diagram of the structure of a ZSTD data packet retrieval device disclosed in this application;

[0056] Figure 6 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] When retrieving specific data packets from a ZSTD compressed file, the common approach is to pre-set an initial read length based on experience, combined with dynamic appending of block information from streaming decompression. However, current technology requires parsing the ZSTD compressed file frame by frame to obtain the compressed frame offset. This process involves traversing the header information of multiple compressed frames, which is time-consuming and increases pre-retrieval overhead. Furthermore, the initial read length is based on experience, which may result in reading unnecessary compressed data, requiring disk I / O and consuming memory. This leads to a high proportion of invalid I / O, increasing I / O overhead. When decompressed data is incomplete, appending with a fixed block length (e.g., 64KB) increases the number of I / O operations, significantly prolonging the retrieval time.

[0059] To overcome the aforementioned technical problems, this application provides a ZSTD packet retrieval method that can reduce IO overhead and IO count during the ZSTD packet retrieval process while ensuring data integrity, thereby improving packet retrieval efficiency.

[0060] See Figure 1 As shown, this embodiment of the invention discloses a ZSTD data packet retrieval method, including:

[0061] Step S11: Query the preset index table of the ZSTD compressed file to obtain the retrieval information of the target data packet, determine the first decompressed data length to be used to obtain the header data of the target data packet, and estimate the length of the first compressed frame to be read for the first time based on the retrieval information and the first decompressed data length, so as to read the first compressed data from the ZSTD compressed file, decompress to obtain the first decompressed data and the first actual decompressed data length.

[0062] In this embodiment, the retrieval information of the target data packet is obtained by querying a preset index table of the ZSTD compressed file. Based on the retrieval information, the first decompressed data length for obtaining the header data of the target data packet is determined. Then, the length of the first compressed frame to be read is estimated by combining the retrieval information and the first decompressed data length. The first compressed data is read from the ZSTD compressed file, decompressed, and the first decompressed data and the corresponding first actual decompressed data length are obtained. The preset index table refers to a structured data table created during ZSTD compression, used to store key parameters such as compressed frame offset, intra-frame offset, and file compression ratio, and associated with the data packet header length and data packet length. It allows for quick querying and retrieval, supporting the estimation of the compressed frame read length. Figure 2 The diagram shown is a flowchart of a ZSTD data packet retrieval method provided in this application.

[0063] It should be noted that before querying the preset index table of ZSTD compressed files, the preset index table needs to be constructed synchronously during the compression process. The process is as follows: The file to be compressed is compressed using the ZSTD algorithm to obtain a ZSTD compressed file containing several ZSTD compressed frames. During the generation of these ZSTD compressed frames, the corresponding compression frame offset, intra-frame offset, and file compression ratio of each ZSTD compressed frame are calculated and stored in the preset index table. The compression frame offset is the offset of the starting position of the pre-compression header of the compressed frame relative to the starting address of the ZSTD compressed file; the intra-frame offset is the offset of the starting position of the pre-compression header of the target data packet relative to the corresponding starting address of the compressed frame. In other words, the ZSTD algorithm is used to compress the file to be compressed, generating a ZSTD compressed file containing several ZSTD compressed frames. During the generation of these ZSTD compressed frames (stored as independent compressed files), the compression frame offset O_f, intra-frame offset O_i, and file compression ratio R corresponding to each ZSTD compressed frame are calculated synchronously and stored in the preset index table. Wherein, the compressed frame offset is the offset of the starting position of the pre-compression packet header of the compressed frame relative to the starting address of the ZSTD compressed file; the intra-frame offset is the offset of the starting position of the pre-compression packet header of the target data packet relative to the starting address of the corresponding compressed frame; the file compression ratio is the ratio of the total length of the file before compression to the total length of the file after compression. It can be understood that this embodiment can also record the frame-level compression ratio R_block (the length of a single compressed frame before compression divided by the length after compression) of each compressed block during compression and store it in the preset index table. During retrieval, the read length is estimated based on the R_block of the compressed frame to which the required data packet belongs. In addition, the historical mapping relationship of T1-C1 (first data length to be decrypted - first compressed frame length) and T2-C2 (second data length to be decrypted - second compressed frame length) of the same type of compressed files can be associated and stored in the preset index table. During a new retrieval, historical data is preferentially called to determine the read length; if no historical data is available, the estimated length method of this application is used.

[0064] It should be further pointed out that the processing flow for reading the first compressed data from the ZSTD compressed file, decompressing it to obtain the first decompressed data and the first actual decompressed data length is as follows: The sum of the intra-frame offset and the actual header length recorded in the retrieval-related information is determined as the first decompressed data length to be used to obtain the header data of the target data packet; the quotient of the first decompressed data length and the file compression ratio recorded in the retrieval-related information is determined, and the quotient is rounded up to determine the result as the first compressed frame length to be read for the first time; based on the first compressed frame length and the compressed frame offset in the retrieval-related information, the corresponding first read address range is determined, so as to read the first compressed data from the ZSTD compressed file according to the first read address range; the first compressed data is input to the target streaming decompression interface to obtain the corresponding first decompressed data and the first actual decompressed data length. That is, the intra-frame offset and the actual header length L_h recorded in the retrieval-related information are determined, and the sum of the intra-frame offset and the actual header length is determined as the first decompressed data length T1 for obtaining the header data of the target data packet. This ensures that decompression to this length can contain the complete header. The length of the first compressed frame C1 read for the first time is estimated based on the file compression ratio. Specifically, the quotient of the first decompressed data length and the file compression ratio in the retrieval-related information is calculated and rounded up. The result is set as the length of the first compressed frame read for the first time. For example, if T1 = 400 bytes and R = 10:1, then C1 = 40 bytes; if T1 = 405 bytes... If R=10:1, then C1=41 bytes. This avoids insufficient reading of compressed data due to the decimal part and solves the problem of blindly reading the initial length. Then, combined with the length of the first compressed frame and the compressed frame offset in the retrieved information, the first reading address range from O_f to O_f+C1–1 is determined. The corresponding first compressed data is read from the ZSTD compressed file. For example, if O_f=8000 bytes and C1=40 bytes, 8000-8039 bytes of the first compressed data are read and input into the target streaming decompression interface, that is, the ZSTD streaming decompression interface, to obtain the first decompressed data and the first actual decompressed data length A1 in real time. In this way, this embodiment relies on a preset index table to accurately obtain retrieval information, and combines the estimated header length to locate and read compressed data. This allows for the rapid locking of the header-related compressed data of the target data packet, ensuring a high degree of match between the compressed data read initially and the target header. This avoids redundant reading of invalid data and improves the efficiency of retrieving and decompressing the header data of the target data packet. The simultaneous generation and storage of compression and index information provides accurate location information for subsequent retrieval of the target data packet from the ZSTD compressed file, avoiding blind data reading during decompression, significantly improving the retrieval efficiency of the target data packet, and ensuring the accuracy and convenience of data reading.

[0065] Step S12: If, based on the first decompressed data length to be decompressed and the first actual decompressed data length, it is determined that the first decompressed data meets the preset complete header condition, then the first decompressed data is determined as the header data of the target data packet.

[0066] In this embodiment, when the first decompressed data meets the preset complete header condition based on the first expected decompressed data length and the first actual decompressed data length, the first decompressed data is determined as the header data of the target data packet. Figure 3 The diagram shown is a timing illustration of a process for extracting packet header data provided in this application.

[0067] Specifically, the first actual decompressed data length is compared with the first expected decompressed data length to obtain a corresponding first comparison result. If the first comparison result indicates that the first actual decompressed data length is not less than the first expected decompressed data length, then the first decompressed data is determined to meet the preset complete header condition, and the first decompressed data is identified as the header data of the target data packet. If the first comparison result indicates that the first actual decompressed data length is less than the first expected decompressed data length, then the current first decompressed data does not meet the preset complete header condition. The first undecompressed length of the current data block is obtained through the target streaming decompression interface, and the first undecompressed length is determined to be equal to the data block header length. The first sum of the lengths is determined; the first difference between the first decompressed data length and the first actual decompressed data length is determined, and the quotient between the first difference and the file compression ratio is determined as the first theoretical append length; the maximum value between the first sum and the first theoretical append length is determined as the length of the third compressed frame to be appended, and the third compressed data of the third compressed frame length is appended from the current end address of the ZSTD compressed file; the third compressed data is decompressed through the target streaming decompression interface to update the first decompressed data and the first actual decompressed data length, and the process jumps to the step of comparing the first actual decompressed data length with the first decompressed data length. That is, the first actual decompressed data length is compared with the first expected decompressed data length, i.e., A1 and T1 are compared. If the first actual decompressed data length is not less than the first expected decompressed data length, it means that the decompressed data has a complete header, and the first decompressed data is determined to meet the preset complete header condition and is designated as the header data of the target data packet. If the first actual decompressed data length is smaller, it is determined that the preset complete header condition is not met, and the first undecompressed length S of the current data block is obtained through the target streaming decompression interface and its first sum with the data block header length H (fixed length of 3) is calculated. The value is then calculated, and the first difference between the first decompressed data length and the first actual decompressed data length is calculated. Combined with the file compression ratio, the first theoretical append length is estimated by rounding up. The maximum value of the first sum and the first theoretical append length is taken as the third compressed frame length C_add1. The corresponding length of the third compressed data is appended from the current end address of the ZSTD compressed file, that is, O_f+C1. After decompression by the target streaming decompression interface, the first decompressed data and the first actual decompressed data length are updated. Then, the comparison step is returned and repeated until A1 is not less than T1.For example, the preset index table query yields O_f=8000 bytes, O_i=200 bytes, L_h=200 bytes, R=10:1, calculates T1=400 bytes, C1=40 bytes; reads 8000-8039 bytes of compressed data, after decompression A1=380 bytes (<400 bytes); streaming decompression returns S=2 bytes, H=3 bytes, theoretical length: (400-380) / (10 / 1)=2, 5>2, so C_add1=5 bytes; appends 8040-8044 bytes of compressed data, after decompression adds 25 bytes, A1=405 bytes (≥400 bytes), successfully obtaining the complete packet header. In this way, this embodiment, with length comparison as its core, dynamically calculates the appended reading length and performs cyclic verification, which can flexibly deal with the situation of incomplete decompressed data. Through precise calculation, it ensures that the appended compressed data can make up for the missing part of the packet header, avoiding packet header extraction failure due to insufficient reading in a single time. This not only ensures the integrity of the target data packet header, but also reduces the reading and decompression of redundant data by reasonably calculating the appended length, thereby improving the efficiency and reliability of packet header extraction.

[0068] Step S13: Determine the second decompressed data length for obtaining the target data packet based on the packet header data and the retrieval related information, so as to estimate the length of the second compressed frame to be read in advance, and read the second compressed data in advance from the ZSTD compressed file. After decompression, the second decompressed data and the second actual decompressed data length are obtained.

[0069] In this embodiment, by combining the determined packet header data and the retrieval-related information, the second decompressed data length to be obtained for acquiring the target data packet is determined. Based on this length, the length of the second compressed frame to be additionally read is estimated. The second compressed data is additionally read from the ZSTD compressed file, and after decompression, the second decompressed data and the corresponding second actual decompressed data length are obtained.

[0070] It should be noted that the process for determining the second decompressed data length used to obtain the target data packet is as follows: The packet header data is read to obtain the data packet length of the target data packet; the sum of the data packet length and the intra-frame offset recorded in the retrieval-related information is determined as the second decompressed data length used to obtain the target data packet. That is, the determined packet header data is read to obtain the data packet length L_d of the target data packet, and the data packet length is added to the intra-frame offset recorded in the retrieval-related information. The result is the second decompressed data length T2 used to obtain the target data packet, ensuring that decompression to this length can contain a complete data packet.

[0071] It should be further pointed out that the processing flow for appending second compressed data to the ZSTD compressed file, obtaining the second decompressed data and the second actual decompressed data length after decompression is as follows: determine the second difference between the second decompressed data length and the first actual decompressed data length, and determine the quotient between the second difference and the file compression ratio in the retrieved relevant information as the second compressed frame length; append the second compressed data of the second compressed frame length from the current read end address of the ZSTD compressed file; decompress the second compressed data through the target streaming decompression interface to obtain the second decompressed data and the corresponding newly added decompressed length, and obtain the current decompressed second actual decompressed data length based on the first actual decompressed data length and the newly added decompressed length. That is, calculate the second difference between the second decompressed data length and the first actual decompressed data length to determine the additional decompression length. Divide this difference by the file compression ratio and round up. The result is set as the second compressed frame length C2. From the current end address of the ZSTD compressed file, for example, 8044 bytes, append the second compressed data of this length. After decompression through the target streaming decompression interface, the second decompressed data and the newly added decompressed length are obtained. Combine the first actual decompressed data length and the newly added decompressed length to obtain the current decompressed second actual decompressed data length A2. In this way, the estimated length calculated based on the actual data packet length in the header data and the intra-frame offset of the index record in this embodiment can make the estimated length of the second data to be decompressed more closely match the actual data structure, avoid reading redundancy or insufficiency caused by estimation deviation, and ensure the accuracy and efficiency of target data packet extraction. By accurately calculating the additional reading length through the difference and compression ratio and synchronously updating the actual decompression length, it can be ensured that the additional compressed data read exactly matches the missing part of the data packet body, avoiding redundant reading or insufficient reading, which not only improves the accuracy of data reading, but also ensures the continuity of the decompression process.

[0072] Step S14: If, based on the second actual decompressed data length and the second expected decompressed data length, it is determined that the second decompressed data meets the preset complete data packet condition, then the second decompressed data is determined as the target data packet.

[0073] In this embodiment, when the second decompressed data meets the preset complete data packet conditions based on the second actual decompressed data length and the second expected decompressed data length, the second decompressed data is determined as the target data packet. Figure 4 The diagram shown is a timing illustration of a process for extracting target data packets provided in this application.

[0074] Specifically, the second actual decompressed data length is compared with the second expected decompressed data length to obtain a corresponding second comparison result. If the second comparison result indicates that the second actual decompressed data length is not less than the second expected decompressed data length, then the second decompressed data is determined to meet the preset complete data packet condition, and the second decompressed data is identified as the target data packet. If the second comparison result indicates that the second actual decompressed data length is less than the second expected decompressed data length, then the current second decompressed data does not meet the preset complete data packet condition. The second undecompressed length of the current data block is obtained through the target streaming decompression interface, and the second undecompressed length and the data block header length are determined. The second sum value; determine the second difference between the second decompressed data length and the second actual decompressed data length, and determine the quotient between the second difference and the file compression ratio as the second theoretical append length; determine the maximum value between the second sum value and the second theoretical append length as the appended fourth compressed frame length, and append the fourth compressed data of the fourth compressed frame length from the current read end address of the ZSTD compressed file; decompress the fourth compressed data through the target streaming decompression interface to update the second decompressed data and the second actual decompressed data length, and jump to the step of comparing the second actual decompressed data length with the second decompressed data length. That is, the second actual decompressed data length is compared with the second expected decompressed data length, i.e., A2 and T2 are compared. If the second actual decompressed data length is not less than the second expected decompressed data length, it is determined that the second decompressed data meets the preset complete data packet condition, the complete data packet is extracted, it is designated as the target data packet, and the search ends. If the second actual decompressed data length is smaller, it is determined that the condition is not met. The second undecompressed length S of the current data block is obtained through the target streaming decompression interface, and the second sum of the current data block and the data block header length is calculated. Then, the second difference between the second expected decompressed data length and the second actual decompressed data length is calculated. Combined with the file compression ratio, the second theoretical append length is obtained by rounding up. The maximum value of the second sum and the second theoretical append length is taken as the fourth compressed frame length C_add2. The corresponding length of the fourth compressed data is appended from the current read end address of the ZSTD compressed file, i.e., O_f + C2. After decompression through the target streaming decompression interface, the second decompressed data and the second actual decompressed data length are updated. Then, the comparison step is returned and repeated until A2 is not less than T2.For example, by associating parameters through the preset index table, L_d = 1000 bytes, and T2 = 200 + 1000 = 1200 bytes is calculated; A1 = 405 bytes is determined, and the required decompression length is 795 bytes, C2 = 795 ÷ 10 = 79.5, which is rounded up to 80 bytes; 8042-8121 bytes of compressed data are appended, and after decompression, 800 bytes are added, A2 = 1205 bytes (≥ 1200 bytes), successfully obtaining the complete data packet. In this way, this embodiment uses length matching as the core for cyclical verification and dynamically calculates the appended length, which can flexibly supplement the missing parts of the data packet body, accurately lock the complete target data packet, and avoid data packet extraction failure due to insufficient reading in a single operation. At the same time, by accurately calculating and reducing the reading and decompression of redundant data, it not only ensures the integrity of the target data packet but also improves the efficiency and reliability of data extraction, making the entire extraction process more targeted and coherent.

[0075] As can be seen from the above, in this embodiment, the target data packet is retrieved by querying a preset index table of the ZSTD compressed file. Based on this, the first decompressed data length to be obtained for the header data of the target data packet is determined. Then, the length of the first compressed frame to be read for the first time is estimated by combining the retrieval information and the first decompressed data length. The first compressed data is then read from the ZSTD compressed file. After decompression, the first decompressed data and the first actual decompressed data length are obtained. If the first decompressed data meets the preset complete header condition based on the first decompressed data length and the first actual decompressed data length, the first decompressed data is designated as the header data of the target data packet. Subsequently, the second decompressed data length to be obtained for the target data packet is determined based on the header data and the retrieval information. The length of the second compressed frame to be read is estimated. The second compressed data is read from the ZSTD compressed file and decompressed to obtain the second decompressed data and the second actual decompressed data length. If the second decompressed data meets the preset complete data packet condition based on the second actual decompressed data length and the second decompressed data length, the second decompressed data is designated as the target data packet. In this way, through the above process of the embodiments of this application, based on the index table for accurate positioning, staged decompression and conditional append reading, the relevant compressed data of the target data packet can be quickly located by relying on the index table. The integrity of the decompressed data is ensured by verifying the target length and the actual length twice, avoiding the loss of data packets due to the compressed frame structure or data offset. This improves the efficiency and reliability of accurately extracting the target data packet from the ZSTD compressed file, ensures the integrity and efficiency of data reading, and further reduces the IO overhead and IO count in the ZSTD data packet retrieval process to improve the data packet retrieval efficiency while ensuring data integrity.

[0076] Accordingly, see Figure 5As shown in the illustration, this application also provides a ZSTD data packet retrieval device, comprising:

[0077] The data reading module 11 is used to query the preset index table of the ZSTD compressed file to obtain the retrieval information of the target data packet, so as to determine the first decompressed data length to be used to obtain the header data of the target data packet, and to estimate the first compressed frame length to be read for the first time based on the retrieval information and the first decompressed data length, so as to read the first compressed data from the ZSTD compressed file, decompress it to obtain the first decompressed data and the first actual decompressed data length after decompression;

[0078] The data determination module 12 is used to determine the first decompressed data as the header data of the target data packet if, based on the first decompressed data length to be decompressed and the first actual decompressed data length, it is determined that the first decompressed data satisfies the preset complete header condition.

[0079] The length determination module 13 is used to determine the second decompressed data length for obtaining the target data packet based on the packet header data and the retrieval related information, so as to estimate the length of the second compressed frame to be read, to read the second compressed data from the ZSTD compressed file, and to obtain the second decompressed data and the second actual decompressed data length after decompression.

[0080] The data packet determination module 14 is used to determine the second decompressed data as the target data packet if, based on the second actual decompressed data length and the second expected decompressed data length, it is determined that the second decompressed data satisfies the preset complete data packet condition.

[0081] In some specific embodiments, the ZSTD data packet retrieval device may further include:

[0082] The file compression unit is used to compress the file to be compressed using the ZSTD algorithm to obtain the ZSTD compressed file containing a number of ZSTD compressed frames.

[0083] The compression ratio calculation unit is used to calculate the corresponding compression frame offset, intra-frame offset, and file compression ratio of the ZSTD compressed frame during the generation of the plurality of ZSTD compressed frames, and store them in the preset index table;

[0084] Wherein, the compressed frame offset is the offset of the starting position of the pre-compression header of the compressed frame relative to the starting address of the ZSTD compressed file; the intra-frame offset is the offset of the starting position of the pre-compression header of the target data packet relative to the starting address of the corresponding compressed frame.

[0085] In some specific embodiments, the data reading module 11 may specifically include:

[0086] The first length determination unit is used to determine the sum of the intra-frame offset and the actual header length recorded in the retrieval related information as the first decipherable data length for obtaining the header data of the target data packet;

[0087] The rounding-up unit is used to determine the quotient of the first data length to be decrypted and the file compression ratio recorded in the retrieved information, and to round the quotient up to determine the result as the length of the first compressed frame read for the first time.

[0088] The range determination unit is used to determine a corresponding first read address range based on the first compressed frame length and the compressed frame offset in the retrieval related information, so as to read the first compressed data from the ZSTD compressed file according to the first read address range;

[0089] The data input unit is used to input the first compressed data into the target streaming decompression interface to obtain the corresponding first decompressed data and the length of the first actual decompressed data.

[0090] In some specific embodiments, the data determination module 12 may specifically include:

[0091] The first length comparison unit is used to compare the first actual solved data length with the first solved data length to obtain a corresponding first comparison result.

[0092] The first condition determination unit is used to determine that the first decompressed data satisfies the preset complete header condition if the first comparison result indicates that the length of the first actual decompressed data is not less than the length of the first data to be decompressed, so as to determine the first decompressed data as the header data of the target data packet.

[0093] The first sum determination unit is used to determine that if the first comparison result indicates that the length of the first actual decompressed data is less than the length of the first data to be decompressed, the current first decompressed data does not meet the preset complete packet header condition, obtain the first undecompressed length of the current data block through the target streaming decompression interface, and determine the first sum of the first undecompressed length and the length of the data block header.

[0094] The second length determination unit is used to determine the first difference between the first data length to be decoded and the first actual data length, and to determine the quotient between the first difference and the file compression ratio as the first theoretical append length.

[0095] The first data reading unit is used to determine the maximum value between the first sum and the first theoretical append length as the length of the third compressed frame to be appended and read the third compressed data of the third compressed frame length from the current end address of the ZSTD compressed file.

[0096] The data decompression unit is used to decompress the third compressed data through the target streaming decompression interface to update the length of the first decompressed data and the first actual decompressed data, and then jump to the step of comparing the length of the first actual decompressed data with the length of the first data to be decompressed.

[0097] In some specific embodiments, the length determination module 13 may specifically include:

[0098] A header data reading unit is used to read the header data to obtain the data packet length of the target data packet;

[0099] The third length determination unit is used to determine the sum of the data packet length and the intra-frame offset recorded in the retrieval related information as the second decrypted data length for obtaining the target data packet.

[0100] In some specific embodiments, the length determination module 13 may specifically include:

[0101] The fourth length determination unit is used to determine a second difference between the second data length to be decoded and the first actual data length, and to determine the quotient between the second difference and the file compression ratio in the retrieved related information as the second compressed frame length;

[0102] The second data reading unit is used to append second compressed data of the second compressed frame length from the current end address of the ZSTD compressed file.

[0103] The length acquisition unit is used to decompress the second compressed data through the target streaming decompression interface to obtain the second decompressed data and the corresponding newly added decompressed length, and to obtain the current decompressed second actual decompressed data length based on the first actual decompressed data length and the newly added decompressed length.

[0104] In some specific embodiments, the data packet determination module 14 may specifically include:

[0105] The second length comparison unit is used to compare the second actual solved data length with the second data length to be solved in order to obtain the corresponding second comparison result;

[0106] The second condition determination unit is used to determine that the second decompressed data meets the preset complete data packet condition if the second comparison result indicates that the length of the second actual decompressed data is not less than the length of the second data to be decompressed, so as to determine the second decompressed data as the target data packet;

[0107] The second sum determination unit is used to determine that if the second comparison result indicates that the actual decompressed data length is less than the data length to be decompressed, the current second decompressed data does not meet the preset complete data packet condition, obtain the second undecompressed length of the current data block through the target streaming decompression interface, and determine the second sum of the second undecompressed length and the data block header length.

[0108] The fifth length determination unit is used to determine a second difference between the second data length to be decoded and the second actual data length, and to determine the quotient between the second difference and the file compression ratio as the second theoretical append length;

[0109] The third data reading unit is used to determine the maximum value between the second sum and the second theoretical append length as the length of the fourth compressed frame to be appended and read from the current end address of the ZSTD compressed file, and append the fourth compressed data of the fourth compressed frame length.

[0110] The step jump unit is used to decompress the fourth compressed data through the target streaming decompression interface to update the second decompressed data and the second actual decompressed data length, and jump to the step of comparing the second actual decompressed data length with the second data length to be decompressed.

[0111] Furthermore, embodiments of this application also disclose an electronic device, Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the ZSTD data packet retrieval method disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0112] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0113] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0114] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the ZSTD packet retrieval method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0115] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned disclosed ZSTD data packet retrieval method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0116] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0117] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0118] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0119] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0120] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A ZSTD data packet retrieval method, characterized in that, include: The preset index table of the ZSTD compressed file is queried to obtain the retrieval information of the target data packet, so as to determine the first decompressed data length to be used to obtain the header data of the target data packet. Based on the retrieval information and the first decompressed data length, the length of the first compressed frame to be read for the first time is estimated, so as to read the first compressed data from the ZSTD compressed file, decompress it to obtain the first decompressed data and the first actual decompressed data length after decompression. If, based on the first decompressed data length to be decompressed and the first actual decompressed data length, it is determined that the first decompressed data meets the preset complete header condition, then the first decompressed data is determined as the header data of the target data packet; The second decompressed data length to be obtained for acquiring the target data packet is determined based on the packet header data and the retrieval related information, so as to estimate the length of the second compressed frame to be read in advance. The second compressed data is read in advance from the ZSTD compressed file, and the second decompressed data and the second actual decompressed data length are obtained after decompression. If, based on the second actual decompressed data length and the second expected decompressed data length, it is determined that the second decompressed data meets the preset complete data packet condition, then the second decompressed data is identified as the target data packet.

2. The ZSTD data packet retrieval method according to claim 1, characterized in that, Before querying the preset index table of ZSTD compressed files, the following is also included: The ZSTD algorithm is used to compress the file to be compressed, so as to obtain the ZSTD compressed file containing several ZSTD compressed frames; During the generation of the aforementioned ZSTD compressed frames, the corresponding compressed frame offset, intra-frame offset, and file compression ratio of the ZSTD compressed frames are calculated and stored in the preset index table. Wherein, the compressed frame offset is the offset of the starting position of the pre-compression header of the compressed frame relative to the starting address of the ZSTD compressed file; the intra-frame offset is the offset of the starting position of the pre-compression header of the target data packet relative to the starting address of the corresponding compressed frame.

3. The ZSTD data packet retrieval method according to claim 1, characterized in that, The process of determining the first decompressed data length for obtaining the header data of the target data packet, and estimating the first compressed frame length for initial reading based on the retrieved relevant information and the first decompressed data length, to read the first compressed data from the ZSTD compressed file, and decompressing to obtain the first decompressed data and the first actual decompressed data length, includes: The sum of the intra-frame offset and the actual header length recorded in the retrieved information is determined as the first decipherable data length for obtaining the header data of the target data packet; Determine the quotient of the first data length to be decrypted and the file compression ratio recorded in the relevant retrieval information, and round the quotient up to determine the result as the length of the first compressed frame read for the first time. Based on the first compressed frame length and the compressed frame offset in the retrieved related information, a corresponding first read address range is determined so as to read the first compressed data from the ZSTD compressed file according to the first read address range; The first compressed data is input into the target streaming decompression interface to obtain the corresponding first decompressed data and the length of the first actual decompressed data.

4. The ZSTD data packet retrieval method according to claim 3, characterized in that, If, based on the first decompressed data length to be decompressed and the first actual decompressed data length, it is determined that the first decompressed data meets the preset complete header condition, then the first decompressed data is determined as the header data of the target data packet, including: The first actual solved data length is compared with the first data length to be solved to obtain the corresponding first comparison result; If the first comparison result indicates that the length of the first actual decompressed data is not less than the length of the first data to be decompressed, then the first decompressed data is determined to meet the preset complete header condition, so that the first decompressed data is determined as the header data of the target data packet; If the first comparison result indicates that the actual decompressed data length is less than the expected decompressed data length, then it is determined that the current first decompressed data does not meet the preset complete packet header condition. The first undecompressed length of the current data block is obtained through the target streaming decompression interface, and the first sum of the first undecompressed length and the data block header length is determined. Determine the first difference between the first data length to be decoded and the first actual data length, and determine the quotient between the first difference and the file compression ratio as the first theoretical append length; The maximum value between the first sum and the first theoretical append length is determined as the length of the third compressed frame to be appended and read, and the third compressed data of the third compressed frame length is appended and read from the current end address of the ZSTD compressed file. The third compressed data is decompressed through the target streaming decompression interface to update the length of the first decompressed data and the first actual decompressed data, and then the process jumps to the step of comparing the length of the first actual decompressed data with the length of the first data to be decompressed.

5. The ZSTD data packet retrieval method according to claim 1, characterized in that, The step of determining the second decoded data length for obtaining the target data packet based on the packet header data and the retrieval related information includes: Read the header data to obtain the length of the target data packet; The sum of the data packet length and the intra-frame offset recorded in the retrieval-related information is determined as the second decrypted data length used to obtain the target data packet.

6. The ZSTD data packet retrieval method according to any one of claims 1 to 5, characterized in that, The estimated length of the second compressed frame to be read, the second compressed data to be read from the ZSTD compressed file, and the second decompressed data and the length of the second actual decompressed data obtained after decompression include: Determine the second difference between the second data length to be decoded and the first actual data length, and determine the quotient between the second difference and the file compression ratio in the retrieved relevant information as the second compressed frame length; The second compressed data, the second compressed frame length, is appended to the current end address of the ZSTD compressed file. The second compressed data is decompressed through the target streaming decompression interface to obtain the second decompressed data and the corresponding newly added decompressed length, and the current decompressed second actual decompressed data length is obtained based on the first actual decompressed data length and the newly added decompressed length.

7. The ZSTD data packet retrieval method according to claim 6, characterized in that, If, based on the second actual decompressed data length and the second expected decompressed data length, it is determined that the second decompressed data meets the preset complete data packet condition, then the second decompressed data is identified as the target data packet, including: The second actual solved data length is compared with the second data length to be solved to obtain the corresponding second comparison result; If the second comparison result indicates that the actual decompressed data length is not less than the expected decompressed data length, then the second decompressed data is determined to meet the preset complete data packet condition, so that the second decompressed data is identified as the target data packet; If the second comparison result indicates that the actual decompressed data length is less than the expected decompressed data length, then it is determined that the current second decompressed data does not meet the preset complete data packet condition. The second undecompressed length of the current data block is obtained through the target streaming decompression interface, and the second sum of the second undecompressed length and the data block header length is determined. Determine the second difference between the second expected data length and the second actual data length, and determine the quotient between the second difference and the file compression ratio as the second theoretical append length; The maximum value between the second sum and the second theoretical append length is determined as the length of the fourth compressed frame to be appended and read, and the fourth compressed data of the fourth compressed frame length is appended and read from the current end address of the ZSTD compressed file. The fourth compressed data is decompressed through the target streaming decompression interface to update the second decompressed data and the second actual decompressed data length, and then the process jumps to the step of comparing the second actual decompressed data length with the second data length to be decompressed.

8. A ZSTD data packet retrieval device, characterized in that, include: The data reading module is used to query the preset index table of the ZSTD compressed file to obtain the retrieval information of the target data packet, so as to determine the first decompressed data length to be used to obtain the header data of the target data packet, and to estimate the length of the first compressed frame to be read for the first time based on the retrieval information and the first decompressed data length, so as to read the first compressed data from the ZSTD compressed file, decompress it to obtain the first decompressed data and the first actual decompressed data length after decompression; The data determination module is used to determine the first decompressed data as the header data of the target data packet if, based on the first decompressed data length to be decompressed and the first actual decompressed data length, it is determined that the first decompressed data satisfies the preset complete header condition. The length determination module is used to determine the second decompressed data length for obtaining the target data packet based on the packet header data and the retrieval related information, so as to estimate the length of the second compressed frame to be read, to read the second compressed data from the ZSTD compressed file, and to obtain the second decompressed data and the second actual decompressed data length after decompression. The data packet determination module is used to determine the second decompressed data as the target data packet if, based on the second actual decompressed data length and the second expected decompressed data length, it is determined that the second decompressed data satisfies the preset complete data packet condition.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the ZSTD data packet retrieval method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the ZSTD data packet retrieval method as described in any one of claims 1 to 7.

Images