Method, device and medium for decompressing stream data of compressed file

Abstract

The application relates to the technical field of application development, and provides a compressed file stream data decompression method, which firstly adopts a preset decompression plug-in to perform decompression processing on batch file stream data to be decompressed; if an abnormal response occurs in the decompression processing, whether the file stream data to be decompressed is damaged is judged; if the file stream data to be decompressed is not damaged, the file stream data to be decompressed is read into the memory to form a file input stream; according to sub-file information in the file input stream, a target position of a byte where a detailed entry of the file stream data to be decompressed is located is positioned; the detailed entry in the byte is read according to the target position, and a to-be-pulled position is acquired according to the detailed entry; target data is pulled in a decompressed file package according to the to-be-pulled position, and the target data is stored in a third folder to complete stream data decompression; thus, the offset value of each sub-file data in the compressed file is actively corrected, and the starting value of the internal sub-file of the data stream is accurately positioned.

Description

Technical Field

[0001] This invention relates to the field of data processing, and more particularly to a method, apparatus, electronic device, and computer-readable storage medium for decompressing streaming data of compressed files. Background Technology

[0002] Everyday .zip compressed files can be decompressed using common tools like 7-Zip and WinRAR, producing decompressed files for human reading or use. However, in system applications, it's necessary to decompress compressed files in batches, and then perform tasks such as identification, parsing, annotation, and organization of the decompressed file content to complete the file operations in one go. To solve this problem, the open-source software project organization Apache created a Java library package called commons-compress for compressing and decompressing files. This library can handle files in ar, cpio, Unix dump, tar, zip, gzip, XZ, Pack200, and bzip2 formats, directly extracting internal subfile information from file stream data, and its functionality is quite powerful. The latest version of the commons-compress package on the Apache website is Commons Compress 1.21, released on July 31, 2021.

[0003] In project work, the Commons Compress 1.21 package from this library is used for decompressing compressed file streams. While it decompresses the vast majority of file streams successfully, a small percentage of compressed files exhibit a `java.util.zip.ZipException: Unexpected record signature:` exception, rendering the files unusable after decompression. Compressible files can be opened and decompressed normally using common tools like 7-Zip and WinRAR, proving that the compressed files are not corrupted. Currently, there are no patches or documentation available on the Apache website for this exception, and manually decompressing each file one by one using 7-Zip and WinRAR is extremely time-consuming and laborious.

[0004] Therefore, there is an urgent need for a method that can identify and determine the files to be decompressed, extract normal sub-file information, be compatible with the original batch decompression function, verify the accuracy of sub-file data content offset, and perform batch fast decompression of abnormal files. Summary of the Invention

[0005] This invention provides a method for decompressing streaming data of compressed files to solve the problem that in existing project work, when decompressing compressed streaming data, most file streams can be decompressed normally, but when an abnormal response is issued, the file cannot be decompressed and read normally. The compressed file can be opened and decompressed normally using commonly used software tools such as 7Zip and WinRAR, proving that the compressed file has not been damaged. However, for this abnormal situation, directly using software tools such as 7Zip and WinRAR to decompress the files one by one is extremely labor-intensive.

[0006] To achieve the above objectives, the present invention provides a method for decompressing streaming data of compressed files, comprising:

[0007] A preset decompression plugin is used to decompress batches of file stream data to be decompressed. If an abnormal response occurs during the decompression process, it is determined whether the file stream data to be decompressed is corrupted. The file stream data to be decompressed is stored in a file package to be decompressed.

[0008] If the file stream data to be decompressed is not corrupted, then the file stream data to be decompressed is read into memory to form a file input stream;

[0009] Based on the sub-file information in the file input stream, locate the target byte position of the detailed entry of the file stream data to be decompressed;

[0010] Read the detailed entries in the bytes according to the target location, and obtain the location to be pulled according to the detailed entries;

[0011] According to the location to be pulled, the target data is pulled from the file package to be decompressed, and the target data is stored in the third folder to complete the decompression of the streaming data.

[0012] Optionally, when using a preset decompression plugin to decompress a batch of file stream data to be decompressed, the following steps are included:

[0013] The decompression plugin iterates through the file stream data to be decompressed one by one, and reads the decompressed data contained in the file stream data one by one.

[0014] If the decompressed data is corrupted or the sub-file information of the decompressed data does not match, the decompression plugin will issue an abnormal response. Optionally, if the file stream data to be decompressed is not corrupted, reading the file stream data to be decompressed into memory to form a file input stream includes:

[0015] If the file stream data to be decompressed is not damaged, the decompressed data in the file stream data to be decompressed is disassembled and judged to determine whether the sub-file information of the decompressed data matches.

[0016] If the sub-file information of the decompressed data does not match, the file stream data to be decompressed is stored in memory to form a file input stream.

[0017] Optionally, the step of disassembling and judging the decompressed data in the file stream data to determine whether the sub-file information of the decompressed data matches includes:

[0018] The decompressed data is categorized to form sub-files of each category, and the sub-file information of each sub-file is obtained; the sub-file information includes at least a data area, a directory record area, and a directory record tail area;

[0019] Based on the location information of the tail area of ​​the directory record, the record information in the directory record area is located, and based on the record information, suspicious data in the data area portion of the decompressed data is obtained;

[0020] Determine whether the parameters of the directory record area of ​​the suspicious data do not fully match the parameters of the data area, or whether there is a positioning offset difference; if they do not fully match or there is a positioning offset, it is determined that the sub-file information of the decompressed data does not match.

[0021] Optionally, locating the target byte position of the detailed entry of the file stream data to be decompressed based on the sub-file information in the file input stream includes:

[0022] The data area in the sub-file information is deeply divided by a preset data analysis program to form a local file header, file data and file description block;

[0023] Locate the file header identifier of the data to be decompressed in the local file header;

[0024] The target location of the detailed entry of the data to be decompressed is found based on the file header identifier.

[0025] Optionally, the step of reading detailed entries from the bytes according to the target location and obtaining the location to be retrieved according to the detailed entries includes...

[0026] The detailed entry is located based on the target location;

[0027] The detailed entries are read using a preset data entry reading plugin;

[0028] Traverse and read the detailed entries to obtain the detailed header of the detailed entries and the detailed data corresponding to the detailed header, forming corresponding data about the detailed header and the detailed data;

[0029] Based on the corresponding data, obtain the location of the data to be extracted from the file package to be extracted.

[0030] Optionally, the step of retrieving the target data from the file package to be unzipped according to the location to be retrieved, and storing the target data in a third folder, includes:

[0031] Based on the location to be retrieved, the file retrieval area is locked in the file package to be decompressed;

[0032] The target data is selected in the file retrieval area by the preset file retrieval plugin, the target data is read and copied, and the target data is stored in a pre-created third folder for decompression.

[0033] To address the above problems, the present invention also provides a streaming data decompression device for compressed files, the device comprising:

[0034] The conventional decompression unit is used to decompress batches of file stream data to be decompressed using a preset decompression plugin. If an abnormal response occurs during the decompression process, it is determined whether the file stream data to be decompressed is corrupted. The file stream data to be decompressed is stored in the file package to be decompressed.

[0035] The decompression diagnostic unit is used to read the file stream data to be decompressed into memory to form a file input stream if the file stream data to be decompressed is not damaged.

[0036] The detail positioning unit is used to locate the target position of the byte where the detail entry of the file stream data to be decompressed is located based on the sub-file information in the file input stream;

[0037] A location locking unit is used to read the detailed entries in the bytes according to the target location, and to obtain the location to be pulled according to the detailed entries;

[0038] The decompression and retrieval unit is used to retrieve target data from the file package to be decompressed according to the location to be retrieved, and store the target data in a third folder to complete the decompression of the streaming data.

[0039] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0040] Memory, storing at least one instruction; and

[0041] The processor executes the instructions stored in the memory to implement the steps in the above-described method for decompressing stream data of compressed files.

[0042] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the above-described method for decompressing streaming data of compressed files.

[0043] This invention first uses a preset decompression plugin to decompress a batch of file stream data to be decompressed. If an abnormal response occurs during the decompression process, it is determined whether the file stream data to be decompressed is corrupted. The file stream data to be decompressed is stored in a file package to be decompressed. If the file stream data to be decompressed is not corrupted, it is read into memory to form a file input stream. The target position of the byte containing the detailed entry of the file stream data to be decompressed is located based on the sub-file information in the file input stream. The detailed entry in the byte is read according to the target position, and the position to be retrieved is obtained based on the detailed entry. The system retrieves the target data from the file package to be decompressed based on the location to be retrieved, and stores the target data in a third folder to complete the decompression of the streaming data. In this way, by actively correcting the offset value of each sub-file in the compressed file, it accurately locates the starting value of the sub-file inside the data stream. It is compatible with the decompression anomaly caused by inaccurate sub-file data offsets during compressed file filling. This provides a stable and reliable method for batch decompressing compressed file data streams in memory in JAVA project application development, which can help enterprise applications quickly, efficiently, and in large quantities extract the content information of each sub-file in the compressed file. Attached Figure Description

[0044] Figure 1 This is a flowchart illustrating a method for decompressing stream data of compressed files according to an embodiment of the present invention.

[0045] Figure 2 This is a schematic diagram of a streaming data decompression device for compressed files provided in an embodiment of the present invention;

[0046] Figure 3 A schematic diagram of the internal structure of an electronic device for a method of decompressing stream data of compressed files according to an embodiment of the present invention;

[0047] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0048] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0049] In existing projects, when decompressing compressed file streams, we encounter situations where most file streams can be decompressed normally, but when an abnormal response is issued, the file cannot be decompressed and read. However, the compressed file can be opened and decompressed normally using commonly used tools such as 7-Zip and WinRAR, proving that the compressed file has not been damaged. For this abnormal situation, directly using 7-Zip and WinRAR to decompress the files one by one is extremely labor-intensive.

[0050] To address the aforementioned problems, embodiments of the present invention provide a method for decompressing streaming data of compressed files.

[0051] In this embodiment, the execution entity is the streaming data decompression system of the compressed files of the entire server cluster. The streaming data decompression system of the compressed files is integrated in the server cluster. That is, different modules of the streaming data decompression system of the compressed files under the server cluster perform different operation steps. The architecture of the server cluster includes multiple servers, and multiple cluster instances run under each server. Multiple scheduled tasks are stored under each cluster instance. Thus, the orderly execution of the scheduled tasks is achieved through the following steps.

[0052] like Figure 1 As shown, in this embodiment, the method for decompressing stream data of compressed files includes:

[0053] S1: A preset decompression plugin is used to decompress a batch of file stream data to be decompressed. If an abnormal response occurs during the decompression process, it is determined whether the file stream data to be decompressed is corrupted; wherein, the file stream data to be decompressed is stored in a file package to be decompressed.

[0054] S2: If the file stream data to be decompressed is not corrupted, then the file stream data to be decompressed is read into memory to form a file input stream;

[0055] S3: Locate the target byte position of the detailed entry of the file stream data to be decompressed based on the sub-file information in the file input stream;

[0056] S4: Read the detailed entries in the bytes according to the target position, and obtain the position to be pulled according to the detailed entries;

[0057] S5: Retrieve the target data from the file package to be decompressed according to the location to be retrieved, and store the target data in the third folder to complete the decompression of the streaming data.

[0058] exist Figure 1In the illustrated embodiment, step S1 involves using a preset decompression plugin to decompress a batch of file stream data to be decompressed. If an abnormal response occurs during the decompression process, it is determined whether the file stream data to be decompressed is corrupted. The file stream data to be decompressed is stored in a file package to be decompressed. During this process, using the preset decompression plugin to decompress the batch of file stream data includes:

[0059] S11: The decompression plugin iterates through the file stream data to be decompressed one by one, and reads the decompressed data contained in the file stream data to be decompressed one by one;

[0060] S12: When the decompressed data is damaged or the sub-file information of the decompressed data does not match, the decompression plugin issues an abnormal response.

[0061] In this embodiment, the decompression plugin uses the Commons Compress 1.21 package; that is, the commons-compress Java library package created by the open-source software project organization Apache for compressing and decompressing files. This library can handle files in ar, cpio, Unix dump, tar, zip, gzip, XZ, Pack200, and bzip2 formats, directly extracting internal sub-file information from file stream data, and its functionality is quite powerful. The latest version of the commons-compress package on the Apache website is Commons Compress 1.21, released on July 31, 2021.

[0062] In project work, the Commons Compress 1.21 package of this library is used for decompressing compressed file stream data. While it can decompress most file stream data normally, a small number of compressed files exhibit a `java.util.zip.ZipException:Unexpected record signature:` exception, preventing them from being decompressed and read. Compressible files can be opened and decompressed normally using commonly used tools like 7-Zip and WinRAR, proving that the compressed files are not corrupted. Currently, there are no corresponding patches or explanations on the Apache website for this exception, and using common tools like 7-Zip and WinRAR for batch decompression would be a significant waste of resources. Therefore, to solve this problem, this application proposes step S2: first, determine the cause of the exception response, i.e., proceed to step S2: if the file stream data to be decompressed is not corrupted, then read the file stream data into memory to form a file input stream.

[0063] In step S2, if the file stream data to be decompressed is not corrupted, then reading the file stream data to be decompressed into memory to form a file input stream includes:

[0064] S21: If the file stream data to be decompressed is not damaged, the decompressed data in the file stream data to be decompressed is disassembled and judged to determine whether the sub-file information of the decompressed data matches.

[0065] S22: If the sub-file information of the decompressed data does not match, then the data of the file stream to be decompressed is stored in memory to form a file input stream; wherein,

[0066] The step of disassembling and judging the decompressed data in the file stream data to determine whether the sub-file information of the decompressed data matches includes:

[0067] S211: The decompressed data is categorized to form sub-files of each category, and the sub-file information of the sub-files is obtained; the sub-file information includes at least a data area, a directory record area, and a directory record tail area;

[0068] S212: Locate the record information in the directory record area based on the location information of the directory record tail area, and obtain suspicious data in the data area portion of the decompressed data based on the record information; that is, divide the decompressed data into a data area, a directory record area, and a directory record tail area. The application package first processes the information located in the directory record area based on the directory record tail area, and then finds the data area information of the compressed file based on the directory record area.

[0069] S213: Determine whether the parameters of the directory record area of ​​the suspicious data do not fully match the parameters of the data area, or whether there is a positioning offset difference; if they do not fully match or there is a positioning offset, then it is determined that the sub-file information of the decompressed data does not match.

[0070] Specifically, when viewing a compressed file containing n files, if an exception occurs when the application code relies on the CommonsCompress 1.21 package to read data from a specific file within the compressed file: Unexpected record signature: 0X8001404, then step S2 is required. If the compressed file can be opened normally using 7Zip or the preset detection plugin shows that the data of the file to be decompressed is normal, it means that the data of the file to be decompressed is not corrupted. Then, the data of the file to be decompressed is read into memory to form a file input stream.

[0071] In this embodiment, step S3 is the process of locating the target byte position of the detailed entry of the file stream data to be decompressed based on the sub-file information in the file input stream. This process of locating the target byte position of the detailed entry of the file stream data to be decompressed based on the sub-file information in the file input stream includes:

[0072] S31: The data area in the sub-file information is deeply divided by a preset data analysis program to form a local file header, file data and file description block;

[0073] S32: Locate the file header identifier of the data to be decompressed in the local file header;

[0074] S33: Locate the target position of the byte containing the detailed entry of the data to be decompressed based on the file header identifier;

[0075] When performing depth partitioning on the data area, it is divided into three main parts: local file header, file data, and file description. The local file header identifier can be something like 50 4B 03 04. The start identifier of the file description block can be something like 50 4B 07 08. Between the local file header and the file description block is the file data. In this way, the target position of the byte containing the detailed entry of the data to be decompressed can be found by comparing the number of the file header identifier with a known sequence of numbers.

[0076] Step S4 is the process of reading detailed entries from the bytes according to the target position and obtaining the position to be pulled based on the detailed entries. The process of reading detailed entries from the bytes according to the target position and obtaining the position to be pulled based on the detailed entries includes:

[0077] S41: Lock the detailed entry according to the target location; the detailed entry is the detailed storage information about the target data to be decompressed;

[0078] S42: Read the detailed entries through the preset data entry reading plugin; that is, obtain the detailed storage information of the target data to be decompressed;

[0079] S43: Traverse and read the detail entries to obtain the detail header of the detail entries and the detail data corresponding to the detail header; the detail header is usually the storage location, storage size, etc., and the detail data is the specific value corresponding to the detail header, such as the symbol of the storage location, the specific value of the storage size, etc.

[0080] S44: Based on the corresponding data, obtain the location of the data to be uncompressed in the file package to be uncompressed, that is, after obtaining information such as the storage location, the location of the target data to be uncompressed that can be pulled can be directly obtained.

[0081] Step S5 involves retrieving the target data from the file package to be unzipped according to the location to be retrieved, and storing the target data in a third folder to complete the process of stream data decompression; wherein, retrieving the target data from the file package to be unzipped according to the location to be retrieved and storing the target data in a third folder includes:

[0082] S51: Lock the file extraction area in the file package to be extracted based on the extraction location;

[0083] S52: Select the target data in the file retrieval area using the preset file retrieval plugin; that is, after locking the file retrieval area, you can directly select the target data in the retrieval area, read and copy the target data, and store the target data in a pre-created third folder for decompression; that is, after selecting the target data, you can directly move or copy the target data to the pre-created third folder.

[0084] Furthermore, this embodiment also includes a process for determining the data area of ​​the compressed file in the directory record area. When the data area information of the found compressed file matches the pre-obtained start identifier and the filename immediately following the start identifier is also the same, there is no need to search for other information; the data area corresponding to the start identifier can be directly retrieved to obtain the target data. Otherwise, the search will be repeated according to the nearest preceding and following identifier rules. After verification, the file data area information is obtained. This function is compatible with cases where the zip file is filled abnormally or the data is not matched, accurately finding the file data area information, thereby providing support for the smooth extraction of data from each file in the zip package using a file input stream.

[0085] In this embodiment, the server can be a standalone server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0086] As described above, the compressed file stream data decompression method provided in this embodiment of the invention first uses a preset decompression plugin to decompress a batch of file stream data to be decompressed. If an abnormal response occurs during the decompression process, it is determined whether the file stream data to be decompressed is corrupted. The file stream data to be decompressed is stored in a file package to be decompressed. If the file stream data to be decompressed is not corrupted, it is read into memory to form a file input stream. The target position of the byte containing the detailed entry of the file stream data to be decompressed is located based on the sub-file information in the file input stream. The detailed entry in the byte is read according to the target position, and then... The entry retrieves the location to be pulled; based on the location, it pulls the target data from the file package to be decompressed and stores the target data in a third folder to complete the decompression of the streaming data. In this way, by actively correcting the offset value of each sub-file data in the compressed file, it accurately locates the starting value of the sub-file inside the data stream, and is compatible with the decompression anomaly caused by inaccurate sub-file data offsets during compressed file filling. It provides a stable and reliable method for batch decompressing compressed file data streams in memory in JAVA project application development, which can help enterprise applications quickly, efficiently, and in large quantities extract the content information of each sub-file in the compressed file.

[0087] like Figure 2 As shown, this invention provides a streaming data decompression device 100 for compressed files, which can be installed in an electronic device. Depending on the functions implemented, the streaming data decompression device 100 may include a conventional decompression unit 101, a decompression diagnostic unit 102, a detailed positioning unit 103, a position locking unit 104, and a decompression retrieval unit 105. The module described in this invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0088] In this embodiment, the functions of each module / unit are as follows:

[0089] The conventional decompression unit 101 is used to decompress batches of file stream data to be decompressed using a preset decompression plugin. If an abnormal response occurs during the decompression process, it is determined whether the file stream data to be decompressed is corrupted. The file stream data to be decompressed is stored in the file package to be decompressed.

[0090] The decompression diagnostic unit 102 is used to read the file stream data to be decompressed into memory to form a file input stream if the file stream data to be decompressed is not damaged.

[0091] The detail positioning unit 103 is used to locate the target position of the byte of the detail entry of the file stream data to be decompressed based on the sub-file information in the file input stream;

[0092] The location locking unit 104 is used to read the detailed entries in the bytes according to the target location, and to obtain the location to be pulled according to the detailed entries;

[0093] The decompression and retrieval unit 105 is used to retrieve target data from the file package to be decompressed according to the location to be retrieved, and store the target data in a third folder to complete the decompression of the streaming data.

[0094] The conventional decompression unit 101 uses a preset decompression plugin to decompress batches of file stream data to be decompressed. If an abnormal response occurs during the decompression process, it is determined whether the file stream data to be decompressed is corrupted. During the process of storing the file stream data in the file package to be decompressed, the preset decompression plugin decompresses the batches of file stream data. If an abnormal response occurs during the decompression process, it includes:

[0095] S11: The decompression plugin iterates through the file stream data to be decompressed one by one, and reads the decompressed data contained in the file stream data to be decompressed one by one;

[0096] S12: When the decompressed data is damaged or the sub-file information of the decompressed data does not match, the decompression plugin issues an abnormal response.

[0097] If the file stream data to be decompressed is not corrupted, the decompression diagnostic unit 102 reads the file stream data to be decompressed into memory to form a file input stream. This process includes:

[0098] S21: If the file stream data to be decompressed is not damaged, the decompressed data in the file stream data to be decompressed is disassembled and judged to determine whether the sub-file information of the decompressed data is mismatched;

[0099] S22: If the sub-file information of the decompressed data does not match, then the file stream data to be decompressed is read into memory to form a file input stream; wherein,

[0100] The step of disassembling and judging the decompressed data in the file stream data to determine whether the sub-file information of the decompressed data does not match includes:

[0101] S211: The decompressed data is categorized to form sub-file information; the sub-file information includes at least a data area, a directory record area, and a directory record tail area;

[0102] S212: Locate the record information in the directory record area based on the location information of the directory record tail area, and obtain suspicious data in the data area portion of the decompressed data based on the record information; that is, divide the decompressed data into a data area, a directory record area, and a directory record tail area. The application package first processes the information located in the directory record area based on the directory record tail area, and then finds the data area information of the compressed file based on the directory record area.

[0103] S213: Determine whether the parameters of the directory record area of ​​the suspicious data do not fully match the parameters of the data area, or whether there is a positioning offset difference; if they do not fully match or there is a positioning offset, then it is determined that the sub-file information of the decompressed data does not match.

[0104] In the process of locating the target position of the byte of the detailed entry of the file stream data to be decompressed based on the sub-file information in the file input stream, the detailed positioning unit 103 locates the target position of the byte of the detailed entry of the file stream data to be decompressed based on the sub-file information in the file input stream. This process includes:

[0105] S31: The data area in the sub-file information is deeply divided by a preset data analysis program to form a local file header, file data and file description block;

[0106] S32: Locate the file header identifier of the data to be decompressed in the local file header;

[0107] S33: Locate the target position of the byte containing the detailed entry of the data to be decompressed based on the file header identifier;

[0108] The location locking unit 104 reads the detailed entries in the bytes according to the target location, and obtains the location to be pulled according to the detailed entries. This process includes:

[0109] S41: Lock the detailed entry according to the target location; the detailed entry is the detailed storage information about the target data to be decompressed;

[0110] S42: Read the detailed entries through the preset data entry reading plugin; that is, obtain the detailed storage information of the target data to be decompressed;

[0111] S43: Traverse the detail entries to form the corresponding data about the detail header and detail data; the detail header is usually the storage location, storage size, etc., and the detail data is the specific value corresponding to the detail header, such as the symbol of the storage location, the specific value of the storage size, etc.

[0112] S44: Based on the corresponding data, obtain the location of the data to be uncompressed in the file package to be uncompressed, that is, after obtaining information such as the storage location, the location of the target data to be uncompressed that can be pulled can be directly obtained.

[0113] The decompression and retrieval unit 105 retrieves target data from the file package to be decompressed according to the location to be retrieved, and stores the target data in a third folder to complete the process of stream data decompression. This includes:

[0114] S51: Lock the file extraction area in the file package to be extracted based on the extraction location;

[0115] S52: Select target data in the file fetch area; that is, after locking the file fetch area, you can directly select target data in the fetch area;

[0116] S53: Read and copy the target data, and store the target data in a pre-created third folder for decompression; that is, after selecting the target data, simply move or copy the target data to the pre-created third folder.

[0117] As described above, the compressed file stream data decompression device provided by the present invention first uses a conventional decompression unit 101 to decompress a batch of file stream data to be decompressed using a preset decompression plugin. If an abnormal response occurs during the decompression process, it is determined whether the file stream data to be decompressed is corrupted. The file stream data to be decompressed is stored in a file package to be decompressed. Then, if the file stream data to be decompressed is not corrupted, the decompression diagnostic unit 102 reads the file stream data into memory to form a file input stream. Next, the detail positioning unit 103 locates the target byte position of the detail entry of the file stream data to be decompressed based on the sub-file information in the file input stream. Finally, the position locking unit 104 reads the target byte position based on the target position. The system retrieves detailed entries from the byte array and obtains the pull position based on these entries. Finally, the decompression pull unit 104 pulls the target data from the file package to be decompressed according to the pull position and stores the target data in a third folder to complete the stream data decompression. In this way, by actively correcting the offset value of each sub-file data in the compressed file, the system accurately locates the starting value of the sub-file within the data stream. This approach is compatible with decompression anomalies caused by inaccurate sub-file data offsets during compressed file filling. It provides a stable and reliable method for batch decompressing compressed file data streams in memory in JAVA application development, enabling enterprise applications to quickly, efficiently, and in large quantities extract the content information of each sub-file in the compressed file.

[0118] like Figure 3 As shown, the present invention provides an electronic device 1 for a method of decompressing streaming data of compressed files.

[0119] The electronic device 1 may include a processor 10, a memory 11 and a bus, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a streaming data decompression program 12 for compressed files.

[0120] The memory 11 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device 1, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 11 can include both internal and external storage units of the electronic device 1. The memory 11 can be used not only to store application software and various types of data installed on the electronic device 1, such as application development code for decompressing compressed file stream data, but also to temporarily store data that has been output or will be output.

[0121] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., a stream data decompression program for compressed files) and calls data stored in the memory 11 to perform various functions of the electronic device 1 and process data.

[0122] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0123] Figure 3 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 3The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0124] For example, although not shown, the electronic device 1 may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0125] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device 1 and other electronic devices.

[0126] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device 1 and to display a visual user interface.

[0127] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0128] The stream data decompression program 12 of the compressed file stored in the memory 11 of the electronic device 1 is a combination of multiple instructions, which, when run in the processor 10, can achieve the following:

[0129] A preset decompression plugin is used to decompress batches of file stream data to be decompressed. If an abnormal response occurs during the decompression process, it is determined whether the file stream data to be decompressed is corrupted. The file stream data to be decompressed is stored in a file package to be decompressed.

[0130] If the file stream data to be decompressed is not corrupted, then the file stream data to be decompressed is read into memory to form a file input stream;

[0131] Based on the sub-file information in the file input stream, locate the target byte position of the detailed entry of the file stream data to be decompressed;

[0132] Read the detailed entries in the bytes according to the target location, and obtain the location to be pulled according to the detailed entries;

[0133] According to the location to be pulled, the target data is pulled from the file package to be decompressed, and the target data is stored in the third folder to complete the decompression of the streaming data.

[0134] It should be emphasized that, in order to further ensure the privacy and security of the decompression of the above compressed file's streaming data, the data of the decompressed streaming data of the above compressed file is stored in the node of the blockchain where this server cluster is located.

[0135] A server can be a standalone server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0136] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0137] This invention also provides a computer-readable storage medium, which may be non-volatile or volatile, and stores a computer program that is implemented when executed by a processor:

[0138] A preset decompression plugin is used to decompress batches of file stream data to be decompressed. If an abnormal response occurs during the decompression process, it is determined whether the file stream data to be decompressed is corrupted. The file stream data to be decompressed is stored in a file package to be decompressed.

[0139] If the file stream data to be decompressed is not corrupted, then the file stream data to be decompressed is read into memory to form a file input stream;

[0140] Based on the sub-file information in the file input stream, locate the target byte position of the detailed entry of the file stream data to be decompressed;

[0141] Read the detailed entries in the bytes according to the target location, and obtain the location to be pulled according to the detailed entries;

[0142] According to the location to be pulled, the target data is pulled from the file package to be decompressed, and the target data is stored in the third folder to complete the decompression of the streaming data.

[0143] Specifically, the specific implementation method of the computer program when executed by the processor can be referred to the description of the relevant steps in the embodiment of the stream data decompression method based on compressed files, which will not be repeated here.

[0144] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0145] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0146] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0147] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0148] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0149] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0150] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The term "second class" is used to indicate names and does not indicate any specific order.

[0151] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for decompressing streaming data of compressed files, characterized in that, include: A preset decompression plugin is used to decompress batches of file stream data to be decompressed. If an abnormal response occurs during the decompression process, it is determined whether the file stream data to be decompressed is corrupted. The file stream data to be decompressed is stored in a file package to be decompressed. If the file stream data to be decompressed is not corrupted, the file stream data to be decompressed is read into memory to form a file input stream; wherein, if the file stream data to be decompressed is not corrupted, the decompressed data in the file stream data to be decompressed is disassembled to determine whether the sub-file information of the decompressed data matches; The process for determining whether the sub-file information matches includes: classifying the decompressed data into categories, generating multiple category sub-files, and obtaining the sub-file information of the category sub-files; the sub-file information includes at least a data area, a directory record area, and a directory record tail area; locating the record information in the directory record area based on the position information of the directory record tail area, and obtaining suspicious data in the data area portion of the decompressed data based on the record information; determining whether the parameters of the directory record area of ​​the suspicious data do not completely match the parameters of the data area, or whether there is a positioning offset difference; if they do not completely match or have a positioning offset difference, then it is determined that the sub-file information of the decompressed data does not match. Based on the sub-file information in the file input stream, locate the target byte position of the detailed entry of the file stream data to be decompressed; Read the detailed entries in the bytes according to the target location, and obtain the location to be pulled according to the detailed entries; According to the location to be pulled, the target data is pulled from the file package to be decompressed, and the target data is stored in the third folder to complete the decompression of the streaming data.

2. The method for decompressing stream data of compressed files as described in claim 1, characterized in that, The process of decompressing a batch of file stream data using a preset decompression plugin includes: The decompression plugin iterates through the file stream data to be decompressed one by one, and reads the decompressed data contained in the file stream data one by one. If the decompressed data is corrupted or the sub-file information of the decompressed data does not match, the decompression plugin will issue an abnormal response.

3. The method for decompressing stream data of compressed files as described in claim 1, characterized in that, The step of locating the target byte position of the detailed entry of the file stream data to be decompressed based on the sub-file information in the file input stream includes: The data area in the sub-file information is deeply divided by a preset data analysis program to form a local file header, file data and file description block; Locate the file header identifier of the data to be decompressed in the local file header; The target location of the detailed entry of the data to be decompressed is found based on the file header identifier.

4. The method for decompressing stream data of compressed files as described in claim 3, characterized in that, The step of reading the detailed entries from the bytes according to the target location and obtaining the location to be fetched based on the detailed entries includes... The detailed entry is located based on the target location; The detailed entries are read using a preset data entry reading plugin; Traverse and read the detailed entries to obtain the detailed header of the detailed entries and the detailed data corresponding to the detailed header; Based on the corresponding data, obtain the location of the data to be extracted from the file package to be extracted.

5. The method for decompressing stream data of compressed files as described in claim 4, characterized in that, The step of retrieving target data from the file package to be unzipped according to the location to be retrieved, and storing the target data in a third folder, includes: Based on the location to be retrieved, the file retrieval area is locked in the file package to be decompressed; The target data is selected in the file retrieval area by the preset file retrieval plugin, the target data is read and copied, and the target data is stored in a pre-created third folder for decompression.

6. A device for decompressing streaming data of compressed files, characterized in that, The device includes: The conventional decompression unit is used to decompress batches of file stream data to be decompressed using a preset decompression plugin. If an abnormal response occurs during the decompression process, it is determined whether the file stream data to be decompressed is corrupted. The file stream data to be decompressed is stored in the file package to be decompressed. The decompression diagnostic unit is used to read the file stream data to be decompressed into memory to form a file input stream if the file stream data to be decompressed is not damaged; wherein, if the file stream data to be decompressed is not damaged, the decompressed data in the file stream data to be decompressed is disassembled to determine whether the sub-file information of the decompressed data matches. The process for determining whether the sub-file information matches includes: classifying the decompressed data into categories, generating multiple category sub-files, and obtaining the sub-file information of the category sub-files; the sub-file information includes at least a data area, a directory record area, and a directory record tail area; locating the record information in the directory record area based on the position information of the directory record tail area, and obtaining suspicious data in the data area portion of the decompressed data based on the record information; determining whether the parameters of the directory record area of ​​the suspicious data do not completely match the parameters of the data area, or whether there is a positioning offset difference; if they do not completely match or have a positioning offset difference, then it is determined that the sub-file information of the decompressed data does not match. The detail positioning unit is used to locate the target position of the byte where the detail entry of the file stream data to be decompressed is located based on the sub-file information in the file input stream; A location locking unit is used to read the detailed entries in the bytes according to the target location, and to obtain the location to be pulled according to the detailed entries; The decompression and retrieval unit is used to retrieve target data from the file package to be decompressed according to the location to be retrieved, and store the target data in a third folder to complete the decompression of the streaming data.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the steps in the streaming data decompression method for compressed files as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the method for decompressing stream data of compressed files as described in any one of claims 1 to 5.

Images