Compressed file processing method and device, equipment, storage medium and product

Abstract

The invention discloses a compressed file processing method and device, equipment, a storage medium and a product, and the method comprises the steps: firstly, responding to a directory preview instruction for a target compressed file, and downloading directory positioning data of the target compressed file from a storage end; analyzing the directory positioning data, and accurately determining the position information of the file directory structure in the target compressed file; and finally, obtaining the file directory structure from the storage end according to the position information obtained through analysis. Therefore, according to the embodiment of the invention, the position of the file directory structure is determined by firstly downloading a small amount of directory positioning data, and then the file directory structure is directionally acquired, so that the redundant operation of downloading a complete compressed file and executing full decompression in the prior art is avoided, the waiting time of a user is effectively shortened, and the user experience is improved. Meanwhile, the network load of the server side and the storage side is reduced, the waste of storage and computing resources of the server side is reduced, and the processing efficiency and the resource utilization rate of the online compressed file directory preview scene are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically, to a method, apparatus, device, storage medium, and product for processing compressed files. Background Technology

[0002] With the rapid development of cloud computing and online storage services, cloud storage platforms have become a core tool for users to manage files. Compressed files, with their advantages of efficiently integrating multiple files and saving storage space, are widely used in file transfer and archiving scenarios. When using online storage services to process such compressed files, users often need to preview the directory structure within the compressed package before deciding whether to download, decompress, or perform other targeted operations. This requirement has become a basic functional demand in online compressed file processing scenarios.

[0003] However, existing technologies have significant limitations in implementing this function: the entire compressed file must be downloaded from the cloud to the server, and then a full decompression operation must be performed on the entire compressed file before the file directory structure can be extracted and returned. This method not only significantly increases user waiting time when the compressed file is large or network bandwidth is limited, affecting the user experience; at the same time, the download process of the entire compressed file increases the network load between the server and the storage, and the decompression operation also consumes a large amount of storage and computing resources on the server, resulting in resource waste. Summary of the Invention

[0004] Based on this, the present invention provides a method, apparatus, device, storage medium and product for processing compressed files, in order to solve the defects of the prior art that when obtaining the file directory structure, the complete target compressed file needs to be downloaded from the storage end and fully decompressed, which leads to extended user waiting time, increased network load on the server and storage end, and waste of server storage and computing resources.

[0005] To achieve the above objectives, embodiments of the present invention provide a method for processing compressed files, including: In response to a directory preview command for a target compressed file, the directory location data of the target compressed file is downloaded from the storage device; Parse the directory location data to obtain the location information of the file directory structure in the target compressed file; The file directory structure is obtained from the storage terminal based on the location information.

[0006] To achieve the above objectives, embodiments of the present invention also provide a compressed file processing apparatus, comprising: The location data download module is used to download the directory location data of the target compressed file from the storage terminal in response to the directory preview command of the target compressed file; The location information acquisition module is used to parse the directory location data to obtain the location information of the file directory structure in the target compressed file; The directory acquisition module is used to obtain the file directory structure from the storage terminal based on the location information.

[0007] To achieve the above objectives, embodiments of the present invention also provide a compressed file processing device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the compressed file processing method as described in any of the above embodiments.

[0008] To achieve the above objectives, embodiments of the present invention also provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the compressed file processing method as described in any of the above embodiments.

[0009] To achieve the above objectives, embodiments of the present invention also provide a computer program product, including a computer program / instructions, which, when executed by a processor, implement the compressed file processing method as described in any of the above embodiments.

[0010] Compared with existing technologies, the compressed file processing method, apparatus, device, storage medium, and product disclosed in this invention first respond to a directory preview command for a target compressed file by downloading directory location data of the target compressed file from the storage end; then, the directory location data is parsed to accurately determine the location information of the file directory structure within the target compressed file; finally, the file directory structure is obtained from the storage end based on the parsed location information. Therefore, this invention, by first downloading a small amount of directory location data to determine the location of the file directory structure and then selectively obtaining the file directory structure, avoids the redundant operation of downloading the entire compressed file and performing full decompression as required by existing technologies. This effectively shortens user waiting time, reduces network load on the server and storage ends, minimizes waste of server storage and computing resources, and significantly improves processing efficiency and resource utilization in online compressed file directory preview scenarios. Attached Figure Description

[0011] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1This is a flowchart illustrating a compressed file processing method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a compressed file processing device provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a compressed file processing device provided in an embodiment of the present invention. Detailed Implementation

[0013] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0014] See Figure 1 , Figure 1 This is a flowchart illustrating a compressed file processing method provided in an embodiment of the present invention, the compressed file processing method including steps S11 to S13: S11. In response to a directory preview command for the target compressed file, download the directory location data of the target compressed file from the storage end; S12. Parse the directory location data to obtain the location information of the file directory structure in the target compressed file; S13. Obtain the file directory structure from the storage terminal based on the location information.

[0015] It's worth noting that the method is executed by the server, and the storage can be a cloud storage platform, such as a cloud drive. The directory preview command can be triggered by real-time user actions or pre-set on the server for scheduled triggering; the specific triggering method depends on the actual situation and is not limited here. The compressed file format can be ZIP, RAR, or 7Z, etc., and is not limited here. The compressed file structure includes a data storage area, a directory area, and a directory location area. Assuming the compressed file is a ZIP archive, the data storage area is the first part of the ZIP archive and constitutes the main content, including the local file header, file data, and data descriptors. The central directory area (i.e., the directory area of ​​the ZIP archive) is located after the data storage area and serves as the index center of the ZIP archive, summarizing the metadata of all files. This information allows for quick location and reading of the data storage area's contents, recording the location and attributes of all files within the ZIP archive. The directory location area is the last part of the ZIP archive, storing key information from the central directory area, which allows for quick location of the central directory area.

[0016] For example, the basic process of the method is as follows: 1. Download a 1-megabyte stream (i.e., directory location data) from the end of the compressed file from the storage terminal; this byte stream overwrites the data in the directory location area. 2. Parse the byte stream to determine the location information of the directory area. 3. Use this location information to locate the directory area of ​​the compressed file on the storage terminal, download the directory area data, and parse the target area data to obtain the file directory structure. It can be understood that after obtaining the file directory structure, the server returns this information to the business caller so that the business caller can view the directory information of the compressed file. Preferably, the method is initiated by the business caller, who must obtain an authorization token before the method can be executed to trigger the directory preview instruction. An authorization token is a credential used in a computer system to verify the legitimacy and operating permissions of the requesting business caller.

[0017] Optionally, for encrypted compressed files, only the data storage area is encrypted, while the directory area and directory location area are not encrypted. The method can obtain the file directory structure of the compressed file without performing a decryption operation.

[0018] Compared with existing technologies, the embodiments of the present invention determine the location of the file directory structure by first downloading a small amount of directory location data, and then obtain the file directory structure in a targeted manner. This avoids the redundant operation of downloading the complete compressed file and performing full decompression in existing technologies, effectively shortens the user's waiting time, reduces the network load on the server and storage, reduces the waste of server storage and computing resources, and significantly improves the processing efficiency and resource utilization of online compressed file directory preview scenarios.

[0019] In a preferred embodiment, based on steps S11-S13, step S12 specifically includes: Parse the directory location data to find the parsing anchor point; wherein the parsing anchor point is a preset fixed value; Based on the parsing anchor point, and according to the predefined field offset rules, the starting position and size of the directory area are found in the directory location data.

[0020] For example, taking a ZIP archive as an example, the directory location area is the End of Central Directory Record (EOCD), which records the following data: the EOCD signature (i.e., the parsing anchor, a fixed value used for reverse locating the EOCD structure), the current disk number, the disk number at the beginning of the central directory, the number of central directory entries recorded on that disk, the total number of entries in the central directory, the size of the central directory area (i.e., the directory area size of the ZIP archive), the offset of the beginning position of the central directory area (i.e., the offset of the beginning position of the directory area of ​​the ZIP archive), the length of the ZIP file comment, and the ZIP file comment, etc. The meanings of each data point in the EOCD are as follows: Central Directory End Marker Signature: Occupies 4 bytes, is a fixed value, and marks the beginning of the EOCD structure.

[0021] Current disk number: Occupies 2 bytes. In a multi-volume compression scenario, it identifies the central directory and the disk group number to which the current volume belongs.

[0022] The disk number of the central directory start position (i.e. the directory start position of the ZIP archive): occupies 2 bytes. In the case of multi-volume compression, it identifies which multi-volume disk the central directory area is located on.

[0023] Central directory entry count: occupies 2 bytes and records the number of central directory entries contained in the volume where the central directory area is located (each entry corresponds to the metadata of a file in the ZIP archive).

[0024] Total number of entries in the central directory: occupies 2 bytes and records the total number of entries in the central directory of the entire ZIP archive (i.e., the total number of files contained in the ZIP archive).

[0025] Central directory area size: occupies 4 bytes, records the total number of bytes in the central directory area, and is used to define the scope of the central directory area.

[0026] Central directory area start offset: occupies 4 bytes and records the starting position of the central directory area in its respective volume.

[0027] ZIP file comment length: Occupies 2 bytes, records the byte length of the ZIP file comment, and is used to define the scope of the comment information.

[0028] ZIP file comments: The length is determined by the "Comment Length" field (the size in bytes is variable), and it is user-defined file description information.

[0029] Based on the above description of the ZIP archive structure, it can be determined that the specific process of step S12 is as follows: First, locate the parsing anchor point from the directory location data; then, based on the pre-loaded compressed file format specification, utilize the field offset rules in that specification to find the starting position of the directory area (such as the disk number and offset of the starting position of the directory area) and the size of the directory area based on the parsing anchor point. It can be understood that if the target compressed file is not stored in multiple volumes, then it is only necessary to find the offset of the starting position of the directory area and the size of the directory area based on the parsing anchor point.

[0030] In a preferred embodiment, based on steps S11 to S13, step S11 specifically includes: In response to a directory preview command for a target compressed file, the file feature value of the target compressed file is obtained; The file feature value of the target compressed file is used as the search condition to query a preset data storage unit; wherein, the data storage unit pre-stores the file feature values ​​of processed compressed files; the processed compressed file refers to a compressed file whose file directory structure has been obtained; If the file feature value of the target compressed file is not found, download the directory location data of the target compressed file from the storage terminal. The file characteristic values ​​of the target compressed file are recorded in the data storage unit.

[0031] Furthermore, when the file feature value is retrieved from the data storage unit, the obtained file directory structure is read from the preset directory storage unit.

[0032] For example, the data storage unit is Elasticsearch, which builds a massive compressed file fingerprint database and stores information such as file feature values ​​of various compressed files whose file directory structures have been obtained. In response to a directory preview command for a target compressed file, the server interacts with the storage end to obtain the file feature value (such as MD5, SHA1, or SHA256) of the target compressed file. Then, it queries Elasticsearch to see if the file feature value exists. If it does, it means the server has already obtained the file directory structure of the target compressed file, so it only needs to find the file directory structure of the target compressed file from the storage service used to store the file directory structure. If the file feature value is not found, the server downloads the directory location data of the target compressed file from the storage end, finds the location information of the file directory structure within the target compressed file from this data, and downloads the original data corresponding to this location information from the storage end. After the security check of the target compressed file on the storage end passes, the downloaded original data is parsed to obtain the file directory structure, and the target file result is saved to the storage service. The file feature value is a fixed-length feature value generated by calculating hash / digest algorithms such as Message Digest Algorithm 5 (MD5), Secure Hash Algorithm 1 (SHA1), and Secure Hash Algorithm 256 (SHA256) on the target compressed file. Elasticsearch is a distributed, highly scalable, and real-time open-source full-text search engine and data analysis engine.

[0033] In a preferred embodiment, based on steps S11 to S13, step S13 specifically includes: when the file size of the target compressed file is less than or equal to a preset size threshold, and / or the file antivirus service scans the target compressed file and finds it safe, obtaining the file directory structure from the storage terminal based on the location information.

[0034] For example, before obtaining the file directory structure, the target compressed file needs to undergo security testing. Testing methods include: 1. Using file feature values ​​as search criteria, Elasticsearch searches a pre-built file security fingerprint database. If data matching the file feature values ​​of the target compressed file is found, file security-related information is returned directly without performing subsequent security checks.

[0035] 2. Obtain the number of files, directory levels, and total file size of the target compressed file. If any of these values ​​exceeds the corresponding preset size threshold, the target compressed file is determined to be an insecure compressed file.

[0036] 3. For target compressed files that pass the security test in item 2, a second scan by the file antivirus service is required. If the file antivirus service finds the target compressed file safe, the file security fingerprint database is updated based on the security test results of the target compressed file. The security test results should include at least the file feature value.

[0037] In a preferred embodiment, based on steps S11-S13, the method further includes: In response to a download command for the target compressed file, perform an operation corresponding to the download command.

[0038] It is worth noting that the download command can be triggered by the user or by the system periodically; no specific limitation is made here.

[0039] Optionally, the download command can instruct either to download only the target compressed file, or to decompress the compressed file after downloading it on the server. If the download command instructs to decompress the compressed file after downloading it on the server, then, to avoid duplicate downloads, after the download command is triggered, the system first checks if the target compressed file is stored in the local storage unit. If the target compressed file is not stored in the local storage unit, it is downloaded from the storage unit and decompressed to obtain a decompressed file. If the target compressed file is stored in the local storage unit, it is decompressed to obtain a decompressed file. Optionally, the decompressed file can be stored in a temporary directory on the server.

[0040] Furthermore, after decompressing the target compressed file, the decompressed file can be uploaded to the storage terminal for subsequent users to preview. Additionally, the server records the most recent usage time for each compressed file; if the time interval between the current time and the most recent usage time exceeds a preset threshold, the compressed file is deleted.

[0041] It is worth noting that the compressed file processing method can directly respond to the download command for the target compressed file without executing steps S11~S13. Optionally, it can directly download the target compressed file or directly download and decompress the target compressed file without checking whether it exists in the local storage unit.

[0042] In a preferred embodiment, based on steps S11-S13, the method further includes: In response to a download instruction for the target compressed file, the system checks whether a node corresponding to the target compressed file exists in the local linked list of the local storage unit. When the local linked list contains a node corresponding to the target compressed file, the node corresponding to the target compressed file is updated to the first endpoint of the local linked list; When the local linked list does not contain a node corresponding to the target compressed file, if the file size of the target compressed file is greater than the free capacity of the local storage unit, nodes are sequentially deleted from the second endpoint of the local linked list and the storage space occupied by the compressed file corresponding to the deleted node is released until the file size of the target compressed file is less than or equal to the free capacity of the local storage unit. Then, the target compressed file is downloaded and linked to the first endpoint of the local linked list.

[0043] Further, the target compressed file corresponding to the first endpoint of the local linked list is decompressed.

[0044] Furthermore, each node in the local linked list also records its recent usage time; the method further includes: deleting idle nodes from the local linked list and releasing the storage space occupied by the compressed file corresponding to the idle node; wherein, the idle node is a node whose idle time exceeds a preset time threshold.

[0045] Optionally, the local linked list can be a doubly linked list or a singly linked list. The local linked list consists of multiple nodes, each node corresponding to a compressed file.

[0046] For example, to improve file parsing and decompression efficiency, the server needs to temporarily store the target compressed file locally. Considering the limited server storage capacity, this invention adopts a hybrid caching strategy combining expiration time, storage capacity limits, and the Least Recently Used algorithm, and manages compressed files through a doubly linked list to make reasonable use of limited storage resources. The specific mechanism is as follows: 1. Linked list structure and core rules The local cache for compressed files is managed using a doubly linked list. Taking the first endpoint as the head of the list and the second endpoint as the tail as an example, the head of the list contains the most recently accessed node, and the tail contains the least recently accessed node. Each node records key information about the corresponding compressed file, including file characteristics, file size, last access time, information about the previous node, and information about the next node.

[0047] 2. File download and linked list update process When a download instruction for the target compressed file is received, the system first checks whether a node corresponding to the target compressed file exists in the local doubly linked list: (1) If a corresponding node exists: move the node to the head of the linked list and update its most recent usage time; (2) If the corresponding node does not exist: First, check if the free capacity of the local storage unit is sufficient to store the target compressed file. Let the total cache capacity be Ts, the currently used capacity be Cs, and the file size of the target compressed file be Fs: When Fs > Ts-Cs, delete nodes sequentially starting from the end of the linked list to release the storage space of the corresponding compressed file until the free capacity is greater than or equal to the file size of the target compressed file. Download the target compressed file, generate a new node for the target compressed file, and insert it at the head of the linked list; record the current time as the most recent usage time of the target compressed file.

[0048] 3. File Acquisition and Linked List Maintenance When retrieving a compressed file from local storage, the recent usage time of the node corresponding to the compressed file must be updated synchronously, and the node must be moved to the head of the linked list.

[0049] 4. Expired document cleanup mechanism Start a separate expiration cleanup thread to periodically traverse all nodes starting from the head of the linked list: calculate the idle time of each node; if the idle time exceeds the preset time threshold, delete the node and release the storage space of the corresponding compressed file to ensure that cache resources are not occupied by long-term idle compressed files.

[0050] 5. Decompression operation After completing the above caching process, decompress the target compressed file in the local storage unit.

[0051] This mechanism prioritizes the retention of recently used compressed files to improve access efficiency, while also preventing storage resource exhaustion through capacity control and expiration cleanup, thus achieving efficient utilization of limited space.

[0052] Optionally, the decompressed files can also be managed using the caching mechanism described above.

[0053] In a preferred embodiment, based on any of the above embodiments, the method further includes, in response to a download instruction for the target compressed file, performing multi-threaded segmented download of the target compressed file according to a preset segment size, and merging the segmented files after all segmented files have been downloaded to obtain the target compressed file.

[0054] Specifically, to improve the download efficiency of the target compressed file, a multi-threaded, segmented download method is used to acquire the target compressed file. The specific process consists of 5 core steps, with each step closely linked to ensure download integrity and efficiency: 1. After the download task starts, first obtain the file size of the target compressed file.

[0055] 2. The target compressed file is cut into multiple slices according to the preset slice size.

[0056] 3. Specify the download thread pool to obtain a single segment download thread, and start a separate download task for each segment file.

[0057] 4. The main download thread continuously listens to the message queue, collects download completion notifications for each file segment, and merges all file segments after the download of all file segments is complete.

[0058] In a preferred embodiment, based on any of the above embodiments, the method further includes: In response to a download command for the target compressed file, query the pre-built file-to-service instance mapping relationship; When no service instance corresponding to the target compressed file is found, the download command is forwarded to an idle service instance so that the idle service instance can respond to the download command and perform the corresponding operation. When a target service instance is found and the target service instance is alive, the download command is forwarded to the target service instance so that the target service instance can respond to the download command and perform the operation corresponding to the download command; wherein, the target service instance is the service instance corresponding to the target compressed file.

[0059] Optionally, the operation corresponding to the download instruction can be a file download operation, or a file download and decompression operation.

[0060] For example, taking file download and decompression as the operation corresponding to a download command, the correspondence between files and service instances is presented in the form of a file forwarding index. This index directs identical compressed files to the same service instance, avoiding duplicate downloads and improving overall decompression efficiency. The file forwarding index is an index table used to record the correspondence between compressed files and the service instances that process them. Its core information includes: the unique identifier of the compressed file (such as a combination of file characteristics, filename, and version number) and the corresponding service instance identifier. The specific process of file download and decompression is as follows: 1. Receive download instructions for the target compressed file.

[0061] 2. Query the file forwarding index: Based on the unique identifier of the target compressed file, query the index table to see if there is a corresponding service instance record.

[0062] (1) When no corresponding service instance record is found: Retrieve all available service instances from the registry center and filter out idle service instances; The download command is forwarded to the idle service instance, which then performs the file download and decompression operations. After the file download and decompression operations are completed, a new record is added to the file forwarding index, which is the unique identifier of the target compressed file and the corresponding service instance identifier for this processing.

[0063] (2) When the corresponding service instance record is found: Check whether the service instance (i.e. the target service instance) corresponding to the record is alive; If the service instance is alive, the download command will be forwarded directly to the service instance, which will then perform the file download and decompression operations. It can be understood that the service instance can reuse the already downloaded file and does not need to download the target compressed file again. If the service instance has been destroyed, delete the record in the file forwarding index and handle it as if "no corresponding service instance record was found".

[0064] In a preferred embodiment, based on any of the above embodiments, the method further includes: In response to an upload command for a decompressed file stored on the server, the file identifier of the decompressed file is calculated, and a pre-built file fingerprint database is queried based on the file identifier of the decompressed file; wherein, the file fingerprint database records the file identifier of the uploaded file and the cloud storage location of the uploaded file; If the file identifier of the decompressed file is found, the cloud storage location of the decompressed file is returned; If the file identifier of the decompressed file is not found, the decompressed file will be uploaded in multiple threads in segments.

[0065] It is worth noting that the upload command can be triggered by the user, by a system timer, or automatically after the compressed file is decompressed; no specific restrictions are imposed here.

[0066] For example, a file fingerprint database can be pre-built, which records the file identifier code and cloud storage location of uploaded files. The server downloads and decompresses the compressed file from the cloud storage platform, obtaining several decompressed files and generating a list of decompressed files. The business caller can preview, download, or save the decompressed files based on the list. For instance, the business caller sends an upload command to the server for the decompressed files, and the server responds by uploading the decompressed files to the cloud storage platform. Specifically, the file upload process is as follows: 1. Calculate the file identifier (such as file digest value) of the decompressed file.

[0067] 2. Match the calculated file identifier with the file fingerprint database: (1) If the file fingerprint database contains a file identifier corresponding to the decompressed file: there is no need to perform subsequent upload operations. The storage information (such as cloud storage location) corresponding to the file is directly extracted from the fingerprint database and returned to the business caller; (2) If the file fingerprint database does not contain a file identifier corresponding to the decompressed file, proceed to the next step.

[0068] 3. Compare the size of the decompressed file with the preset file chunk upload threshold. If the size of the decompressed file is less than or equal to the file chunk upload threshold, the decompressed file is uploaded directly. If the size of the decompressed file is greater than the file chunk upload threshold, the decompressed file is chunked according to the preset chunk size, and a separate file upload task is started for each chunk to be uploaded, so as to realize parallel file upload. After all the chunks to be uploaded are uploaded, the cloud storage platform will merge the files to obtain the decompressed file.

[0069] In a preferred embodiment, based on any of the above embodiments, the method further includes: obtaining task-related information in response to a task information query instruction.

[0070] Specifically, the task information query command can be triggered by user operation, by system timer, or automatically when various tasks are executed; no specific limitation is made here. Task-related information includes task status, total number of files after decompression, information on decompressed files, etc.

[0071] Compared with existing technologies, the compressed file processing method disclosed in this invention determines the location of the file directory structure by first downloading a small amount of directory location data, and then obtains the file directory structure in a targeted manner. This avoids the redundant operation of downloading the complete compressed file and performing full decompression in existing technologies, effectively shortens the user's waiting time, reduces the network load on the server and storage, reduces the waste of server storage and computing resources, and significantly improves the processing efficiency and resource utilization of online compressed file directory preview scenarios.

[0072] See Figure 2 This invention also provides a compressed file processing apparatus, comprising: Location data download module 21 is used to download the directory location data of the target compressed file from the storage end in response to the directory preview command of the target compressed file; Location information acquisition module 22 is used to parse the directory location data to obtain the location information of the file directory structure in the target compressed file; The directory acquisition module 23 is used to acquire the file directory structure from the storage terminal based on the location information.

[0073] In one embodiment, the location information includes the start position of the directory area and the size of the directory area; the location information acquisition module 22 is specifically used for: Parse the directory location data to find the parsing anchor point; wherein the parsing anchor point is a preset fixed value; Based on the parsing anchor point, and according to the predefined field offset rules, the starting position and size of the directory area are found in the directory location data.

[0074] In one embodiment, the location data download module 21 is specifically used for: In response to a directory preview command for a target compressed file, the file feature value of the target compressed file is obtained; The file feature values ​​are used as search criteria to query a preset data storage unit; wherein, the data storage unit pre-stores the file feature values ​​of processed compressed files; the processed compressed files refer to compressed files whose file directory structure has been obtained; If the file feature value is not found, download the directory location data of the target compressed file from the storage terminal; The file feature values ​​are recorded in the data storage unit.

[0075] In one embodiment, the directory acquisition module 23 is specifically used to: acquire the file directory structure from the storage terminal based on the location information when the file size of the target compressed file is less than or equal to a preset size threshold, and / or when the file antivirus service scans the target compressed file and finds it safe.

[0076] In one embodiment, the device further includes a download module for: In response to a download instruction for the target compressed file, the system checks whether a node corresponding to the target compressed file exists in the local linked list of the local storage unit. When the local linked list contains a node corresponding to the target compressed file, the node corresponding to the target compressed file is updated to the first endpoint of the local linked list; When the local linked list does not contain a node corresponding to the target compressed file, if the file size of the target compressed file is greater than the free capacity of the local storage unit, nodes are sequentially deleted from the second endpoint of the local linked list and the storage space occupied by the compressed file corresponding to the deleted node is released until the file size of the target compressed file is less than or equal to the free capacity of the local storage unit. Then, the target compressed file is downloaded and linked to the first endpoint of the local linked list.

[0077] In one implementation, the download module is further configured to: decompress the target compressed file corresponding to the first endpoint of the local linked list.

[0078] In one implementation, each node in the local linked list also records its recent usage time; the method further includes: deleting idle nodes from the local linked list and releasing the storage space occupied by the compressed file corresponding to the idle node; wherein the idle node is a node whose idle time exceeds a preset time threshold.

[0079] In one embodiment, the download module is further configured to: in response to a download instruction for the target compressed file, perform multi-threaded segmented download of the target compressed file according to a preset segment size, and merge the segmented files after all segmented files have been downloaded to obtain the target compressed file.

[0080] In one implementation, the download module is further configured to: In response to a download command for the target compressed file, query the pre-built file-to-service instance mapping relationship; When no service instance corresponding to the target compressed file is found, the download command is forwarded to an idle service instance so that the idle service instance can respond to the download command and perform the operation corresponding to the download command. When a target service instance is found and the target service instance is alive, the download command is forwarded to the target service instance so that the target service instance can respond to the download command and perform the operation corresponding to the download command; wherein, the target service instance is the service instance corresponding to the target compressed file.

[0081] In one embodiment, the device further includes a file upload module, used for: In response to an upload command for a decompressed file stored on the server, the file identifier of the decompressed file is calculated, and a pre-built file fingerprint database is queried based on the file identifier; wherein, the file fingerprint database records the file identifier of the uploaded file and the cloud storage location of the uploaded file; If the file identifier of the decompressed file is found, the cloud storage location of the decompressed file is returned; If the file identifier of the decompressed file is not found, the decompressed file will be uploaded in multiple threads in segments.

[0082] In one embodiment, the device further includes a query module for: obtaining task-related information in response to a task information query command.

[0083] It is worth noting that the working principle of the compressed file processing device provided in the above embodiments can be found in the workflow of the compressed file processing method provided in any of the above embodiments, and will not be repeated here.

[0084] Compared with existing technologies, the compressed file processing device provided in this embodiment of the invention determines the location of the file directory structure by first downloading a small amount of directory location data, and then obtains the file directory structure in a targeted manner. This avoids the redundant operation of downloading the complete compressed file and performing full decompression in existing technologies, effectively shortens the user's waiting time, reduces the network load on the server and storage, reduces the waste of server storage and computing resources, and significantly improves the processing efficiency and resource utilization of online compressed file directory preview scenarios.

[0085] See Figure 3 This invention also provides a compressed file processing device, including a processor 31, a memory 32, and a computer program stored in the memory 32 and configured to be executed by the processor 31. When the processor 31 executes the computer program, it implements the steps described in the compressed file processing method embodiments above, for example... Figure 1 S11~S13 in the above; or, when the processor 31 executes the computer program, it implements the functions of each module in the above-described device embodiments.

[0086] For example, the computer program can be divided into one or more modules, which are stored in the memory 32 and executed by the processor 31 to complete the present invention. The one or more modules can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the compressed file processing device. For example, the computer program can be divided into multiple modules, and the specific working process of each module can be referred to the working process of the compressed file processing device described in the above embodiments, which will not be repeated here.

[0087] The compressed file processing device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The compressed file processing device may include, but is not limited to, a processor 31 and a memory 32. Those skilled in the art will understand that the compressed file processing device may also include input / output devices, network access devices, buses, etc.

[0088] The processor 31 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 31 is the control center of the compressed file processing device, connecting all parts of the device via various interfaces and lines.

[0089] The memory 32 can be used to store the computer programs and / or modules. The processor 31 implements various functions of the compressed file processing device by running or executing the computer programs and / or modules stored in the memory 32 and calling the data stored in the memory 32. The memory 32 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as image playback function), etc.; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory 32 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0090] If the modules integrated into the compressed file processing device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by the processor 31, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0091] This invention also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the compressed file processing method as described in any of the above embodiments.

[0092] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for processing compressed files, characterized in that, include: In response to a directory preview command for a target compressed file, the directory location data of the target compressed file is downloaded from the storage device; Parse the directory location data to obtain the location information of the file directory structure in the target compressed file; The file directory structure is obtained from the storage terminal based on the location information.

2. The compressed file processing method as described in claim 1, characterized in that, The location information includes the start position of the directory area and the size of the directory area; parsing the directory location data to obtain the location information of the file directory structure in the target compressed file includes: Parse the directory location data to find the parsing anchor point; wherein the parsing anchor point is a preset fixed value; Based on the parsing anchor point, and according to the predefined field offset rules, the starting position and size of the directory area are found in the directory location data.

3. The compressed file processing method as described in claim 1, characterized in that, The step of downloading the directory location data of the target compressed file from the storage terminal in response to a directory preview command for the target compressed file includes: In response to a directory preview command for a target compressed file, the file feature value of the target compressed file is obtained; The file feature value of the target compressed file is used as the search condition to query a preset data storage unit; wherein, the data storage unit pre-stores the file feature values ​​of processed compressed files; the processed compressed file refers to a compressed file whose file directory structure has been obtained; If the file feature value of the target compressed file is not found, download the directory location data of the target compressed file from the storage terminal. The file characteristic values ​​of the target compressed file are recorded in the data storage unit.

4. The compressed file processing method as described in claim 1, characterized in that, The step of obtaining the file directory structure from the storage terminal based on the location information includes: obtaining the file directory structure from the storage terminal based on the location information when the file size of the target compressed file is less than or equal to a preset size threshold, and / or when the file antivirus service scans the target compressed file and finds it safe.

5. The compressed file processing method as described in claim 1, characterized in that, The method further includes: In response to a download instruction for the target compressed file, the system checks whether a node corresponding to the target compressed file exists in the local linked list of the local storage unit. When the local linked list contains a node corresponding to the target compressed file, the node corresponding to the target compressed file is updated to the first endpoint of the local linked list; When the local linked list does not contain a node corresponding to the target compressed file, if the file size of the target compressed file is greater than the free capacity of the local storage unit, nodes are sequentially deleted from the second endpoint of the local linked list and the storage space occupied by the compressed file corresponding to the deleted node is released until the file size of the target compressed file is less than or equal to the free capacity of the local storage unit. Then, the target compressed file is downloaded and linked to the first endpoint of the local linked list.

6. The compressed file processing method as described in claim 5, characterized in that, The method further includes: decompressing the target compressed file corresponding to the first endpoint of the local linked list.

7. The compressed file processing method as described in claim 5, characterized in that, Each node in the local linked list also records its most recent usage time; the method further includes: deleting idle nodes from the local linked list and releasing the storage space occupied by the compressed file corresponding to the idle node; wherein, the idle node is a node whose idle time exceeds a preset time threshold.

8. The compressed file processing method as described in claim 1, characterized in that, Also includes: In response to a download command for the target compressed file, the target compressed file is downloaded in multiple threads according to a preset slice size. After all slices are downloaded, the slices are merged to obtain the target compressed file.

9. The compressed file processing method as described in claim 1, characterized in that, Also includes: In response to a download command for the target compressed file, query the pre-built file-to-service instance mapping relationship; When no service instance corresponding to the target compressed file is found, the download command is forwarded to an idle service instance so that the idle service instance can respond to the download command and perform the operation corresponding to the download command. When a target service instance is found and the target service instance is alive, the download command is forwarded to the target service instance so that the target service instance can respond to the download command and perform the operation corresponding to the download command; wherein, the target service instance is the service instance corresponding to the target compressed file.

10. The compressed file processing method as described in claim 1, characterized in that, Also includes: In response to an upload command for a decompressed file stored on the server, the file identifier of the decompressed file is calculated, and a pre-built file fingerprint database is queried based on the file identifier of the decompressed file; wherein, the file fingerprint database records the file identifier of the uploaded file and the cloud storage location of the uploaded file; If the file identifier of the decompressed file is found, the cloud storage location of the decompressed file is returned; If the file identifier of the decompressed file is not found, the decompressed file will be uploaded in multiple threads in segments.

11. A compressed file processing device, characterized in that, include: The location data download module is used to download the directory location data of the target compressed file from the storage terminal in response to the directory preview command of the target compressed file; The location information acquisition module is used to parse the directory location data to obtain the location information of the file directory structure in the target compressed file; The directory acquisition module is used to obtain the file directory structure from the storage terminal based on the location information.

12. A compressed file processing device, characterized in that, The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the compressed file processing method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the compressed file processing method as described in any one of claims 1 to 10.

14. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the compressed file processing method as described in any one of claims 1 to 10.

Images