Attachment packaging and unpackaging method, system and equipment based on compression algorithm and medium

Abstract

The invention provides an attachment packing and unpacking method, system and device based on a compression algorithm and a medium, and belongs to the technical field of file compression. The method comprises the steps that all attachment files needing to be packaged are obtained by traversing a specified directory, screening and filtering are conducted according to file attributes, and a renamed file list and a path mapping table are generated; according to the renamed file list, using a Brotli algorithm to compress each attachment file based on a preset compression level, and packaging the compressed data into a packaged file containing metadata, compressed data and verification information; when the packed file needs to be unpacked, the legality of the packed file is confirmed through magic number verification and index analysis, file data are recovered through Brotli streaming decompression, and the original directory structure of the file is restored. According to the method, efficient compression and streaming decompression are achieved through the Brotli algorithm, and the attachment storage efficiency, the transmission speed and the data integrity are remarkably improved in combination with the intelligent screening and directory structure reduction technology.

Description

Technical Field

[0001] The present invention belongs to the technical field of file compression, and more particularly relates to a method, system, device and medium for packaging and unpacking attachments based on a compression algorithm. Background Art

[0002] In traditional software system architectures, file bundling and compression are crucial for ensuring efficient data storage and transmission. Among numerous compression algorithms, the Gzip algorithm, due to its maturity and convenience, is a common choice for handling file bundling and compression tasks. In the early stages of project development, developers often choose the JDK's native compression algorithms to handle the various attachment files within the system's memory, considering time constraints, prioritizing development efficiency, and platform compatibility. Ultimately, when the business scale is small and file processing requirements are relatively simple, these native algorithms can generally meet the basic operational needs of the business, ensuring the proper implementation of file compression and bundling functions and providing fundamental support for business development.

[0003] However, with the continued expansion and deepening of software system services, file processing scenarios have undergone tremendous changes. File sizes have continued to expand, from early documents and images to large database backups, multimedia resources, and other resources, often reaching several GB or even larger. File hierarchies have also become increasingly complex, with multiple nested directories and multiple file types intertwined. The number of files has also increased exponentially. Against this backdrop, traditional packaging methods based on JDK's native compression algorithms (Gzip, for example) have become increasingly inadequate. Their efficiency has significantly declined, and the compression process is time-consuming, severely impacting business responsiveness. Stability has also been severely challenged, with frequent compression failures and file corruption, and the algorithm's usability has been dramatically reduced.

[0004] Furthermore, the currently mainstream Gzip and deflate algorithms also have significant shortcomings when addressing complex file processing requirements. Their compression performance is poor, and the compression process is time-consuming, reducing business processing efficiency. Their low compression ratios make it difficult to effectively compress large attachments, consuming significant storage space. Furthermore, compressed file transfer efficiency is low, consuming system read and write resources for extended periods, further burdening the system, disrupting the normal operation of other services, and potentially causing serious consequences such as system deadlocks or downtime. Summary of the Invention

[0005] In response to the above problems, the purpose of the present invention is to provide an attachment packaging and unpacking method, system, device and medium based on a compression algorithm, which realizes efficient compression and streaming decompression through the Brotli algorithm, and combines intelligent screening and directory structure restoration technology to significantly improve attachment storage efficiency, transmission speed and data integrity.

[0006] To achieve the above-mentioned purpose, the present invention is implemented through the following technical solutions: In a first aspect, an embodiment of the present application provides a method for packaging and unpacking attachments based on a compression algorithm, comprising: Get all the attachment files that need to be packaged by traversing the specified directory, filter them according to file attributes, and generate a renamed file list and path mapping table; Based on the renamed file list, each attachment file is compressed using the Brotli algorithm at the preset compression level, and the compressed data is encapsulated into a package file containing metadata, compressed data, and verification information; When the packaged file needs to be unpacked, the validity of the packaged file is confirmed through magic number verification and index analysis, and Brotli streaming decompression is used to recover the file data and restore the original directory structure of the file.

[0007] In an optional embodiment, the step of traversing a specified directory to obtain all attachment files that need to be packaged includes: Use the depth-first or breadth-first algorithm to traverse the specified directory and its subdirectories to obtain the paths of all attachment files that need to be packaged; Generate a file path collection based on all attachment file paths and obtain the corresponding files.

[0008] In an optional embodiment, the filtering based on file attributes to generate a renamed file list and a path mapping table includes: Based on all attachment files that need to be packaged, the extension and MIME type to be filtered are matched by regular expressions according to the file path set to filter out invalid type files; Call the virus scanning interface to mark malicious files, filter out files that exceed the preset threshold based on file size, perform encoding verification and integrity checks on the remaining files, and generate a set of valid file paths; Based on the valid file path set, a new file name is generated according to the timestamp or serial number, a mapping relationship between the old and new names is established, and the correspondence between the original relative path of each file and the renamed path is recorded to generate a renamed file list and a path mapping table.

[0009] In an optional embodiment, compressing each attachment file using the Brotli algorithm based on a preset compression level according to the renamed file list includes: Configure the compression level, initialize the Brotli sliding window and 120KB static dictionary, and configure a multi-threaded environment; According to the renamed file list, each file is divided into blocks according to the preset size, and a block data list with identification is generated; According to the block data list, the block data of the rename file is grouped by the number of threads, and polled and assigned to each thread for parallel compression processing; Searching for repeated sequences in the block data and the sliding window using the LZ77 algorithm to generate a mixed literal and pair phrase sequence; Building a dynamic Huffman tree based on the phrase sequence frequency and converting the phrase sequence into a binary coded intermediate stream; Scan the intermediate stream and match it with the static dictionary, classify the context by byte features and apply the probability model to generate compressed data blocks; Merge the compressed data blocks output by the threads, sort them by file ID, and aggregate metadata; verify the CRC32 checksum of the compressed data is consistent with the original file, calculate the compression ratio, and output a set of compressed data blocks with metadata in sequence, sort them by file ID, and aggregate metadata; Verify the CRC32 checksum of the compressed data and the consistency of the original file, calculate the compression ratio, and output the compressed data block set with metadata in sequence.

[0010] In an optional embodiment, encapsulating the compressed data into a package file containing metadata, compressed data, and verification information includes: Constructing a file structure of a packaged file, the file structure including a file header, a file index area, a compressed data area, and a check information area; By calculating the CRC32 check value of the original file and the compressed data corresponding to the compressed data block, combined with the file name and file size, an index array containing the file name, original size, compressed size, and check value is generated; The magic number, version number, number of files, and compression level are stored in the file header; the index array is converted to binary format and stored in the file index area; the compressed data blocks are sequentially spliced together in index order and stored in the compressed data area; the hash value of the contents of the compressed data area is calculated to generate verification information and store it in the verification information area; Storage information based on the file structure of the packaged file is encapsulated as binary data; Using memory mapping technology, binary data is written into temporary files in the disk buffer in blocks. After the data is verified, the temporary file is renamed to the target file name to generate a packaged file.

[0011] In an optional embodiment, confirming the legitimacy of the packaged file through magic number verification and index parsing includes: Get the file header of the packaged file, confirm the validity of the file format by checking the magic number, verify the version compatibility of the packaged file based on the version number, and verify the data integrity of the file header; By parsing the file index area, the package file name, the original size of the package file, and the compressed file are obtained, and the obtained information is used as metadata to build a file directory mapping table.

[0012] In an optional embodiment, recovering file data using Brotli streaming decompression and restoring the original directory structure of the file includes: Locate the compressed data block of the target file according to the file directory mapping table; Read compressed data block by block through memory mapping, Initialize the Brotli decompression context, load the static dictionary, and use multi-threaded parallel processing of compressed data blocks, performing streaming decompression through Huffman decoding and LZ77 inverse transform; Calculate the CRC32 checksum of the decompressed data, compare it with the CRC32 checksum of the corresponding original file, verify the data integrity, and obtain the decompressed original file data; According to the path mapping table, query the mapping relationship by renaming the file name to obtain the original relative path; recursively create subdirectories at all levels in the original path to build a complete directory structure; Write the decompressed original file data into the corresponding file according to the original path.

[0013] In a second aspect, the embodiments of the present application further provide an attachment packaging and unpacking system based on a compression algorithm, including: The file collection and preprocessing module is used to obtain all attachment files that need to be packaged by traversing the specified directory, filtering them according to file attributes, and generating a renamed file list and path mapping table; The file compression and packaging module is used to compress each attachment file based on the renamed file list using the Brotli algorithm at a preset compression level, and encapsulate the compressed data into a package file containing metadata, compressed data, and verification information; The file unpacking and restoration module is used to confirm the legitimacy of the packaged file through magic number verification and index analysis when the packaged file needs to be unpacked, use Brotli streaming decompression to recover the file data, and restore the original directory structure of the file.

[0014] In a third aspect, an embodiment of the present application further provides an electronic device comprising a memory, a processor, and a computer program stored on the memory and runnable on the processor. When the processor executes the program, the steps of the attachment packaging and unpacking method based on the compression algorithm as described in any one of the above items are implemented.

[0015] In a fourth aspect, an embodiment of the present application further provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the attachment packaging and unpacking method based on the compression algorithm as described in any one of the above items are implemented.

[0016] It can be seen from the above technical solutions that the present invention has the following advantages: In the attachment packaging and unpacking method based on the compression algorithm provided in this application, the validity and security of the packaged files are ensured by combining depth- and breadth-first traversal with multiple screening, the multi-threaded block compression of the Brotli algorithm and static dictionary optimization are used to improve the compression efficiency and compression ratio, the CRC32 checksum and hash checksum are used to ensure data integrity, the file header magic number checksum and version compatibility verification are used to ensure the legitimacy of the packaged files, the memory mapping technology is used to achieve reliable disk writing, and the original directory structure is accurately restored based on the path mapping table during unpacking. The overall advantages include efficient screening, secure compression, fast unpacking, data integrity verification and lossless restoration of the directory structure, which effectively improve the security, reliability and processing efficiency of attachment packaging and unpacking.

[0017] This application uses regular expressions to filter invalid extensions and MIME types, combines it with a virus scanning interface to intercept malicious files, and removes files that exceed the threshold size to ensure data legitimacy. Encoding verification and CRC32 integrity checks further eliminate corrupted files, reduce manual review costs, and improve the security of the packaging process. It is particularly suitable for scenarios with strict data compliance requirements.

[0018] This application utilizes Brotli's sliding window, static dictionary, and multi-threaded block compression to achieve high compression ratios and low resource consumption. Dynamic Huffman trees and the LZ77 algorithm optimize redundant data encoding, significantly reducing storage and transmission times. It is particularly adept at processing highly repetitive content such as text and code, balancing speed and efficiency, and is suitable for cold storage archiving and CDN distribution scenarios.

[0019] This application's hierarchical file structure supports fast magic number identification, version compatibility verification, and random file access. SHA-256 hashing and block-level CRC32 checksums ensure data has not been tampered with or corrupted, providing traceable integrity assurance suitable for long-term archiving and cross-platform data exchange scenarios.

[0020] This application uses Brotli streaming decoding combined with multi-threaded parallel processing to achieve instant decompression with low memory usage. Path mapping tables and recursive directory creation techniques accurately restore the original file structure, and CRC32 checksums ensure data consistency, making it suitable for fast business recovery in embedded devices and low-end environments.

[0021] This application dynamically adjusts block size and compression level based on file type, and a hybrid encoding model automatically adapts to high- and low-redundancy data. Sliding window expansion and probabilistic models optimize encoding efficiency, balance CPU and memory resources, and support flexible deployment in resource-constrained scenarios such as real-time communications and edge computing. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solution of the present invention, the following is a brief introduction to the drawings required for the description. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 A flowchart of the attachment packaging and unpacking method based on the compression algorithm provided in this application.

[0024] Figure 2 This is a structural diagram of the attachment packaging and unpacking system based on the compression algorithm provided in this application.

[0025] Figure 3 This is a schematic diagram of the structure of the electronic device provided in this application. DETAILED DESCRIPTION

[0026] The various embodiments of the present disclosure will be described more fully below in detail in the specific steps of the method for packing and unpacking attachments based on the compression algorithm. The present disclosure can have various embodiments, and adjustments and changes can be made therein. However, it should be understood that there is no intention to limit the various embodiments of the present disclosure to the specific embodiments disclosed herein, but rather that the present disclosure should be understood to encompass all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of the present disclosure.

[0027] Hereinafter, the terms "include" or "may include" as used in various embodiments of the present disclosure indicate the presence of disclosed functions, operations, or elements, and do not limit the addition of one or more functions, operations, or elements. In addition, as used in various embodiments of the present disclosure, the terms "include," "have," and their cognates are intended only to indicate specific features, numbers, steps, operations, elements, components, or combinations of the foregoing, and should not be understood as excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing, or the possibility of adding one or more features, numbers, steps, operations, elements, components, or combinations of the foregoing.

[0028] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0029] See also Figure 1FIG. 1 is a flowchart of a method for packaging and unpacking attachments based on a compression algorithm in a specific embodiment. The method includes: S1: Obtain all attachment files that need to be packaged by traversing the specified directory, filter them according to file attributes, and generate a renamed file list and path mapping table.

[0030] In a specific implementation, first use the depth-first search (DFS) or breadth-first search (BFS) algorithm to traverse the specified directory and its subdirectories to obtain all attachment file paths that need to be packaged; generate a file path set based on all attachment file paths, and obtain the corresponding files.

[0031] Then, the collected files are screened and invalid files can be filtered out based on file type, size or other attributes.

[0032] Specifically, it includes: based on all the attachment files that need to be packaged, matching the extensions and MIME types to be filtered through regular expressions according to the file path set, filtering out invalid type files; calling the virus scanning interface to mark malicious files, filtering out files that exceed the preset threshold according to the file size, performing encoding verification and integrity checks on the remaining files, and generating a valid file path set.

[0033] For example, the file filtering and selection are performed along the following four dimensions: File type: filter by extension or MIME type, as well as whitelist and blacklist.

[0034] Security: Detect and exclude malicious files.

[0035] File size: Set a threshold, exceeding which files are marked as invalid.

[0036] Content validity: file encoding verification, image integrity check.

[0037] When filtering, define rules based on file names or content (such as log files) and filter files using regular expression matching. For example, filter temporary files (~$*) or system hidden files. Define filtering rules using JSON / YAML configuration files, and support hot updates.

[0038] Finally, based on the valid file path set, a new file name is generated according to the timestamp or serial number, a mapping relationship between the old and new names is established, and the correspondence between the original relative path of each file and the renamed path is recorded to generate a renamed file list and a path mapping table.

[0039] For example, the purpose of this process is to rename files and process paths.

[0040] The renaming strategy employed includes embedding timestamps, automatically generating and filling serial numbers, preserving relative paths during path processing to store files in their original directory structure within the compressed package and restore the hierarchy upon decompression, and generating a path mapping table in CSV / JSON format for accurate restoration upon decompression.

[0041] S2: Based on the renamed file list, each attachment file is compressed using the Brotli algorithm at a preset compression level, and the compressed data is encapsulated into a package file containing metadata, compressed data, and verification information.

[0042] In a specific implementation, this step includes a file compression process, a compressed data packaging process, and a packaged file generation process.

[0043] 1. File compression process: First, configure the compression level, initialize the Brotli sliding window and a 120KB static dictionary, and set up a multi-threaded environment. Then, based on the list of renamed files, split each file into blocks of preset sizes and generate a list of identified blocks. Based on this list, the blocks of the renamed files are grouped by the number of threads and assigned to each thread for parallel compression processing.

[0044] At this time, the LZ77 algorithm is used to search for repeated sequences in the block data and the sliding window to generate a phrase sequence of mixed literals and pairs, and a dynamic Huffman tree is constructed based on the frequency of the phrase sequence to convert the phrase sequence into a binary coded intermediate stream.

[0045] Finally, the intermediate stream is scanned and matched against a static dictionary. Context is classified by byte features and a probabilistic model is applied to generate compressed data blocks. The compressed data blocks output by the threads are merged, sorted by file identifier, and metadata is aggregated. The compressed data CRC32 checksum is verified for consistency with the original file, the compression ratio is calculated, and a set of compressed data blocks with metadata is output in sequence.

[0046] For example, in this process, each file is compressed using the Brotli algorithm. You can choose an appropriate compression level and consider concurrent processing to improve compression efficiency.

[0047] The core components of the Brotli algorithm are the following three: LZ77 variant algorithm: identifies duplicate data blocks through a sliding window mechanism with a window size of up to 16MB (24-level window parameter), supporting long-distance duplicate matching; Huffman coding optimization: Using a dynamic Huffman tree structure, the coding table is dynamically adjusted according to the data flow to improve the compression efficiency of high-frequency characters; Second-order context model: Combined with a predefined 120KB static dictionary (containing more than 13,000 common words), it significantly improves the compression density of text files, especially suitable for web resources such as HTML / CSS / JS.

[0048] 2. Compressed data packaging process: First, the file structure of the packaged file is constructed, which includes a file header, a file index area, a compressed data area, and a check information area. By calculating the CRC32 check value of the original file and the CRC32 check value of the compressed data block corresponding to the compressed data block, combined with the file name and file size, an index array containing the file name, original size, compressed size, and check value is generated.

[0049] At this time, the magic number, version number, number of files, and compression level are stored in the file header; the index array is converted into binary format and stored in the file index area; the compressed data blocks are spliced together in sequence according to the index order and stored in the compressed data area; the hash value of the content of the compressed data area is calculated, and the verification information is generated and stored in the verification information area; finally, the storage information based on the file structure of the packaged file is encapsulated as binary data.

[0050] 3. Package file generation process: Using memory mapping technology, binary data is written into temporary files in the disk buffer in blocks. After the data is verified, the temporary file is renamed to the target file name to generate a packaged file.

[0051] In this process, the compressed binary data is encapsulated into a unified packaging format, and specific file header information can be used to identify the number of compressed files, the name of each file, the original size, the compressed size, the compression algorithm version, etc.

[0052] It's also important to note that during this process, you can choose to encrypt the file content or digitally sign it to enhance data security. Encryption can use either the symmetric AES-256-GCM algorithm or an asymmetric algorithm.

[0053] S3: When the packaged file needs to be unpacked, the legitimacy of the packaged file is confirmed through magic number check and index analysis, and the file data is recovered using Brotli streaming decompression to restore the original directory structure of the file. In a specific implementation, the file is first parsed, specifically including: Get the file header of the packaged file, confirm the validity of the file format by checking the magic number, verify the version compatibility of the packaged file based on the version number, and verify the data integrity of the file header.

[0054] Locate and read the file index area, obtain the package file name, original size of the package file, and compressed file by parsing each FileEntry structure in the file index area, and use the obtained information as metadata to build a file directory mapping table.

[0055] In addition, if the compressed file is an encrypted file, it is also necessary to verify the security of the file by verifying the authentication expression and digital signature.

[0056] For example, by reading the magic check number "0xBRTL" in the first 4 bytes of the packaged file, parsing the version number and verifying the CRC32 checksum value of the file header, the validity of the file format is confirmed; the index area is located, metadata such as the file name, original size, and compressed size are extracted, and a file directory tree is generated. At the same time, the authentication tag of the encrypted package and the digital signature of the signed package are verified as required to complete the security verification and provide accurate information for subsequent decompression and restoration.

[0057] Then, restore the files, including: Locate the compressed data block of the target file according to the file directory mapping table; Read compressed data block by block through memory mapping, Initialize the Brotli decompression context, load the static dictionary, and use multi-threaded parallel processing of compressed data blocks, performing streaming decompression through Huffman decoding and LZ77 inverse transform; Calculate the CRC32 checksum of the decompressed data, compare it with the CRC32 checksum of the corresponding original file, verify the data integrity, and obtain the decompressed original file data; For example, memory mapping technology is used to map compressed data blocks to memory to avoid full loading; the Brotli decompression context is initialized, a 120KB static dictionary is loaded, multi-threaded parallel processing of block data is used, and streaming decompression is performed through Huffman decoding and LZ77 inverse transform; the CRC32 checksum of the decompressed data is calculated and compared with the original CRC32 checksum stored in the index to verify data integrity and obtain the decompressed original file data set.

[0058] Finally, restore the original directory structure, including: According to the path mapping table, query the mapping relationship by renaming the file name to obtain the original relative path; recursively create subdirectories at all levels in the original path to build a complete directory structure; Write the decompressed original file data into the corresponding file according to the original path.

[0059] For example, based on the path mapping table (JSON / CSV format) generated during the packaging phase, the mapping relationship is queried by renaming the file name to obtain the original relative path; subdirectories at all levels in the original path are recursively created to build a complete directory structure; the decompressed original file data is written to the corresponding file according to the original path to complete the accurate restoration of the file system structure, making it completely consistent with before packaging.

[0060] This embodiment optimizes the entire attachment processing process by integrating intelligent filtering, efficient compression, and structured storage technologies. First, a multi-dimensional filtering mechanism based on regular expressions, virus scanning, and size thresholds effectively eliminates invalid files and potential security risks, improving data processing compliance. Second, multi-threaded block compression using the Brotli algorithm and static dictionary optimization significantly reduces storage space usage and transmission time while maintaining low resource consumption, making it particularly suitable for deep compression of text files. Furthermore, a hierarchical file structure design and a dual validation mechanism (magic number checksum + hash verification) ensure the integrity and cross-platform compatibility of packaged files, supporting random access and incremental updates. Furthermore, streaming decompression technology combined with a path mapping table enables memory-efficient, instant decompression and restoration of the original directory structure, while CRC32 verification further ensures the accuracy of decompressed data. Finally, a dynamic resource allocation strategy adaptively adjusts compression parameters based on file type, balancing processing speed and hardware resource utilization, enabling flexible adaptation to diverse scenarios such as real-time communications and edge computing. The overall solution demonstrates significant advantages in improving storage and transmission efficiency, strengthening security, and simplifying the data restoration process.

[0061] like Figure 2 As shown, the following is an embodiment of the attachment packaging and unpacking system based on the compression algorithm provided by the embodiment of the present disclosure. This system and the attachment packaging and unpacking method based on the compression algorithm of the above-mentioned embodiments belong to the same inventive concept. For details not fully described in the embodiment of the attachment packaging and unpacking system based on the compression algorithm, please refer to the embodiment of the attachment packaging and unpacking method based on the compression algorithm.

[0062] The invention discloses an attachment packaging and unpacking system based on a compression algorithm, comprising a file collection and preprocessing module, a file compression and packaging module, and a file unpacking and restoration module.

[0063] The file collection and preprocessing module is used to obtain all attachment files that need to be packaged by traversing the specified directory, filtering them according to file attributes, and generating a renamed file list and path mapping table.

[0064] The file compression and packaging module is used to compress each attachment file based on the renamed file list and the preset compression level using the Brotli algorithm, and encapsulate the compressed data into a package file containing metadata, compressed data and verification information.

[0065] The file unpacking and restoration module is used to confirm the legitimacy of the packaged file through magic number verification and index analysis when the packaged file needs to be unpacked, use Brotli streaming decompression to recover the file data, and restore the original directory structure of the file.

[0066] The compression algorithm-based attachment packaging and unpacking system provided in this embodiment achieves efficient and secure attachment processing through intelligent screening and Brotli algorithm optimization: it combines regular filtering, virus scanning and size threshold control to eliminate invalid / malicious files, and uses Brotli's sliding window, static dictionary and multi-threaded block compression technology to significantly improve the compression ratio and speed. It ensures data integrity and cross-platform compatibility through a hierarchical file structure and a double verification mechanism, uses streaming decompression and path mapping technology to quickly restore the original directory structure, and supports dynamic resource allocation to adapt to different scenarios, comprehensively optimizing storage efficiency, transmission performance and data reliability.

[0067] Figure 3 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0068] The attachment packing and unpacking method based on the compression algorithm provided in the embodiment of the present application can be applied to electronic devices. Those skilled in the art will understand that the electronic device structure involved in the embodiment of the present invention does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. In the embodiment of the present invention, the electronic device includes but is not limited to a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the present application described and / or required herein.

[0069] The electronic device may include a processor, an external memory interface, an internal memory, a universal serial bus (USB) interface, a charging management module, a power management module, a battery, a wireless communication module, an audio module, a speaker, a microphone, a sensor module, a button, a camera, a display, and a SIM card interface, etc.

[0070] A processor may include one or more processing units, such as a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0071] The processor can be the nerve center and command center of the electronic device. The controller can generate operation control signals based on the instruction opcode and timing signal to complete the control of instruction fetching and execution.

[0072] The processor may also include a memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can store instructions or data that the processor has just used or is reusing. If the processor needs to use the instruction or data again, it can directly call it from the memory. This avoids repeated accesses, reduces processor latency, and thus improves system efficiency.

[0073] The external memory interface can be used to connect an external memory card, such as a MicroSD card, to expand the storage capacity of an electronic device. The external memory card communicates with the processor through the external memory interface, enabling data storage. For example, files such as music and videos can be stored on the external memory card.

[0074] Internal memory can be used to store computer-executable program code, which includes instructions. The processor executes the instructions stored in the internal memory to perform various functional applications and data processing of the electronic device. The internal memory can include a program storage area and a data storage area. The internal memory can include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0075] The wireless communication function of an electronic device can be implemented through an antenna, a wireless communication module, a modem processor, and a baseband processor.

[0076] Wireless communication modules can provide wireless communication solutions for electronic devices, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc.

[0077] Electronic devices can implement audio functions through audio modules, speakers, receivers, microphones, headphone jacks, and application processors.

[0078] Electronic devices can achieve shooting functions through ISP, camera, video codec, GPU, display and application processor.

[0079] Electronic devices can achieve display functions through GPU, display screen and application processor.

[0080] A GPU is a microprocessor for image processing that connects the display screen to the application processor. The GPU performs mathematical and geometric calculations for graphics rendering. A processor may include one or more GPUs, which execute program instructions to generate or modify display information.

[0081] The display screen is used to display images, videos, etc. The display screen includes a display panel.

[0082] The above-mentioned electronic device implements the attachment packaging and unpacking method based on the compression algorithm of this application through intelligent screening combined with Brotli algorithm multi-threaded block compression and static dictionary optimization, and is supplemented by file header magic number check, CRC32 integrity verification and SHA-256 hash protection. At the same time, streaming decompression and path mapping technology are used to realize directory structure restoration. Finally, through dynamic resource allocation to adapt to diversified scenarios, it achieves the beneficial effects of significantly improving storage and transmission efficiency, ensuring data security and compliance, ensuring decompression and restoration accuracy, and taking into account the balance between resource consumption and processing speed.

[0083] The storage medium provided in the present application stores a program product that can implement an attachment packaging and unpacking method based on a compression algorithm.

[0084] Attachment packaging and unpacking methods based on compression algorithms include: Get all the attachment files that need to be packaged by traversing the specified directory, filter them according to file attributes, and generate a renamed file list and path mapping table; Based on the renamed file list, each attachment file is compressed using the Brotli algorithm at the preset compression level, and the compressed data is encapsulated into a package file containing metadata, compressed data, and verification information; When the packaged file needs to be unpacked, the validity of the packaged file is confirmed through magic number verification and index analysis, and Brotli streaming decompression is used to recover the file data and restore the original directory structure of the file.

[0085] In some possible implementations, the compression algorithm-based attachment packaging and unpacking disclosed herein can be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps described in the above "Exemplary Method" section of this specification according to various exemplary implementations of the present disclosure.

[0086] The storage medium of the present disclosure can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0087] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for packaging and unpacking attachments based on a compression algorithm, characterized in that: include: Get all the attachment files that need to be packaged by traversing the specified directory, filter them according to file attributes, and generate a renamed file list and path mapping table; Based on the renamed file list, each attachment file is compressed using the Brotli algorithm at the preset compression level, and the compressed data is encapsulated into a package file containing metadata, compressed data, and verification information; When the packaged file needs to be unpacked, the validity of the packaged file is confirmed through magic number verification and index analysis, and Brotli streaming decompression is used to recover the file data and restore the original directory structure of the file.

2. The attachment packaging and unpacking method based on compression algorithm according to claim 1 is characterized in that: The method of traversing the specified directory to obtain all attachment files that need to be packaged includes: Use the depth-first or breadth-first algorithm to traverse the specified directory and its subdirectories to obtain the paths of all attachment files that need to be packaged; Generate a file path collection based on all attachment file paths and obtain the corresponding files.

3. The attachment packaging and unpacking method based on compression algorithm according to claim 2, characterized in that: The filtering based on file attributes to generate a renamed file list and path mapping table includes: Based on all attachment files that need to be packaged, the extension and MIME type to be filtered are matched by regular expressions according to the file path set to filter out invalid type files; Call the virus scanning interface to mark malicious files, filter out files that exceed the preset threshold based on file size, perform encoding verification and integrity checks on the remaining files, and generate a set of valid file paths; Based on the valid file path set, a new file name is generated according to the timestamp or serial number, a mapping relationship between the old and new names is established, and the correspondence between the original relative path of each file and the renamed path is recorded to generate a renamed file list and a path mapping table.

4. The method for packaging and unpacking attachments based on a compression algorithm according to claim 3, characterized in that: The method of compressing each attachment file using the Brotli algorithm based on a preset compression level according to the renamed file list includes: Configure the compression level, initialize the Brotli sliding window and 120KB static dictionary, and configure a multi-threaded environment; According to the renamed file list, each file is divided into blocks according to the preset size, and a block data list with identification is generated; According to the block data list, the block data of the rename file is grouped by the number of threads, and polled and assigned to each thread for parallel compression processing; Searching for repeated sequences in the block data and the sliding window using the LZ77 algorithm to generate a mixed literal and pair phrase sequence; Building a dynamic Huffman tree based on the phrase sequence frequency and converting the phrase sequence into a binary coded intermediate stream; Scan the intermediate stream and match it with the static dictionary, classify the context by byte features and apply the probability model to generate compressed data blocks; Merge the compressed data blocks output by the threads, sort them by file ID and aggregate metadata; Verify the CRC32 checksum of the compressed data and the consistency of the original file, calculate the compression ratio, and output the compressed data block set with metadata in sequence.

5. The attachment packaging and unpacking method based on compression algorithm according to claim 4 is characterized in that: The method of packaging the compressed data into a package file containing metadata, compressed data, and verification information includes: Constructing a file structure of a packaged file, the file structure including a file header, a file index area, a compressed data area, and a check information area; By calculating the CRC32 check value of the original file and the compressed data corresponding to the compressed data block, combined with the file name and file size, an index array containing the file name, original size, compressed size, and check value is generated; The magic number, version number, number of files, and compression level are stored in the file header; the index array is converted to binary format and stored in the file index area; the compressed data blocks are sequentially spliced together in index order and stored in the compressed data area; the hash value of the contents of the compressed data area is calculated to generate verification information and store it in the verification information area; Storage information based on the file structure of the packaged file is encapsulated as binary data; Using memory mapping technology, binary data is written into temporary files in the disk buffer in blocks. After the data is verified, the temporary file is renamed to the target file name to generate a packaged file.

6. The attachment packaging and unpacking method based on compression algorithm according to claim 5, characterized in that: The validation of the packaged file through magic number verification and index analysis includes: Get the file header of the packaged file, confirm the validity of the file format by checking the magic number, verify the version compatibility of the packaged file based on the version number, and verify the data integrity of the file header; By parsing the file index area, the package file name, the original size of the package file, and the compressed file are obtained, and the obtained information is used as metadata to build a file directory mapping table.

7. The attachment packaging and unpacking method based on compression algorithm according to claim 6, characterized in that: The method of recovering file data by using Brotli streaming decompression and restoring the original directory structure of the file includes: Locate the compressed data block of the target file according to the file directory mapping table; Read compressed data block by block through memory mapping, Initialize the Brotli decompression context, load the static dictionary, and use multi-threaded parallel processing of compressed data blocks, performing streaming decompression through Huffman decoding and LZ77 inverse transform; Calculate the CRC32 checksum of the decompressed data, compare it with the CRC32 checksum of the corresponding original file, verify the data integrity, and obtain the decompressed original file data; According to the path mapping table, query the mapping relationship by renaming the file name to obtain the original relative path; recursively create subdirectories at all levels in the original path to build a complete directory structure; Write the decompressed original file data into the corresponding file according to the original path.

8. An attachment packaging and unpacking system based on a compression algorithm, characterized in that: The system adopts the attachment packaging and unpacking method based on the compression algorithm as claimed in any one of claims 1 to 7; The system comprises: The file collection and preprocessing module is used to obtain all attachment files that need to be packaged by traversing the specified directory, filtering them according to file attributes, and generating a renamed file list and path mapping table; The file compression and packaging module is used to compress each attachment file based on the renamed file list using the Brotli algorithm at a preset compression level, and encapsulate the compressed data into a package file containing metadata, compressed data, and verification information; The file unpacking and restoration module is used to confirm the legitimacy of the packaged file through magic number verification and index analysis when the packaged file needs to be unpacked, use Brotli streaming decompression to recover the file data, and restore the original directory structure of the file.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the attachment packaging and unpacking method based on the compression algorithm as described in any one of claims 1 to 7 are implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the attachment packaging and unpacking method based on a compression algorithm as claimed in any one of claims 1 to 7 are implemented.

Images