A high-compression and high-encryption security system and method for unstructured files

Abstract

The present application provides a high-compression and high-encryption security system and method for unstructured files. The high-compression and high-encryption security system for unstructured files includes: an intelligent chunking module for dynamically partitioning a file into blocks according to the file content type to obtain file blocks; a hybrid compression engine for matching an optimal compression algorithm combination for the file blocks and performing compression processing; and a dynamic hierarchical encryption module for encrypting the file blocks. In the above technical solution, the compression efficiency and security of unstructured files are improved.

Description

Background Art

[0002] Existing compression and encryption technologies for unstructured files have the following disadvantages:

[0003] Low compression efficiency: Traditional algorithms (such as ZIP, RAR) have limited compression rates for unstructured files, especially poor effects on files with mixed content (such as PDFs containing text and images).

[0004] Insufficient encryption security: A single encryption key is easily cracked, and compression after encryption may lead to an increase in data redundancy.

[0005] Risk of metadata leakage: Existing technologies ignore the separate protection of file metadata (such as creation time, author information), which is easily used for side-channel attacks.

[0006] Slow processing speed: Encryption and compression are performed step by step, resulting in excessive time consumption when processing large files. Summary of the Invention

[0007] This application provides a high-compression and high-encryption security system and method for unstructured files to improve the compression efficiency and security of unstructured files.

[0008] In the first aspect, a high-compression and high-encryption security system for unstructured files is provided, including:

[0009] An intelligent chunking module for dynamically dividing a file into blocks according to the file content type to obtain file blocks;

[0010] A hybrid compression engine for matching an optimal compression algorithm combination for the file blocks and performing compression processing;

[0011] A dynamic hierarchical encryption module for encrypting the file blocks.

[0012] In the above technical solution, by setting an intelligent chunking module for dynamically dividing a file into blocks according to the file content type to obtain file blocks; a hybrid compression engine for matching an optimal compression algorithm combination for the file blocks and performing compression processing; a dynamic hierarchical encryption module for encrypting the file blocks; the compression efficiency and security of unstructured files are improved.

[0013] In a specific feasible implementation, it further includes:

[0014] A metadata protection module for obfuscating and encrypting file metadata and storing it separately.

[0015] In a specific feasible implementation, it further includes:

[0016] A dynamic key management system for generating temporary keys based on file content and timestamps and automatically destroying them after each processing is completed.

[0017] In a specific feasible implementation, AES-256 is used to encrypt critical blocks, and ChaCha20 is used to encrypt non-critical blocks.

[0018] In a specific feasible implementation, the optimal compression algorithm combination includes:

[0019] LZ77 and Huffman coding combination is used to compress text data;

[0020] WebP and quantization combination is used to compress image data.

[0021] In a second aspect, a high-compression and high-encryption security method for unstructured files is provided, including the following steps:

[0022] Using an intelligent chunking module to dynamically partition a file into chunks according to the file content type to obtain file chunks;

[0023] Using a hybrid compression engine to match the optimal compression algorithm combination for the file chunks and perform compression processing;

[0024] Using a dynamic hierarchical encryption module to encrypt the file chunks.

[0025] In the above technical solution, by setting an intelligent chunking module for dynamically partitioning a file into chunks according to the file content type to obtain file chunks; a hybrid compression engine for matching the optimal compression algorithm combination for the file chunks and performing compression processing; and a dynamic hierarchical encryption module for encrypting the file chunks, the compression efficiency and security of unstructured files are improved.

[0026] In a specific feasible implementation, it further includes:

[0027] Using a metadata protection module to obfuscate and encrypt file metadata and store it separately.

[0028] In a specific feasible implementation, it further includes:

[0029] Using a dynamic key management system to generate temporary keys based on file content and timestamps and automatically destroy them after each processing is completed.

[0030] In a third aspect, an electronic device is provided. The electronic device includes a processor, the processor is coupled to a memory, and at least one computer program is stored in the memory. The at least one computer program is loaded and executed by the processor so that the electronic device implements any one of the high-compression and high-encryption security methods for unstructured files.

[0031] In the above technical solution, by setting an intelligent chunking module for dynamically partitioning a file into chunks according to the file content type to obtain file chunks; a hybrid compression engine for matching an optimal compression algorithm combination to the file chunks and performing compression processing; and a dynamic hierarchical encryption module for encrypting the file chunks, the compression efficiency and security of unstructured files are improved.

[0032] In a fourth aspect, a computer-readable storage medium is provided. At least one computer program is stored in the computer-readable storage medium and is loaded and executed by a processor to enable the computer-readable storage medium to implement any one of the high-compression and high-encryption security methods for unstructured files.

[0033] In the above technical solution, by setting an intelligent chunking module for dynamically partitioning a file into chunks according to the file content type to obtain file chunks; a hybrid compression engine for matching an optimal compression algorithm combination to the file chunks and performing compression processing; and a dynamic hierarchical encryption module for encrypting the file chunks, the compression efficiency and security of unstructured files are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a structural block diagram of a high-compression and high-encryption security system for unstructured files provided by an embodiment of the present application;

[0035] Figure 2 It is a flow block diagram of a high-compression and high-encryption security method for unstructured files provided by an embodiment of the present application;

[0036] Figure 3 It is a specific flowchart of a high-compression and high-encryption security method for unstructured files provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0037] The present application will be further described in detail below with reference to the drawings and embodiments. Through these descriptions, the features and advantages of the present application will become more clearly defined.

[0038] Here, the special term "exemplary" means "serving as an example, embodiment, or illustration". Any embodiment described as "exemplary" here does not have to be construed as superior to or better than other embodiments. Although various aspects of the embodiments are shown in the drawings, the drawings do not have to be drawn to scale unless otherwise specified.

[0039] In addition, the technical features involved in different embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0040] To facilitate the understanding of the high-compression and high-encryption security system and method for unstructured files provided by the embodiments of the present application, the application scenario thereof will be described first. The high-compression and high-encryption security system and method for unstructured files provided by the embodiments of the present application are used to improve the compression efficiency and security of unstructured files. The existing compression and encryption technologies for unstructured files have the following disadvantages: low compression efficiency: The compression ratio of traditional algorithms (such as ZIP, RAR) for unstructured files is limited, especially for mixed-content files (such as PDFs containing text and images). Insufficient encryption security: A single encryption key is easily cracked, and compression after encryption may lead to an increase in data redundancy. Risk of metadata leakage: The existing technologies ignore the separate protection of file metadata (such as creation time, author information), which is easily used for side-channel attacks. Slow processing speed: Encryption and compression are performed step by step, resulting in excessive time consumption when processing large files. Therefore, the embodiments of the present application provide a high-compression and high-encryption security system and method for unstructured files to improve the compression efficiency and security of unstructured files. The following will be described in detail with specific drawings by way of examples.

[0041] Reference Figures 1 to 3 , Figure 1 is the structural block diagram of the high-compression and high-encryption security system for unstructured files provided by the embodiments of the present application; Figure 2 is the flow block diagram of the high-compression and high-encryption security method for unstructured files provided by the embodiments of the present application; Figure 3 is the specific flow chart of the high-compression and high-encryption security method for unstructured files provided by the embodiments of the present application.

[0042] In Figure 1 , the embodiments of the present application provide a high-compression and high-encryption security system for unstructured files, including:

[0043] An intelligent chunking module, configured to dynamically divide a file into chunks according to the file content type to obtain file chunks;

[0044] A hybrid compression engine, configured to match an optimal compression algorithm combination for the file chunks and perform compression processing;

[0045] A dynamic hierarchical encryption module, configured to perform encryption processing on the file chunks.

[0046] In the above technical solution, by setting an intelligent chunking module, configured to dynamically divide a file into chunks according to the file content type to obtain file chunks; a hybrid compression engine, configured to match an optimal compression algorithm combination for the file chunks and perform compression processing; a dynamic hierarchical encryption module, configured to perform encryption processing on the file chunks; the compression efficiency and security of unstructured files are improved.

[0047] In a specific feasible implementation, it further includes:

[0048] A metadata protection module for obfuscating and encrypting file metadata and storing it separately.

[0049] In a specific implementable embodiment, it further includes:

[0050] A dynamic key management system for generating a temporary key based on the file content and timestamp and automatically destroying it after each processing is completed.

[0051] In a specific implementable embodiment, AES-256 is used to encrypt critical blocks, and ChaCha20 is used to encrypt non-critical blocks.

[0052] In a specific implementable embodiment, the optimal compression algorithm combination includes:

[0053] Combined LZ77 and Huffman coding are used to compress text data;

[0054] Combined WebP and quantization are used to compress image data.

[0055] Specifically, the unstructured file high-compression and high-encryption security system includes:

[0056] An intelligent chunking module: Dynamically divides blocks according to the file content type (such as text segments, image areas, video frames).

[0057] A hybrid compression engine: Matches the optimal compression algorithm combination for different blocks (such as LZ77 + Huffman coding for text, WebP + quantization compression for images).

[0058] A dynamic hierarchical encryption module: Uses AES-256 to encrypt critical blocks (such as document titles, sensitive areas of images), and ChaCha20 to encrypt non-critical blocks.

[0059] A metadata protection module: Obfuscates and encrypts file metadata (such as file names, attributes) and stores it separately.

[0060] A dynamic key management system: Generates a temporary key based on the file content and timestamp and automatically destroys it after each processing is completed.

[0061] Furthermore, the intelligent chunking module adopts content-aware chunking and compression technology, including:

[0062] 1. A semantic analysis and chunking unit for:

[0063] Text files: Segmented by semantic paragraphs (such as each paragraph as a block);

[0064] Image files: Chunked according to visual importance (such as the face area as a high-priority block);

[0065] Video file: Split by key frames (I-frames) and non-key frames.

[0066] 2. Adaptive compression strategy unit, for:

[0067] Highly repetitive text: Enable dictionary compression;

[0068] High-resolution images: Use lossy compression + lossless metadata retention;

[0069] Video dynamic regions: Use inter-frame difference compression.

[0070] Furthermore, the dynamic hierarchical encryption module adopts a hierarchical dynamic encryption mechanism, including:

[0071] 1. Encryption strength grading unit, for:

[0072] For core data, adopt the AES-256-GCM encryption algorithm with a key length of 256 bits, used for document titles and sensitive image regions;

[0073] For ordinary data, adopt the ChaCha20-Poly1305 encryption algorithm with a key length of 256 bits, used for body text and background video frames;

[0074] For metadata, adopt the SM4 + confusion algorithm with a key length of 128 bits, used for file names and creation times.

[0075] 2. Dynamic key generation unit, for:

[0076] Generate a temporary key based on the file hash value and timestamp, and destroy it immediately after encryption;

[0077] Support key segment management, and different blocks use independent sub-keys.

[0078] Furthermore, it also includes a compression and encryption co-optimization module, including:

[0079] Embedded encryption unit, for encrypting block data in real time during the compression process to avoid secondary reading and writing;

[0080] Parallel processing unit, for processing text, image, and video blocks in a pipelined manner to increase the throughput by more than 30%.

[0081] Furthermore, the metadata protection module includes:

[0082] Metadata separation and storage unit, for encrypting and storing metadata in an independent container, physically isolated from the main file;

[0083] False metadata injection unit, for generating random false metadata (such as forged creation times) to interfere with reverse analysis.

[0084] In a specific feasible implementation, encrypt and compress a corporate financial report (PDF file) containing sensitive charts. The specific steps are as follows:

[0085] 1. Intelligent chunking:

[0086] It is divided into text chunks (financial data), image chunks (bar charts, line charts), and metadata (file name "Q4_Report.pdf").

[0087] 2. Hybrid compression:

[0088] Text chunks: LZ77 compression (compression ratio of 70%);

[0089] Image chunks: WebP lossy compression (compression ratio of 50%), retaining chart coordinate metadata;

[0090] Metadata: Base64 encoding + obfuscation compression.

[0091] 3. Hierarchical encryption:

[0092] Text core data (such as amount figures): AES-256 encryption;

[0093] Image chunks: ChaCha20 encryption;

[0094] Metadata: SM4 encryption and injecting a false creation time of "2020-01-01".

[0095] 4. Dynamic key management:

[0096] Generate temporary keys Key_A (for text), Key_B (for images), and Key_C (for metadata);

[0097] After encryption is completed, all keys are destroyed, and only the ciphertext is retained.

[0098] 5. Output and storage:

[0099] The compressed and encrypted file is stored in chunks, and the metadata is stored in an independent encrypted container.

[0100] In this embodiment, the compression ratio is increased by 40% (compared with traditional ZIP encryption); the time to resist brute-force cracking is extended from 1 year to more than 10 years; the risk of metadata leakage is reduced by 90%.

[0101] In the above technical solution, the beneficial effects include:

[0102] Content-aware chunking: Improves the compression efficiency of unstructured files and adapts to multi-type data mixing scenarios.

[0103] Dynamic hierarchical encryption: Balances security and performance through differential encryption strategies.

[0104] Active Metadata Protection: Confusion and Separation Storage Technology Effectively Resists Side-Channel Attacks.

[0105] Collaborative Processing Architecture: Compression and Encryption Processes are Deeply Integrated to Reduce Processing Latency.

[0106] In Figure 2 this application embodiment provides a high-compression and high-encryption security method for unstructured files, including the following steps:

[0107] Use the intelligent chunking module to dynamically divide the file into blocks according to the file content type to obtain file blocks;

[0108] Use the hybrid compression engine to match the optimal compression algorithm combination for the file blocks and perform compression processing;

[0109] Use the dynamic hierarchical encryption module to encrypt the file blocks.

[0110] In the above technical solution, by setting an intelligent chunking module for dynamically dividing the file into blocks according to the file content type to obtain file blocks; a hybrid compression engine for matching the optimal compression algorithm combination for the file blocks and performing compression processing; a dynamic hierarchical encryption module for encrypting the file blocks; the compression efficiency and security of unstructured files are improved.

[0111] In a specific feasible implementation, it further includes:

[0112] Use the metadata protection module to confuse and encrypt the file metadata and store it separately.

[0113] In a specific feasible implementation, it further includes:

[0114] Use the dynamic key management system to generate a temporary key based on the file content and timestamp and automatically destroy it after each processing is completed.

[0115] This application embodiment also provides an electronic device, the electronic device includes a processor, the processor is coupled with a memory, and at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor so that the electronic device implements any one of the high-compression and high-encryption security methods for unstructured files.

[0116] In the above technical solution, by setting an intelligent chunking module for dynamically dividing the file into blocks according to the file content type to obtain file blocks; a hybrid compression engine for matching the optimal compression algorithm combination for the file blocks and performing compression processing; a dynamic hierarchical encryption module for encrypting the file blocks; the compression efficiency and security of unstructured files are improved.

[0117] The embodiments of the present application also provide a computer-readable storage medium. At least one computer program is stored in the computer-readable storage medium. The at least one computer program is loaded and executed by a processor so that the computer-readable storage medium implements any of the non-structured file high-compression and high-encryption security methods.

[0118] In the above technical solution, by setting an intelligent chunking module for dynamically partitioning a file into chunks according to the file content type to obtain file chunks; a hybrid compression engine for matching an optimal compression algorithm combination for the file chunks and performing compression processing; and a dynamic hierarchical encryption module for encrypting the file chunks, the compression efficiency and security of non-structured files are improved.

[0119] Those skilled in the art of the present technology know that the present application can be implemented as a system, a method, or a computer program product.

[0120] Therefore, the present disclosure can be specifically implemented in the following forms: it can be completely hardware, can be completely software (including firmware, resident software, microcode, etc.), or can also be a combination of hardware and software, which is generally referred to as "circuit", "module" or "system" in this article. In addition, in some embodiments, the present application can also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable media contain computer-readable program codes.

[0121] Any combination of one or more computer-readable media can be adopted. The computer-readable media can be computer-readable signal media or computer-readable storage media. The computer-readable storage media can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage media can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, apparatus, or device.

[0122] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application. On this basis, various substitutions and improvements can be made to the present application, and all of these fall within the protection scope of the present application.

Claims

1. A high-compression and high-encryption security system for unstructured files, characterized in that, Comprising: An intelligent chunking module, used to dynamically divide a file into blocks according to the file content type, obtaining file blocks; The intelligent chunking module adopts content-aware chunking compression technology, including: A semantic analysis chunking unit, used to: Segment text files by semantic paragraphs; Chunk image files according to visual importance; Segment video files into key frames and non-key frames; An adaptive compression strategy unit, used to: Enable dictionary compression for highly repetitive text; Adopt lossy compression + lossless metadata retention for high-resolution images; Use inter-frame difference compression for dynamic regions of videos; A hybrid compression engine, used to match the optimal compression algorithm combination for the file blocks and perform compression processing; The optimal compression algorithm combination includes: Compress text data using a combination of LZ77 and Huffman coding; Compress image data using a combination of WebP and quantization; A dynamic hierarchical encryption module, used to encrypt the file blocks; The dynamic hierarchical encryption module adopts a hierarchical dynamic encryption mechanism, including: An encryption strength grading unit, used to: For core data, adopt the AES-256-GCM encryption algorithm with a key length of 256 bits, used for document titles and sensitive image regions; For ordinary data, adopt the ChaCha20-Poly1305 encryption algorithm with a key length of 256 bits, used for body text and background video frames; For metadata, adopt the SM4 + confusion algorithm with a key length of 128 bits, used for file names and creation times; A dynamic key generation unit, used to: Generate a temporary key based on the file hash value and timestamp, and immediately destroy it after encryption; support segmented key management, and different blocks use independent sub-keys; It also includes a compression and encryption co-optimization module, including: An embedded encryption unit, used to encrypt block data in real time during the compression process; A parallel processing unit, used for pipeline processing of text, image, and video blocks; It also includes: A metadata protection module, used to confuse and encrypt file metadata and store it separately; The metadata protection module includes: A metadata separation storage unit, used to encrypt and store metadata in an independent container, physically isolated from the main file; A false metadata injection unit, used to generate random false metadata.

2. The high-compression and high-encryption security system for unstructured files according to claim 1, characterized in that, It also includes: A dynamic key management system, used to generate a temporary key based on the file content and timestamp, and automatically destroy it after each processing is completed.

3. The high-compression and high-encryption security system for unstructured files according to claim 2, characterized in that, Encrypt critical blocks using AES-256 and non-critical blocks using ChaCha20.

4. A high-compression and high-encryption security method for unstructured files, characterized in that, Including the following steps: Use the intelligent chunking module to dynamically divide the file into blocks according to the file content type, obtaining file blocks; The intelligent chunking module adopts content-aware chunking compression technology, including: A semantic analysis chunking unit, used to: Segment text files by semantic paragraphs; Chunk image files according to visual importance; Segment video files into key frames and non-key frames; An adaptive compression strategy unit, used to: Enable dictionary compression for highly repetitive text; Adopt lossy compression + lossless metadata retention for high-resolution images; Use inter-frame difference compression for dynamic regions of videos; Use a hybrid compression engine to match the optimal compression algorithm combination for the file block and perform compression processing; The optimal compression algorithm combination includes: Compress text data using a combination of LZ77 and Huffman coding; Compress image data using a combination of WebP and quantization; Use a dynamic hierarchical encryption module to encrypt the file block; The dynamic hierarchical encryption module adopts a hierarchical dynamic encryption mechanism, including: An encryption strength grading unit for: For core data, use the AES-256-GCM encryption algorithm with a key length of 256 bits for document titles and sensitive image areas; For ordinary data, use the ChaCha20-Poly1305 encryption algorithm with a key length of 256 bits for the body text and background video frames; For metadata, use the SM4 + confusion algorithm with a key length of 128 bits for file names and creation times; A dynamic key generation unit for: Generate a temporary key based on the file hash value and timestamp, and destroy it immediately after encryption; support key segment management, and different blocks use independent sub-keys; It also includes a compression and encryption collaborative optimization module, including: An embedded encryption unit for encrypting block data in real time during the compression process; A parallel processing unit for processing text, image, and video blocks in a pipeline; It also includes: Use a metadata protection module to obfuscate and encrypt the file metadata and store it separately; The metadata protection module includes: A metadata separation storage unit for encrypting and storing the metadata in an independent container, physically isolated from the main file; A false metadata injection unit for generating random false metadata.

5. The high-compression and high-encryption security method for unstructured files according to claim 4, wherein It also includes: Use a dynamic key management system to generate a temporary key based on the file content and timestamp, and automatically destroy it after each processing is completed.

6. An electronic device, characterized in that, The electronic device includes a processor, the processor is coupled to a memory, and at least one computer program is stored in the memory. The at least one computer program is loaded and executed by the processor so that the electronic device implements the unstructured file high-compression and high-encryption security method according to any one of claims 4 to 5.

7. A computer-readable storage medium, characterized in that, At least one computer program is stored in the computer-readable storage medium. The at least one computer program is loaded and executed by a processor so that the computer-readable storage medium implements the unstructured file high-compression and high-encryption security method according to any one of claims 4 to 5.

Images