High-compression and high-encryption security system and method for unstructured file

Abstract

The invention provides an unstructured file high-compression and high-encryption security system and method, and the system comprises an intelligent partitioning module which is used for carrying out the dynamic partitioning of a file according to the content type of the file, and obtaining file blocks; the hybrid compression engine is used for matching the file blocks with an optimal compression algorithm combination and performing compression processing; and the dynamic hierarchical encryption module is used for encrypting the file blocks. According to the technical scheme, the compression efficiency and safety of the unstructured file are improved.

Description

Background Art

[0002] The existing compression and encryption technologies for unstructured files have the following disadvantages: Low compression efficiency: Traditional algorithms (such as ZIP, RAR) have limited compression ratios for unstructured files, especially poor effects on mixed-content files (such as PDFs containing text and images).

[0003] Insufficient encryption security: A single encryption key is easily cracked, and compression after encryption may lead to an increase in data redundancy.

[0004] Risk of metadata leakage: Existing technologies ignore the separate protection of file metadata (such as creation time, author information), which is easily used for side-channel attacks.

[0005] Slow processing speed: Encryption and compression are executed step by step, resulting in excessive time consumption when processing large files. Summary of the Invention

[0006] This application provides a high-compression and high-encryption security system and method for unstructured files to improve the compression efficiency and security of unstructured files.

[0007] In the first aspect, a high-compression and high-encryption security system for unstructured files is provided, including: An intelligent chunking module for dynamically partitioning a file into chunks according to the file content type to obtain file chunks; A hybrid compression engine for matching the optimal compression algorithm combination for the file chunks and performing compression processing; A dynamic hierarchical encryption module for encrypting the file chunks.

[0008] In the above technical solution, by setting an intelligent chunking module for dynamically partitioning a file into chunks according to the file content type to obtain file chunks; a hybrid compression engine for matching the optimal compression algorithm combination for the file chunks and performing compression processing; a dynamic hierarchical encryption module for encrypting the file chunks; the compression efficiency and security of unstructured files are improved.

[0009] In a specific feasible implementation, it further includes: A metadata protection module for obfuscating and encrypting file metadata and storing it separately.

[0010] In a specific feasible implementation, it further includes: A dynamic key management system for generating a temporary key based on the file content and timestamp and automatically destroying it after each processing is completed.

[0011] In a specific feasible implementation, AES-256 is used to encrypt critical blocks, and ChaCha20 is used to encrypt non-critical blocks.

[0012] In a specific feasible implementation, the optimal compression algorithm combination includes: LZ77 and Huffman coding combination is used to compress text data; WebP and quantization combination is used to compress image data.

[0013] In a second aspect, a high-compression and high-encryption security method for unstructured files is provided, including the following steps: Use the intelligent chunking module to dynamically divide the file into blocks according to the file content type to obtain file blocks; Use the hybrid compression engine to match the optimal compression algorithm combination for the file blocks and perform compression processing; Use the dynamic hierarchical encryption module to encrypt the file blocks.

[0014] In the above technical solution, by setting an intelligent chunking module for dynamically dividing the file into blocks according to the file content type to obtain file blocks; a hybrid compression engine for matching the optimal compression algorithm combination for the file blocks and performing compression processing; a dynamic hierarchical encryption module for encrypting the file blocks; the compression efficiency and security of unstructured files are improved.

[0015] In a specific feasible implementation, it further includes: Use the metadata protection module to obfuscate and encrypt the file metadata and store it separately.

[0016] In a specific feasible implementation, it further includes: Use the dynamic key management system to generate a temporary key based on the file content and timestamp and automatically destroy it after each processing is completed.

[0017] In a third aspect, an electronic device is provided. The electronic device includes a processor, the processor is coupled with a memory, and at least one computer program is stored in the memory. The at least one computer program is loaded and executed by the processor to enable the electronic device to implement any one of the high-compression and high-encryption security methods for unstructured files.

[0018] In the above technical solution, by setting an intelligent chunking module for dynamically dividing the file into blocks according to the file content type to obtain file blocks; a hybrid compression engine for matching the optimal compression algorithm combination for the file blocks and performing compression processing; a dynamic hierarchical encryption module for encrypting the file blocks; the compression efficiency and security of unstructured files are improved.

[0019] In a fourth aspect, a computer-readable storage medium is provided. At least one computer program is stored in the computer-readable storage medium. The at least one computer program is loaded and executed by a processor so that the computer-readable storage medium implements any one of the non-structured file high-compression and high-encryption security methods.

[0020] In the above technical solution, by setting up an intelligent chunking module for dynamically partitioning a file into blocks according to the file content type to obtain file blocks; a hybrid compression engine for matching an optimal compression algorithm combination to the file blocks and performing compression processing; and a dynamic hierarchical encryption module for encrypting the file blocks, the compression efficiency and security of non-structured files are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a structural block diagram of a non-structured file high-compression and high-encryption security system provided by an embodiment of the present application; Figure 2 It is a flowchart of a non-structured file high-compression and high-encryption security method provided by an embodiment of the present application; Figure 3 It is a specific flowchart of a non-structured file high-compression and high-encryption security method provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] The present application will be further described in detail below with reference to the drawings and embodiments. Through these descriptions, the features and advantages of the present application will become more clearly defined.

[0023] The special term "exemplary" here means "serving as an example, an embodiment or illustrative". Any embodiment described as "exemplary" here does not have to be construed as superior or better than other embodiments. Although various aspects of the embodiments are shown in the drawings, the drawings do not have to be drawn to scale unless otherwise specified.

[0024] In addition, the technical features involved in different embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0025] To facilitate the understanding of the high-compression and high-encryption security system and method for unstructured files provided by the embodiments of the present application, the application scenario thereof will be described first. The high-compression and high-encryption security system and method for unstructured files provided by the embodiments of the present application are used to improve the compression efficiency and security of unstructured files. The existing compression and encryption technologies for unstructured files have the following disadvantages: Low compression efficiency: Traditional algorithms (such as ZIP, RAR) have limited compression rates for unstructured files, especially poor effects on mixed-content files (such as PDFs containing text and images). Insufficient encryption security: A single encryption key is easily cracked, and compression after encryption may lead to an increase in data redundancy. Risk of metadata leakage: Existing technologies ignore the separate protection of file metadata (such as creation time, author information), which is easily used for side-channel attacks. Slow processing speed: Encryption and compression are performed step by step, resulting in excessive time consumption when processing large files. Therefore, the embodiments of the present application provide a high-compression and high-encryption security system and method for unstructured files to improve the compression efficiency and security of unstructured files. The following will be described in detail with specific drawings by way of examples.

[0026] Reference Figures 1 to 3 , Figure 1 is the structural block diagram of the high-compression and high-encryption security system for unstructured files provided by the embodiments of the present application; Figure 2 is the flow block diagram of the high-compression and high-encryption security method for unstructured files provided by the embodiments of the present application; Figure 3 is the specific flow chart of the high-compression and high-encryption security method for unstructured files provided by the embodiments of the present application.

[0027] In Figure 1 , the embodiments of the present application provide a high-compression and high-encryption security system for unstructured files, including: An intelligent chunking module, configured to dynamically divide a file into chunks according to the file content type to obtain file chunks; A hybrid compression engine, configured to match the optimal compression algorithm combination for the file chunks and perform compression processing; A dynamic hierarchical encryption module, configured to perform encryption processing on the file chunks.

[0028] In the above technical solution, by setting an intelligent chunking module, configured to dynamically divide a file into chunks according to the file content type to obtain file chunks; a hybrid compression engine, configured to match the optimal compression algorithm combination for the file chunks and perform compression processing; a dynamic hierarchical encryption module, configured to perform encryption processing on the file chunks; the compression efficiency and security of unstructured files are improved.

[0029] In a specific feasible implementation, it further includes: A metadata protection module, configured to perform obfuscation encryption on file metadata and store it separately.

[0030] In a specific feasible implementation, it further includes: A dynamic key management system, which is used to generate a temporary key based on the file content and timestamp, and automatically destroy it after each processing is completed.

[0031] In a specific feasible implementation, the critical blocks are encrypted using AES-256, and the non-critical blocks are encrypted using ChaCha20.

[0032] In a specific feasible implementation, the optimal compression algorithm combination includes: For text data, it is compressed using a combination of LZ77 and Huffman coding; For image data, it is compressed using a combination of WebP and quantization.

[0033] Specifically, the high-compression and high-encryption security system for unstructured files includes: An intelligent chunking module: Dynamically divides the blocks according to the file content type (such as text segments, image areas, video frames).

[0034] A hybrid compression engine: Matches the optimal compression algorithm combination for different blocks (such as using LZ77 + Huffman coding for text, and WebP + quantization compression for images).

[0035] A dynamic hierarchical encryption module: Encrypts the critical blocks (such as document titles, sensitive areas of images) using AES-256, and encrypts the non-critical blocks using ChaCha20.

[0036] A metadata protection module: Confuses and encrypts the file metadata (such as file names, attributes) and stores them separately.

[0037] A dynamic key management system: Generates a temporary key based on the file content and timestamp, and automatically destroys it after each processing is completed.

[0038] Furthermore, the intelligent chunking module adopts content-aware chunking compression technology, including: 1. A semantic analysis chunking unit, which is used for: Text files: Split by semantic paragraphs (such as each paragraph as a block); Image files: Chunked according to visual importance (such as the face area as a high-priority block); Video files: Split into key frames (I-frames) and non-key frames.

[0039] 2. An adaptive compression strategy unit, which is used for: Highly repetitive text: Enable dictionary compression; High-resolution images: Adopt lossy compression + lossless metadata retention; Video dynamic areas: Use inter-frame difference compression.

[0040] Furthermore, the dynamic hierarchical encryption module adopts a hierarchical dynamic encryption mechanism, including: 1. Encryption strength grading unit, used for: For core data, the AES-256-GCM encryption algorithm is adopted, with a key length of 256 bits, used for document titles and sensitive image areas; For ordinary data, the ChaCha20-Poly1305 encryption algorithm is adopted, with a key length of 256 bits, used for body text and background video frames; For metadata, the SM4 + confusion algorithm is adopted, with a key length of 128 bits, used for file names and creation times.

[0041] 2. Dynamic key generation unit, used for: Generate a temporary key based on the file hash value and timestamp, and destroy it immediately after encryption; Support key segment management, and different blocks use independent sub-keys.

[0042] Furthermore, it also includes a compression and encryption collaborative optimization module, including: Embedded encryption unit, used for real-time encryption of block data during the compression process to avoid secondary reading and writing; Parallel processing unit, used for pipeline processing of text, image, and video blocks, increasing the throughput by more than 30%.

[0043] Furthermore, the metadata protection module includes: Metadata separation and storage unit, used for encrypting and storing metadata in an independent container, physically isolated from the main file; False metadata injection unit, used for generating random false metadata (such as forged creation time) to interfere with reverse analysis.

[0044] In a specific feasible implementation, the steps for encrypting and compressing a corporate financial report (PDF file) containing sensitive charts are as follows: 1. Intelligent chunking: Split into text blocks (financial data), image blocks (bar charts, line charts), and metadata (file name "Q4_Report.pdf").

[0045] 2. Hybrid compression: Text blocks: LZ77 compression (compression rate 70%); Image blocks: WebP lossy compression (compression rate 50%), retaining chart coordinate metadata; Metadata: Base64 encoding + confusion compression.

[0046] 3. Hierarchical encryption: Core data of the text (such as amount figures): AES-256 encryption; Image blocks: ChaCha20 encryption; Metadata: SM4 encryption and injection of false creation time "2020-01-01".

[0047] 4. Dynamic key management: Generate temporary keys Key_A (for text), Key_B (for images), Key_C (for metadata); Destroy all keys after encryption is completed, and only keep the ciphertext.

[0048] 5. Output and storage: The compressed and encrypted file is stored in chunks, and the metadata is stored in an independent encrypted container.

[0049] In this embodiment, the compression ratio is increased by 40% (compared with traditional ZIP encryption); the anti-brute-force cracking time is extended from 1 year to more than 10 years; the risk of metadata leakage is reduced by 90%.

[0050] In the above technical solution, the beneficial effects include: Content-aware chunking: Improve the compression efficiency of unstructured files and adapt to multi-type data mixing scenarios.

[0051] Dynamic hierarchical encryption: Balance security and performance through differential encryption strategies.

[0052] Active protection of metadata: The techniques of obfuscation and separate storage effectively resist side-channel attacks.

[0053] Collaborative processing architecture: Deep integration of compression and encryption processes to reduce processing latency.

[0054] In Figure 2 this application provides a high-compression and high-encryption security method for unstructured files, including the following steps: Use the intelligent chunking module to dynamically divide the file into blocks according to the file content type to obtain file blocks; Use the hybrid compression engine to match the optimal compression algorithm combination for the file blocks and perform compression processing; Use the dynamic hierarchical encryption module to encrypt the file blocks.

[0055] In the above technical solution, by setting an intelligent chunking module for dynamically dividing the file into blocks according to the file content type to obtain file blocks; a hybrid compression engine for matching the optimal compression algorithm combination for the file blocks and performing compression processing; and a dynamic hierarchical encryption module for encrypting the file blocks, the compression efficiency and security of unstructured files are improved.

[0056] In a specific implementable embodiment, it further includes: The metadata protection module is used to obfuscate and encrypt the file metadata and store it separately.

[0057] In a specific implementable embodiment, it further includes: The dynamic key management system is used to generate a temporary key based on the file content and timestamp, and automatically destroy it after each processing is completed.

[0058] The embodiment of the present application also provides an electronic device, which includes a processor, the processor is coupled to a memory, and at least one computer program is stored in the memory. The at least one computer program is loaded and executed by the processor to enable the electronic device to implement any one of the non-structured file high-compression and high-encryption security methods.

[0059] In the above technical solution, by setting an intelligent chunking module for dynamically dividing a file into chunks according to the file content type to obtain file chunks; a hybrid compression engine for matching an optimal compression algorithm combination for the file chunks and performing compression processing; and a dynamic hierarchical encryption module for encrypting the file chunks; the compression efficiency and security of non-structured files are improved.

[0060] The embodiment of the present application also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to enable the computer-readable storage medium to implement any one of the non-structured file high-compression and high-encryption security methods.

[0061] In the above technical solution, by setting an intelligent chunking module for dynamically dividing a file into chunks according to the file content type to obtain file chunks; a hybrid compression engine for matching an optimal compression algorithm combination for the file chunks and performing compression processing; and a dynamic hierarchical encryption module for encrypting the file chunks; the compression efficiency and security of non-structured files are improved.

[0062] Those skilled in the art of the technical field know that the present application can be implemented as a system, a method, or a computer program product.

[0063] Therefore, the present disclosure can be specifically implemented in the following forms: it can be completely hardware, can be completely software (including firmware, resident software, microcode, etc.), or can be a combination of hardware and software, generally referred to as "circuit", "module" or "system" in this article. In addition, in some embodiments, the present application can also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable media contains computer-readable program code.

[0064] Any combination of one or more computer-readable media may be employed. The computer-readable media may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present document, a computer-readable storage medium may be any tangible medium that contains or stores a program which can be used by or in connection with an instruction execution system, apparatus, or device.

[0065] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application. On this basis, various substitutions and improvements can be made to the present application, and all of these fall within the protection scope of the present application.

Claims

1. A high compression and high encryption security system for unstructured files, characterized in that: include: Intelligent block division module, used to dynamically divide files into blocks according to file content types to obtain file blocks; A hybrid compression engine, used for matching the optimal compression algorithm combination to the file blocks and performing compression processing; The dynamic layered encryption module is used to encrypt the file blocks.

2. The unstructured file high compression and high encryption security system according to claim 1 is characterized in that: Also includes: The metadata protection module is used to encrypt and obfuscate file metadata and store it separately.

3. The unstructured file high compression and high encryption security system according to claim 2, characterized in that: Also includes: A dynamic key management system is used to generate temporary keys based on file content and timestamps, and automatically destroy them after each transaction is completed.

4. The unstructured file high compression and high encryption security system according to claim 3 is characterized in that: AES-256 is used to encrypt critical blocks, and ChaCha20 is used to encrypt non-critical blocks.

5. The unstructured file high compression and high encryption security system according to claim 4, characterized in that: The optimal compression algorithm combination includes: The text data is compressed using a combination of LZ77 and Huffman coding; The image data is compressed using a combination of WebP and quantization.

6. A high compression and high encryption security method for unstructured files, characterized in that: The following steps are involved: Using the intelligent block module to dynamically divide the file into blocks according to the file content type to obtain file blocks; Using a hybrid compression engine to match the optimal compression algorithm combination to the file blocks and perform compression processing; The file blocks are encrypted using a dynamic layered encryption module.

7. The high compression and high encryption security method for unstructured files according to claim 6 is characterized in that: Also includes: The metadata protection module is used to obfuscate and encrypt file metadata and store it separately.

8. The high compression and high encryption security method for unstructured files according to claim 7 is characterized in that: Also includes: A dynamic key management system is used to generate temporary keys based on file content and timestamps, and automatically destroy them after each transaction is completed.

9. An electronic device, characterized in that: The electronic device includes a processor, the processor is coupled to a memory, the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor so that the electronic device implements the high compression and high encryption security method for unstructured files as described in any one of claims 6 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by the processor so that the computer-readable storage medium implements the high-compression and high-encryption security method for unstructured files as described in any one of claims 6 to 8.

Images