File mixed block encrypted storage generation method supporting dynamic random reading

Abstract

The invention discloses a generation method for file mixed block encrypted storage supporting dynamic random reading, and belongs to the field of data storage. The generation method for file mixed block encrypted storage supporting dynamic random reading comprises the following steps of: partitioning an original file into a plaintext logic block set according to a given strategy; an independent key is generated for each plaintext logic block and encrypted, and an encrypted logic block is generated; extracting a'complementation factor ', and generating a'incomplete encryption logic block'; for each'incomplete encryption logic block ', generating a'redundant physical block' set by using a redundancy algorithm and storing the'redundant physical block 'set; and the whole process records corresponding to'metadata 'information encryption storage. When a file is read, a target plaintext logic block set is positioned through block mapping according to meta information, a corresponding redundant physical block set is obtained, and plaintext logic blocks are restored through decoding, verification, complementation factor backfilling and decryption. The method supports random reading and writing, gives consideration to safety and efficiency, and improves the storage fault tolerance.

Description

Technical Field

[0001] This invention belongs to the field of computer data storage and focuses on improving the security of data storage. Background Technology

[0002] Currently, the main means of improving data storage security is overall encrypted storage. This method often uses a single key to encrypt the entire file or all blocks. The key granularity is coarse, and once it is leaked or cracked, the entire file / all blocks of data will be exposed. It is also difficult to achieve fine-grained dynamic random access. At the same time, it has obvious shortcomings in terms of fault tolerance and redundancy, performance and security balance and scalability. Summary of the Invention

[0003] The purpose of this invention is to improve the security of data storage in storage systems and to address the shortcomings of existing overall encrypted storage methods in terms of key management, access efficiency, fault tolerance and redundancy, and performance-security balance. This invention provides a hybrid block-based encrypted storage method for files that supports dynamic random access.

[0004] This invention employs a design that combines fine-grained key management with block encryption. It configures independent keys for different blocks of a file to achieve fine-grained encryption, and is equipped with a dynamic key verification mechanism. It also integrates a redundant fault-tolerant architecture that is adaptable to multiple scenarios and a combination strategy of efficient encryption algorithms, thus effectively improving the security of data storage.

[0005] The file hybrid block encryption storage method supporting dynamic random reading provided by this invention includes: dividing the original file into blocks according to a given strategy to generate a set of "plaintext logical blocks"; using an encryption algorithm to generate an independent key for each "plaintext logical block" and encrypting it to generate an "encrypted logical block"; extracting a "completion factor" to generate a "partial encrypted logical block"; for each "partial encrypted logical block", using a redundancy algorithm to generate and store a set of "redundant physical blocks"; recording the corresponding "metadata" information for each step above and encrypting and storing it. When reading the file, the target "plaintext logical block" block number set is located based on the "metadata" and through block mapping, and the corresponding "redundant physical block" set is further obtained, the "redundant physical block" is decoded and verified, the "completion factor" is filled back, and the data is decrypted and extracted to generate a "plaintext logical block". When the file is modified, only the target "plaintext logical block" set is processed and the corresponding "redundant physical block" set and "metadata" are updated.

[0006] The specific technical solution of this invention is as follows:

[0007] A method for generating files with hybrid block encryption that supports dynamic random access is proposed. This method employs an architecture of "double block partitioning + independent key encryption + extraction of 'completion factors' + redundant encoding + dynamic verification," and the specific steps are as follows:

[0008] 1. Original File Preprocessing and Logical Blocking: The original file to be stored is format-validated, its hash value is calculated, and it is divided into n "plaintext logical blocks" { L1, L2, ..., L...} according to a given block-splitting strategy. n The corresponding "metadata" fragment is generated; this "metadata" fragment includes: file identifier and verification information, "plaintext logical block" verification information, block partitioning algorithm and block offset mapping relationship;

[0009] 2. Independent key generation and encryption of "plaintext logic blocks": Generate an independent encryption key for each "plaintext logic block", encrypt the "plaintext logic block" to obtain an "encrypted logic block", and generate a corresponding "metadata" fragment; the "metadata" fragment includes: encryption algorithm and parameters, key information and "encrypted logic block" verification information;

[0010] 3. Extracting "Completion Factors" to Generate "Incomplete Encrypted Logic Blocks": Extracting a portion of data fragments from the "Encrypted Logic Blocks" as "Completion Factors"; then replacing the extracted fragments in the "Encrypted Logic Blocks" with randomly filled data to form "Incomplete Encrypted Logic Blocks" and generating corresponding "Metadata" fragments; these "Metadata" fragments include: "Completion Factor" information and "Incomplete Encrypted Logic Block" verification information;

[0011] 4. For each "incomplete encrypted logic block", generate several "redundant physical blocks" through redundant subdivision: Use a preset redundancy algorithm to perform secondary subdivision on each "incomplete encrypted logic block", generating k original data blocks and m redundant verification blocks, where k and m are the parameters set for the redundancy algorithm, representing the number of original data blocks and the number of redundant verification blocks generated, respectively; the original data blocks and redundant verification blocks together form k+m "redundant physical blocks", and generate corresponding "metadata" fragments; the "metadata" fragments include: redundancy algorithm and parameters;

[0012] Fifth, “Redundant Physical Block” Storage: The “redundant physical blocks” are distributed and stored in different local or remote storage media. The storage path or identification address of each “redundant physical block” is recorded, its hash value is calculated, and a corresponding “metadata” fragment is generated. The “metadata” fragment includes: “redundant physical block” storage path and “redundant physical block” verification information.

[0013] Step 6, "Metadata" Construction and Storage: All "metadata" fragments generated in steps 1 to 5 above are aggregated to construct a complete "metadata", and the user's master key is used to encrypt and store the entire "metadata".

[0014] 7. The program execution has ended.

[0015] Furthermore, the segmentation strategy described in step 1 includes, but is not limited to, segmenting by fixed size, segmenting by dynamic size, segmenting by user-defined size, segmenting by data type, or pre-segmenting by access frequency. The segmentation strategy supports multi-dimensional selection and can be combined as needed.

[0016] Furthermore, the "completion factor" mentioned in step 3 is a data segment randomly selected from the "encrypted logic block". Its core function is to further enhance security and prevent data leakage caused by leakage of the "encrypted logic block" and key. Specifically, it is achieved through the following two precise and controllable extraction methods: Method (1) Fixed-length segment extraction, that is, a uniform extraction byte length threshold is set in advance. Starting from the starting position of the "encrypted logic block", the "encrypted logic block" is continuously and without intervals. Method (2) Fixed offset extraction, which is extraction with fixed offsets. That is, a fixed starting offset and extraction interval are set in advance. First, the first data segment of fixed length is extracted from the starting offset position of the "encrypted logic block". Then, after skipping some data according to the preset extraction interval, the next data segment of the same length is extracted. The extraction is completed in this way.

[0017] Furthermore, the "completion factor" extraction operation described in step 3 can also be performed before the encryption in step 2. That is, after dividing the original file into blocks to generate "plaintext logical blocks", the "completion factor" extraction operation is performed first to form "incomplete logical blocks", and then the "incomplete logical blocks" are encrypted. Specifically, the "completion factor" is extracted from the "plaintext logical blocks" by means of continuous extraction of fixed length or extraction with fixed offset and interval, and the position index is recorded. Random data is filled in to form "incomplete logical blocks", and then an independent key is assigned to the "incomplete logical blocks" to complete the encryption, and "metadata" containing the extraction rules is generated simultaneously.

[0018] A file reading method supporting dynamic random access and hybrid block encryption storage is constructed based on the above-described "generation method" and adopts an architecture of "double block partitioning + independent key encryption + extraction of 'completion factor' + redundant encoding + dynamic verification". The specific steps are as follows:

[0019] First, the user inputs the following parameters when reading a file: fd is the unique identifier of the file; buf is the buffer for reading data; range is the range of bytes to read.

[0020] Second, the acquisition and verification of "metadata": the encrypted "metadata" is obtained based on the file's unique identifier fd, and the user's master key is used to decrypt it to obtain the original complete "metadata";

[0021] 3. Block location: Based on the block offset mapping relationship stored in the "metadata", locate the "plaintext logical block" corresponding to the file and the target byte range, and further locate the "redundant physical block";

[0022] 4. Redundant physical block reading and verification: Based on the storage path of the "redundant physical block" recorded in the "metadata", concurrently read any k "redundant physical blocks" corresponding to each target "plaintext logical block", calculate the hash value, and compare it with the "metadata" to verify the integrity, where k is the minimum number of "redundant physical blocks" corresponding to the "plaintext logical block".

[0023] 5. Recovery and verification of “incomplete encrypted logic block”: Using the verified redundant physical block, call the given redundancy encoding algorithm to decode and recover the “incomplete encrypted logic block”, calculate the hash value of the recovered “incomplete encrypted logic block”, and compare it with the hash value in the metadata to verify its integrity;

[0024] 6. "Completion Factor" backfilling and decryption: Obtain the "completion factor" and its corresponding position index from the "metadata", backfill the "completion factor" into the corresponding position of the "incomplete encrypted logic block" to obtain the "encrypted logic block", and use the key recorded in the "metadata" to align and decrypt to obtain the "plaintext logic block".

[0025] 7. Complete file integration and verification: Integrate and restore the "plaintext logical blocks" according to the block order defined by the "metadata" and perform the final integrity verification;

[0026] 8. Return specified range of data: Copy the data of the specified byte range into the specified buffer.

[0027] Furthermore, this method employs a "block-first, then encryption" preprocessing design: the system generates multiple different keys to encrypt each of the split "plaintext logical blocks" separately, thereby achieving an organic combination of efficiency, security, and privacy protection. This design has dual advantages: firstly, in terms of security, even if a single key is leaked, it will not affect the security of the entire file; secondly, in terms of access efficiency, it supports online random access. When a user only needs to obtain a specific "plaintext logical block," there is no need to extract and decrypt all "encrypted logical blocks," but only to perform extraction and decryption operations on the target "plaintext logical block."

[0028] Furthermore, this method constructs a dual linkage verification mechanism between "metadata" and files / blocks to ensure data integrity and security. First, the encrypted "metadata" is retrieved, decrypted using the account holder's key, and its own hash value is verified to complete the validity verification of the "metadata". Then, based on the block offset mapping relationship, block-level hash value, and file-level hash value stored in the "metadata", the read "redundant physical blocks" are first checked for block-level hash consistency, and then the restored "incomplete encrypted logical blocks" are checked for hash. The "complete factor" backfilling and data decryption operations are performed to merge the data into a complete original file, verify the file integrity, and achieve secure control over the entire data reading process.

[0029] A modified method for storing files with hybrid block encryption that supports dynamic random access is constructed based on the "generation method" described above. It employs an architecture of "double block partitioning + independent key encryption + extraction of 'completion factors' + redundant encoding + dynamic verification," and the specific steps are as follows:

[0030] First, when a user modifies a file, the following parameters are input: fd is the unique identifier of the file; buf is the buffer for writing data; range is the range of bytes to be written.

[0031] 2. Acquisition and verification of metadata: The encrypted metadata is obtained based on the file identifier, and the user's master key is used to decrypt it to obtain the original metadata.

[0032] 3. Block location: Based on the block offset mapping relationship stored in the "metadata", locate the "plaintext logical block" corresponding to the file and the target byte range;

[0033] 4. Modify the "plaintext logical block": Write the data to be written in buf into the specified range of the target "plaintext logical block", generate a new "plaintext logical block", and generate the corresponding "metadata" information; the "metadata" information includes: file identifier and verification information, "plaintext logical block" verification information and block division algorithm;

[0034] 5. Encryption of the new "plaintext logic block": Generate a new independent encryption key for the modified "plaintext logic block", encrypt the new "plaintext logic block" to obtain the new "encrypted logic block", and generate the corresponding "metadata" information; the "metadata" information includes: encryption algorithm and parameters, key information and "encrypted logic block" verification information;

[0035] 6. Extraction of "Completion Factor" after encryption: Extract a portion of data fragments from the new "encrypted logic block" as "completion factor"; then replace the extracted fragments in the new "encrypted logic block" with random padding data to form a new "incomplete encrypted logic block" and generate corresponding "metadata" information; the "metadata" information includes: "completion factor" information and "incomplete encrypted logic block" verification information;

[0036] 7. Redundancy Blocking: The new "incomplete encrypted logic block" is divided into two blocks using a preset redundancy algorithm, generating k original data blocks and m redundant verification blocks, which together form new k+m "redundant physical blocks" and generate corresponding "metadata" information; the "metadata" information includes: redundancy algorithm and parameters;

[0037] 8. Storage of new “redundant physical blocks”: The new “redundant physical blocks” are distributed and stored in different local or remote storage media. The storage path or identification address of the new “redundant physical blocks” is recorded, and the hash value is calculated to generate the corresponding “metadata” information. The “metadata” information includes: “redundant physical block” storage path and “redundant physical block” verification information.

[0038] Step 9, "Metadata" Construction and Storage: All "metadata" fragments generated in steps 4 to 8 above are summarized and constructed into complete "metadata", so that the user's master key can be used to encrypt and store the entire "metadata".

[0039] 9. According to the establishment method provided by the present invention, the following steps are taken: locating the "plaintext logical block" to be modified, decrypting and modifying it, re-encrypting it, extracting the "completeness factor", redundancy segmenting, and replacing the original "redundant physical block"; updating the "metadata" and retaining the old version of the "metadata" and the old version of the "redundant physical block" to ensure multi-version visibility.

[0040] The specific content involved in this invention and the meaning of the terms used are as follows:

[0041] (1) Completion factor: refers to the partial data fragment extracted from the "encrypted logic block" after the original file is preprocessed and divided into blocks, and each "plaintext logic block" is independently encrypted to obtain an "encrypted logic block"; or the partial data fragment extracted before each "plaintext logic block" is encrypted.

[0042] (2) Block offset mapping relationship: It is the corresponding association information between logical blocks and byte ranges recorded in the metadata. It is used in conjunction with the user request byte range when locating the target logical block. Its core function is to achieve accurate location of target data, avoid full file traversal, improve reading efficiency, and support dynamic random reading requirements.

[0043] Advantages and positive effects of the present invention:

[0044] This invention, through its innovative architecture of "dual block division + independent key + extraction of 'completeness factor' + redundant coding + dynamic verification + multi-version management," offers the following advantages over existing technologies:

[0045] Security is significantly improved: It adopts independent key encryption at the "plaintext logical block" level. The leakage of a single key only affects the corresponding "plaintext logical block" and will not lead to the leakage of the entire file; a new "completion factor" extraction mechanism is added to further enhance the security of the "encrypted logical block". Even if the "incomplete encrypted logical block" is stolen, the valid data cannot be restored without obtaining the completion factor; "metadata" is stored after encryption; the hash mechanism realizes full-process data integrity verification, effectively preventing data tampering and damage.

[0046] Dynamic random read / write efficiency optimization: The "logic block first, then encryption" design, combined with the block offset mapping table, supports precise location of the target logical block without processing the entire file, significantly reducing read latency and resource consumption; the encryption algorithm supports multiple types of selection, which can be adapted to the scenario, balancing security and computational efficiency.

[0047] Enhanced fault tolerance and reliability: The second block division supports multiple types of redundancy algorithms, including Reed-Solomon, LDPC, Turbo, Hamming code, and RAID array level, which can be adapted to different storage scenarios.

[0048] It has good scalability: it supports concurrent storage and multi-version management of multiple files under the same user, and achieves the same scheduling through the "metadata" version index table; the block size, encryption algorithm, redundancy calculation and parameters can be adaptively adjusted to meet the needs of different types of files and storage scenarios. Attached Figure Description

[0049] Figure 1 This is a visual diagram of the system process in this invention.

[0050] Figure 2 This is a schematic diagram of the system architecture in this invention. Detailed Implementation

[0051] This invention provides a method for encrypted storage of files in mixed blocks that supports dynamic random reading.

[0052] like Figure 2 The system for file hybrid block encryption storage method supporting dynamic random reading provided by the present invention includes the following components: an application layer 100, a processing layer 200, a verification layer 300, and a storage layer 400. The application layer includes a user terminal 110 and a file operation interface 120. The processing layer includes a file block module 210, an encryption module 220, a completion factor module 230, and a redundancy coding module 240. The verification layer includes a key security module 310 and a hash verification module 320. The storage layer includes a remote storage pool 410 and a local trusted storage 420.

[0053] like Figure 1 As shown, the specific steps of the file hybrid block encryption storage method supporting dynamic random reading provided by the present invention are as follows:

[0054] First, set the target file to be stored;

[0055] Second, preprocess and logically divide the target file to generate "plaintext logical blocks";

[0056] Third, assign an independent key to each "plaintext logic block" and encrypt it to generate an "encrypted logic block";

[0057] 4. Extract the "completion factor" from the "encrypted logic block" to generate the "incomplete encrypted logic block";

[0058] 5. Redundantly divide the "incomplete encrypted logic block" into "redundant physical blocks";

[0059] 7. The program execution has ended.

[0060] To further understand the technical content, features, and effects of this invention, the following application examples are provided, along with detailed descriptions in conjunction with the accompanying drawings:

[0061] Application Example 1

[0062] A method for generating files with hybrid block encryption that supports dynamic random access is proposed. This method employs an architecture of "double block partitioning + independent key encryption + extraction of 'completion factors' + redundant encoding + dynamic verification," and the specific steps are as follows:

[0063] 1. Original File Preprocessing and Logical Blocking: The original file to be stored is format-validated, its hash value is calculated, and it is divided into n "plaintext logical blocks" { L1, L2, ..., L...} according to a given block-splitting strategy. n The corresponding "metadata" fragment is generated; this "metadata" fragment includes: file identifier and verification information, "plaintext logical block" verification information, block partitioning algorithm and block offset mapping relationship;

[0064] 2. Independent key generation and encryption of "plaintext logic blocks": Generate an independent encryption key for each "plaintext logic block", encrypt the "plaintext logic block" to obtain an "encrypted logic block", and generate a corresponding "metadata" fragment; the "metadata" fragment includes: encryption algorithm and parameters, key information and "encrypted logic block" verification information;

[0065] 3. Extracting "Completion Factors" to Generate "Incomplete Encrypted Logic Blocks": Extracting a portion of data fragments from the "Encrypted Logic Blocks" as "Completion Factors"; then replacing the extracted fragments in the "Encrypted Logic Blocks" with randomly filled data to form "Incomplete Encrypted Logic Blocks" and generating corresponding "Metadata" fragments; these "Metadata" fragments include: "Completion Factor" information and "Incomplete Encrypted Logic Block" verification information;

[0066] 4. For each "incomplete encrypted logic block", generate several "redundant physical blocks" through redundant subdivision: Use a preset redundancy algorithm to perform secondary subdivision on each "incomplete encrypted logic block", generating k original data blocks and m redundant verification blocks, where k and m are the parameters set for the redundancy algorithm, representing the number of original data blocks and the number of redundant verification blocks generated, respectively; the original data blocks and redundant verification blocks together form k+m "redundant physical blocks", and generate corresponding "metadata" fragments; the "metadata" fragments include: redundancy algorithm and parameters;

[0067] Fifth, “Redundant Physical Block” Storage: The “redundant physical blocks” are distributed and stored in different local or remote storage media. The storage path or identification address of each “redundant physical block” is recorded, its hash value is calculated, and a corresponding “metadata” fragment is generated. The “metadata” fragment includes: “redundant physical block” storage path and “redundant physical block” verification information.

[0068] Step 6, "Metadata" Construction and Storage: All "metadata" fragments generated in steps 1 to 5 above are aggregated to construct a complete "metadata", and the user's master key is used to encrypt and store the entire "metadata".

[0069] 7. The program execution has ended.

[0070] For example, a company stores an 8MB file named "2024 Annual Financial Summary.xlsx" (containing sensitive financial data). The specific process of applying this generation method is as follows:

[0071] 1. Original File Preprocessing and Logical Blocking: After receiving the file via file operation interface 120, file block segmentation module 210 first verifies that the file format is a valid xlsx format, calculates the overall SHA-256 hash value of the file H_F=0x87654321fedcba09876543210abcdef1234567890abcdef0123456789abcdef; using a "fixed size + data type" combined block segmentation strategy, the 8MB file is divided into 5 "plaintext logical blocks" L1-L5 (L1=1.5MB, L2=1.8MB, L3=1.6MB, L4=1.7MB, L5=1.4MB), and records the byte range of each block (e.g., L1: 0-1572863 bytes) and the block hash value (e.g., L1...). The hash value H_L1=0x1234567890abcdef generates a "metadata" fragment containing information such as the file identifier "FIN-2024-001", the block partitioning algorithm "fixed size + data type", and the block offset mapping relationship;

[0072] 2. Independent Key Generation and Encryption of "Plaintext Logic Blocks": Encryption module 220 uses the AES-256-CBC encryption algorithm to generate independent 256-bit keys for L1-L5 respectively (Key1=0xabcdef0123456789..., Key2=0xfedcba9876543210... etc.). Each "plaintext logic block" is encrypted with the corresponding key to obtain "encrypted logic blocks" E1-E5. The hash values ​​of E1-E5 are calculated (e.g., E1 hash value H_E1=0x9876543210fedcba...). A "metadata" fragment containing encryption algorithm parameters (key length 256 bits, mode CBC), key indices, and verification information of E1-E5 is generated.

[0073] 3. Extracting "Completion Factors" and Generating "Incomplete Encrypted Logic Blocks": The completion factor module 230 extracts "completion factors" from E1-E5. Taking E1 as an example, a 128-byte segment is extracted as CF1 (completion factor). The extracted segment in E1 is replaced with randomly generated 0x00-0xFF bytes of data to form the "incomplete encrypted logic block" CE1. Similarly, CE2-CE5 are generated, and the hash values ​​of CE1-CE5 are calculated (e.g., CE1 hash value H_CE1=0x567890abcdef1234...). A "metadata" segment containing the storage index, extraction position, and verification information of CF1-CF5 is generated.

[0074] 4. Redundant Block Generation of "Redundant Physical Blocks": The redundancy coding module 240 uses Reed-Solomon (3,2) erasure coding (k=3, m=2) to generate 3 original data blocks P1-1, P1-2, P1-3 and 2 redundant check blocks Q1-1, Q1-2 from CE1 through secondary block generation, which together form 5 "redundant physical blocks"; similarly, corresponding "redundant physical blocks" (P2-1~P2-3, Q2-1~Q2-2, etc.) are generated for CE2-CE5, generating "metadata" fragments containing the redundancy algorithm "Reed-Solomon" and parameters k=3, m=2;

[0075] 5. Storage of "Redundant Physical Blocks": Store P1-1 on a local SSD (path: / local / ssd / fin / 2024 / P1-1), P1-2 on local HDD 1 (path: / local / hdd1 / fin / 2024 / P1-2), P1-3 on local HDD 2 (path: / local / hdd2 / fin / 2024 / P1-3), Q1-1 on cloud storage node A (address: cloud: / / nodeA / fin / 2024 / Q1-1), and Q1-2 on cloud storage node B (address: cloud: / / nodeB / fin / 2024 / Q1-2). Calculate the hash value of each "redundant physical block" (e.g., P1-1 hash value H_P1=0x23456789abcdef12...), and generate a "metadata" fragment containing the storage path, identifier address, and hash value of each block.

[0076] 6. Metadata Construction and Storage: Summarize all the above metadata fragments to construct complete metadata, encrypt it with the user's master key (Key_owner=0x1122334455667788...) using AES-128, and store it in local trusted storage 420 (path: / local / trusted / meta / fin2024.meta.enc).

[0077] 7. The program finishes execution and returns a message indicating that the file was saved successfully.

[0078] Application Example 2

[0079] According to the method described above, the segmentation strategy in step 1 includes, but is not limited to, segmenting by fixed size, segmenting by dynamic size, segmenting by user-defined size, segmenting by data type, or pre-segmenting by access frequency. The segmentation strategy supports multi-dimensional selection and can be combined as needed.

[0080] For example, different combinations of partitioning strategies can be used to address different file types and business requirements:

[0081] Scenario 1: For ordinary documents used in daily office work (such as meeting minutes in TXT format, 200KB in size), a "fixed-size block" strategy is adopted, setting the fixed block size to 50KB, dividing the 200KB file into 4 equal-sized "plaintext logical blocks" L1-L4 (50KB each). This strategy is suitable for files with simple data structures and balanced access.

[0082] Scenario 2: The experimental data file of researchers (containing raw sensor data, analysis reports, and charts, with a size of 5GB, of which raw sensor data accounts for 80% of the access frequency) adopts a combined block partitioning strategy of "pre-blocking by access frequency + user-defined size": the user-defined raw sensor data block size is 200MB, the analysis report and chart block size is 50MB, and the "plaintext logical block" corresponding to the frequently accessed raw sensor data is marked as a priority access block. During subsequent storage, the storage path is optimized to improve access speed. Finally, it is divided into 30 "plaintext logical blocks" (20 sensor data blocks, 8 analysis report blocks, and 2 chart blocks).

[0083] Application Example 3

[0084] According to the method described above, the "completion factor" mentioned in step 3 is a data segment randomly selected from the "encrypted logic block". Its core function is to further enhance security and prevent data leakage caused by leakage of the "encrypted logic block" and key. Specifically, it is achieved through the following two precise and controllable extraction methods: Method (1) Fixed-length segment extraction, that is, a unified extraction byte length threshold is set in advance, and the "encrypted logic block" is continuously and without intervals is extracted from the starting position of the "encrypted logic block" according to the threshold. Method (2) Fixed offset extraction, that is, fixed offset interval extraction, that is, a fixed starting offset and extraction interval are set in advance, the first data segment of fixed length is extracted from the starting offset position of the "encrypted logic block", and then some data is skipped according to the preset extraction interval, and the next data segment of the same length is extracted, and so on to complete the extraction.

[0085] For example, taking encrypted logic block E (size 1MB = 1048576 bytes) as an example, the specific applications of the two extraction methods are as follows:

[0086] Fixed-length segmented extraction scenario: Set the extraction byte length threshold to 256 bytes, starting from the beginning position of E (byte 0), and extract bytes sequentially from 0 to 255. After extraction, replace the corresponding segments with random data to form a fragmented encrypted logic block CE. Even if the encryption key of E is leaked, attackers will not be able to recover the client's sensitive information due to the lack of continuous "completion factor" segments.

[0087] Fixed offset extraction scenario: "Completion factors" need to be extracted to protect customer transaction record data. The extraction byte length threshold is set to 256 bytes, the extraction interval to 1024 bytes, and the extraction fragment length to 128 bytes. Extraction process: First, the first data fragment CF1 is extracted from bytes 0-127 of E. Then, skipping 1024 bytes, starting from byte 128+1024=1152, fragment CF2 (bytes 1152-1280) is extracted. This method results in a dispersed distribution of extracted fragments, making it difficult for attackers to piece together complete and valid data even if some "incomplete encrypted logic blocks" are leaked, further enhancing the security of transaction records.

[0088] Application Example 4

[0089] According to the method described, the "completion factor" extraction operation in step 3 can also be performed before the encryption in step 2. That is, after the original file is divided into blocks to generate "plaintext logical blocks", the "completion factor" extraction operation is performed first to form "incomplete logical blocks", and then the "incomplete logical blocks" are encrypted. Specifically, the "completion factor" is extracted from the "plaintext logical blocks" by means of continuous extraction of fixed length or extraction with fixed offset and interval, and the position index is recorded. Random data is filled in to form "incomplete logical blocks", and then an independent key is assigned to the "incomplete logical blocks" to complete the encryption, and "metadata" containing the extraction rules is generated simultaneously.

[0090] For example, the specific process:

[0091] 1. Original file chunking: The file chunking module 210 adopts the "chunking by data type" strategy to divide the file into 3 "plaintext logical blocks" L1 (1MB), L2 (2MB), and L3 (3MB);

[0092] 2. "Completion Factor" Extraction (Fixed-Length Continuous Extraction): Set the extraction byte length threshold to 512 bytes, extract "completion factor" segments from the starting position (byte 0) of L1, and record the position index of each segment (e.g., CF: bytes 0-511); replace the extracted segments in L1 with randomly generated irrelevant random bytes (0x01-0xFE) to form the "incomplete logic block" CL1; similarly, set the extraction threshold of 512 bytes for L2 to generate CL2; set the extraction threshold of 512 bytes for L3 to generate CL3;

[0093] 3. Encrypting the "Incomplete Logic Blocks": Encryption module 220 uses the SM4 encryption algorithm to generate independent 128-bit keys for CL1, CL2, and CL3 respectively (Key_CL1=0x33445566778899aa..., Key_CL2=0x66778899aabbccdd..., Key_CL3=0x99aabbccddeeff11...). CL1-CL3 are then encrypted to obtain the "encrypted logic blocks" EC1-EC3.

[0094] 4. Generate "Metadata" Fragment: Synchronously generate "metadata" fragments containing "completion factor" extraction rules (fixed-length continuous extraction, thresholds of 512 / 512 / 512 bytes respectively), location index, key information, and EC1-EC3 verification information.

[0095] Application Example 5

[0096] A file reading method supporting dynamic random access and hybrid block encryption storage is constructed based on the above-described "generation method" and adopts an architecture of "double block partitioning + independent key encryption + extraction of 'completion factor' + redundant encoding + dynamic verification". The specific steps are as follows:

[0097] First, the user inputs the following parameters when reading a file: fd is the unique identifier of the file; buf is the buffer for reading data; range is the range of bytes to read.

[0098] Second, the acquisition and verification of "metadata": the encrypted "metadata" is obtained based on the file's unique identifier fd, and the user's master key is used to decrypt it to obtain the original complete "metadata";

[0099] 3. Block location: Based on the block offset mapping relationship stored in the "metadata", locate the "plaintext logical block" corresponding to the file and the target byte range, and further locate the "redundant physical block";

[0100] 4. Redundant physical block reading and verification: Based on the storage path of the "redundant physical block" recorded in the "metadata", concurrently read any k "redundant physical blocks" corresponding to each target "plaintext logical block", calculate the hash value, and compare it with the "metadata" to verify the integrity, where k is the minimum number of "redundant physical blocks" corresponding to the "plaintext logical block".

[0101] 5. Recovery and verification of “incomplete encrypted logic block”: Using the verified redundant physical block, call the given redundancy encoding algorithm to decode and recover the “incomplete encrypted logic block”, calculate the hash value of the recovered “incomplete encrypted logic block”, and compare it with the hash value in the metadata to verify its integrity;

[0102] 6. "Completion Factor" backfilling and decryption: Obtain the "completion factor" and its corresponding position index from the "metadata", backfill the "completion factor" into the corresponding position of the "incomplete encrypted logic block" to obtain the "encrypted logic block", and use the key recorded in the "metadata" to align and decrypt to obtain the "plaintext logic block".

[0103] 7. Complete file integration and verification: Integrate and restore the "plaintext logical blocks" according to the block order defined by the "metadata" and perform the final integrity verification;

[0104] 8. Return specified range of data: Copy the data of the specified byte range into the specified buffer.

[0105] For example, if a user needs to read the revenue data for the second quarter of 2024 from the file "2024 Annual Financial Summary.xlsx" (corresponding file byte range: 3,145,728-4,718,591 bytes), the specific process of applying this reading method is as follows:

[0106] Input parameters: The user inputs the file's unique identifier fd="FIN-2024-001", buffer buf=0x7f8000000000 (memory address) and the range of bytes to be read range=3145728-4718591 through the office system;

[0107] Metadata Acquisition and Verification: The metadata management module 250 retrieves the encrypted metadata file ( / local / trusted / meta / fin2024.meta.enc) from the local trusted storage 420 based on the file descriptor (fd). The user's master key is used to decrypt the original metadata, calculate the hash value of the original metadata, and compare it with the preset verification value to verify that the metadata has not been tampered with.

[0108] Block location: Parsing the block offset mapping relationship in the "metadata" reveals that the byte range 3145728-4718591 corresponds to the "plaintext logical blocks" L3 (byte range 2621440-4194303 bytes) and L4 (byte range 4194304-5898239 bytes). Further, the "redundant physical blocks" (P3-1, P3-2, P3-3, Q3-1, Q3-2) corresponding to L3 and the "redundant physical blocks" (P4-1, P4-2, P4-3, Q4-1, Q4-2) corresponding to L4 are located.

[0109] Redundant physical block reading and verification: Based on the storage paths recorded in the "metadata", concurrently read three "redundant physical blocks" from L3 (P3-1: / local / ssd / fin / 2024 / P3-1, P3-3: / local / hdd2 / fin / 2024 / P3-3, Q3-1: cloud: / / nodeA / fin / 2024 / Q3-1) and three "redundant physical blocks" from L4 (P4-2: / local / hdd1 / fin / 2024 / P4-2, P4-3: / local / hdd2 / fin / 2024 / P4-3, Q4-2: cloud: / / nodeB / fin / 2024 / Q4-2); calculate the hash value of each read block and compare it with the corresponding block hash value recorded in the "metadata". If they are consistent, the integrity is verified (k=3).

[0110] Recovery and verification of the "incomplete encrypted logic block": The Reed-Solomon (3,2) decoding algorithm is called to recover the "incomplete encrypted logic block" CE3 using the three redundant physical blocks of L3. The hash value of CE3 is calculated and compared with H_CE3 recorded in the "metadata". They match. Similarly, CE4 is recovered and verified.

[0111] "Completion Factor" Backfilling and Decryption: Obtain the "Completion Factor" CF3 and its position index corresponding to L3 from the "Metadata", backfill CF3 to the corresponding position of CE3 to obtain the "Encrypted Logic Block" E3; decrypt E3 using Key3 recorded in the "Metadata" to obtain the "Plaintext Logic Block" L3; similarly, backfill CF4 to CE4 to obtain E4, and decrypt it using Key4 to obtain L4;

[0112] Complete file integration and verification: According to the block order defined in the "metadata" (L1→L2→L3→L4→L5), extract the data fragments in L3 and L4 corresponding byte ranges of 3145728-4718591, integrate them, calculate the hash value of the fragment, and compare it with the corresponding range check value recorded in the "metadata" to verify completeness;

[0113] Returns data within a specified range: The integrated and verified second-quarter revenue data is copied into the buffer buf=0x7f8000000000, and the user's office system reads and displays the data from the buffer.

[0114] Application Example 6

[0115] Based on the method described above, this method adopts a preprocessing design of "blocking first, then encryption": the system generates multiple different keys to encrypt each "plaintext logical block" after splitting, thereby achieving an organic combination of efficiency, security and privacy protection. This design has dual advantages: firstly, in terms of security, even if a single key is leaked, it will not affect the security of the entire file; secondly, in terms of access efficiency, it supports online random access. When a user only needs to obtain a certain "plaintext logical block", there is no need to extract and decrypt all "encrypted logical blocks", only the extraction and decryption operation needs to be performed on the target "plaintext logical block".

[0116] For example, a security advantage case: An internet company stores user behavior log files (10GB in size, divided into 20 "plaintext logical blocks" L1-L20, corresponding to independent keys Key1-Key20). Due to a system vulnerability, Key5 was obtained by an attacker. The attacker could only decrypt the corresponding "encrypted logical block" E5, obtaining a portion of the user behavior data corresponding to L5 (approximately 500MB), but could not decrypt the other 19 "encrypted logical blocks" (E1-E4, E6-E20), thus preventing the leakage of the entire 10GB user log data and significantly reducing the risk of data leakage.

[0117] Case Study: A university teacher needs to read the content of Chapter 3 (corresponding to L4 bytes ranging from 1.5GB to 2.0GB) from a teaching courseware file (5GB in size, divided into 10 "plaintext logical blocks" L1-L10, corresponding to E1-E10). Using a "block-first, then encrypt" design, the system does not need to download and decrypt all E1-E10 bytes of the entire 5GB file. It only needs to locate the "redundant physical block" corresponding to L4 through "metadata," read and verify it, restore CE4, fill in the "completion factor," and decrypt to obtain L4. The entire process takes approximately 2 seconds. If the traditional overall encryption method is used, it is necessary to download the 5GB encrypted file, decrypt it entirely, and then extract the target chapter, taking approximately 15 seconds. This improves access efficiency by more than 7 times and significantly reduces network bandwidth and computing resource consumption.

[0118] Application Example 7

[0119] Based on the methods described above, this method constructs a dual linkage verification mechanism between "metadata" and files / blocks to ensure data integrity and security. First, the encrypted "metadata" is retrieved, decrypted using the account holder's key, and its own hash value is verified to complete the validity verification of the "metadata". Then, based on the block offset mapping relationship, block-level hash value, and file-level hash value stored in the "metadata", the read "redundant physical blocks" are first checked for block-level hash consistency, and then the restored "incomplete encrypted logical blocks" are checked for hash. The "complete factor" backfilling and data decryption operations are performed to merge the data into a complete original file, verify the file integrity, and achieve secure control over the entire data reading process.

[0120] For example, consider the process of a government system reading an encrypted storage file containing policy documents (divided into blocks L1-L8, corresponding to a total of 40 redundant physical blocks). The specific steps involved in applying this dual-linkage verification mechanism are as follows:

[0121] Metadata validity verification: Retrieve the encrypted metadata file, decrypt it using the user's master key to obtain the original metadata, calculate the SHA-512 hash value of the original metadata H_meta=0x1a2b3c4d5e6f7g8h..., and compare it with the system's preset metadata check hash value H_meta_check=0x1a2b3c4d5e6f7g8h... If they match, the metadata has not been tampered with and is valid and usable.

[0122] Redundant physical block-level hash verification: Read the three "redundant physical blocks" (P6-2, P6-3, Q6-1) corresponding to L6, calculate the hash values ​​of each block as H_P62=0x2c3d4e5f6g7h8i9j..., H_P63=0x3d4e5f6g7h8i9j0k..., H_Q61=0x4e5f6g7h8i9j0k1l..., and compare them with the corresponding block hash values ​​recorded in the "metadata". If they are consistent, the verification passes. If a block (such as P6-2) is tampered with and the calculated hash value is inconsistent with the recorded value, the system will automatically read other redundant blocks (such as P6-1, Q6-2) to replace it, ensuring data integrity.

[0123] Incomplete encrypted logic block hash verification: The "incomplete encrypted logic block" CE6 is recovered by decoding the 3 redundant physical blocks that have passed the verification. The hash value of CE6, H_CE6=0x5f6g7h8i9j0k1l2m..., is calculated. It is consistent with the hash value of CE6 recorded in the "metadata". The verification is successful.

[0124] File integrity verification: The "completeness factor" is backfilled to CE6 to obtain E6, and then decrypted using Key6 to obtain L6. Similarly, after processing other target blocks, they are merged to obtain the complete original policy document. The overall SHA-256 hash value of the file, H_file=0x6g7h8i9j0k1l2m3n..., is calculated and matches the file-level hash value recorded in the "metadata," verifying the file's integrity and lack of tampering. The entire reading process employs four verification methods (metadata, redundant physical blocks, incomplete encrypted logical blocks, and complete file) to achieve full-process security control and effectively prevent security risks such as data tampering and replacement.

[0125] Application Example 8

[0126] A modified method for storing files with hybrid block encryption that supports dynamic random access is constructed based on the "generation method" described above. It employs an architecture of "double block partitioning + independent key encryption + extraction of 'completion factors' + redundant encoding + dynamic verification," and the specific steps are as follows:

[0127] First, when a user modifies a file, the following parameters are input: fd is the unique identifier of the file; buf is the buffer for writing data; range is the range of bytes to be written.

[0128] 2. Acquisition and verification of metadata: The encrypted metadata is obtained based on the file identifier, and the user's master key is used to decrypt it to obtain the original metadata.

[0129] 3. Block location: Based on the block offset mapping relationship stored in the "metadata", locate the "plaintext logical block" corresponding to the file and the target byte range;

[0130] 4. Modify the "plaintext logical block": Write the data to be written in buf into the specified range of the target "plaintext logical block", generate a new "plaintext logical block", and generate the corresponding "metadata" information; the "metadata" information includes: file identifier and verification information, "plaintext logical block" verification information and block division algorithm;

[0131] 5. Encryption of the new "plaintext logic block": Generate a new independent encryption key for the modified "plaintext logic block", encrypt the new "plaintext logic block" to obtain the new "encrypted logic block", and generate the corresponding "metadata" information; the "metadata" information includes: encryption algorithm and parameters, key information and "encrypted logic block" verification information;

[0132] 6. Extraction of "Completion Factor" after encryption: Extract a portion of data fragments from the new "encrypted logic block" as "completion factor"; then replace the extracted fragments in the new "encrypted logic block" with random padding data to form a new "incomplete encrypted logic block" and generate corresponding "metadata" information; the "metadata" information includes: "completion factor" information and "incomplete encrypted logic block" verification information;

[0133] 7. Redundancy Blocking: The new "incomplete encrypted logic block" is divided into two blocks using a preset redundancy algorithm, generating k original data blocks and m redundant verification blocks, which together form new k+m "redundant physical blocks" and generate corresponding "metadata" information; the "metadata" information includes: redundancy algorithm and parameters;

[0134] 8. Storage of new “redundant physical blocks”: The new “redundant physical blocks” are distributed and stored in different local or remote storage media. The storage path or identification address of the new “redundant physical blocks” is recorded, and the hash value is calculated to generate the corresponding “metadata” information. The “metadata” information includes: “redundant physical block” storage path and “redundant physical block” verification information.

[0135] Step 9, "Metadata" Construction and Storage: All "metadata" fragments generated in steps 4 to 8 above are summarized and constructed into complete "metadata", so that the user's master key can be used to encrypt and store the entire "metadata".

[0136] For example, if a user needs to modify the cost data for the second quarter in the "2024 Annual Financial Summary.xlsx" file (corresponding to bytes 3565158-3832831, belonging to L3), the specific process for applying this modification method is as follows:

[0137] Input parameters: User input file unique identifier fd="FIN-2024-001", write data buffer buf=0x7f9000000000 (stores modified cost data), write byte range range=3565158-3832831;

[0138] "Metadata" Acquisition and Verification: The encrypted "metadata" is obtained based on the file descriptor (fd), the user's master key is used to decrypt it to obtain the original "metadata", and the hash value is calculated to verify that it has not been tampered with and is valid and usable;

[0139] Block location: Based on the block offset mapping relationship of "metadata", locate the "plaintext logical block" L3 corresponding to the write range;

[0140] Modification of "plaintext logical block": Obtain the key Key3 corresponding to L3 from the "metadata", decrypt the "encrypted logical block" E3 corresponding to L3, and fill in the "completion factor" to obtain the complete L3; write the modified cost data in buf into the 3565158-3832831 byte range of L3 to generate a new "plaintext logical block" L3_new, calculate the SHA-256 hash value H_L3new=0x7890abcdef123456... of L3_new, and generate a "metadata" fragment containing the file identifier "FIN-2024-001", L3_new verification information, and block division algorithm;

[0141] Encryption of the new "plaintext logic block": Encryption module 220 generates a new independent 256-bit key Key3_new=0x23456789abcdef12... for L3_new, encrypts L3_new using the AES-256-CBC algorithm, obtains a new "encrypted logic block" E3_new, calculates the hash value H_E3new=0x89abcdef12345678... for E3_new, and generates a "metadata" fragment containing encryption algorithm parameters, Key3_new information, and E3_new verification information;

[0142] New "Completion Factor" Extraction: Following the original extraction rules (fixed offset extraction: fragment length 128 bytes), extract the "completion factor" CF3_new from E3_new, replace the extracted fragment with random data to form a new "incomplete encrypted logic block" CE3_new, calculate the hash value H_CE3new=0xabcdef1234567890... of CE3_new, and generate a "metadata" fragment containing CF3_new information and CE3_new verification information;

[0143] Redundant Blocking: Using Reed-Solomon (3,2) erasure coding, CE3_new is divided into two blocks to generate three original data blocks P3_new1-P3_new3 and two redundant check blocks Q3_new1-Q3_new2, forming five new "redundant physical blocks" and generating "metadata" fragments containing redundant algorithms and parameters.

[0144] New "Redundant Physical Block" Storage: Distribute the new "redundant physical blocks" across the original L3 storage media (overwriting the original P3-1-P3-3, Q3-1-Q3-2), record the new storage path (e.g., P3_new1: / local / ssd / fin / 2024 / P3_new1) and hash value, and generate a "metadata" fragment containing the storage path and verification information;

[0145] Metadata Construction and Storage: Summarize the metadata fragments generated in steps 4-8, update the original metadata, and store it in local trusted storage 420 after encryption with the user's master key.

[0146] Application Example 9

[0147] According to the method described, locate the "plaintext logical block" to be modified, decrypt and modify it, re-encrypt it, extract the "completeness factor", redundancy block, replace the original "redundant physical block", update the "metadata" and retain the old version of "metadata" and the old version of "redundant physical block" to ensure multi-version visibility.

[0148] For example, consider a product requirements document file (“Product-Requirement-2024.docx”, divided into L1-L6 blocks, corresponding to 30 redundant physical blocks). A user modified the functional description section of the document twice (both corresponding to L4). The specific process of applying this multi-version management mechanism is as follows:

[0149] First revision (V1→V2):

[0150] Locate the block to be modified: Based on the "metadata" V1, locate the "plaintext logical block" L4 (V1 version) to be modified;

[0151] Modification and Reconstruction: Decrypt L4 (V1), write new functional description data to generate L4 (V2), re-encrypt to generate E4 (V2), extract CF4 (V2) to generate CE4 (V2), and redundancy block to obtain new "redundant physical blocks" P4_V2_1-P4_V2_3 and Q4_V2_1-Q4_V2_2;

[0152] Replacement and Retention: Replace the original L4 (V1) physical block with the new "redundant physical block", and store the original L4 (V1) "redundant physical block" to the history directory (path: / local / history / fin / 2024 / V1 / P4_1, etc.).

[0153] Metadata Management: Update "Metadata" to generate version V2, retain "Metadata" version V1 (path: / local / trusted / meta / PR2024_V1.meta.enc), and record the version index in "Metadata" V2 (V1 creation time 2024-05-10, V2 creation time 2024-05-15);

[0154] Second revision (V2→V3):

[0155] Repeat the above modification and reconstruction steps to generate L4 (V3), E4 (V3), CE4 (V3) and the corresponding new "redundant physical blocks", replace the physical blocks of version V2, and keep the "redundant physical blocks" of version V2 in the history directory;

[0156] Update the "metadata" to generate version V3, retain the "metadata" version V2, and record the complete version index (V1-V3) in V3.

[0157] Multi-version visibility verification:

[0158] Users need to go back and view the document content before the first modification (Version V1). By selecting Version V1 through the office system, the system locates the "redundant physical block" corresponding to the L4 (V1) in the historical storage based on the "metadata" V1. After reading and restoring, the functional description content of Version V1 is successfully displayed.

[0159] If an error is found after modification, the user can choose to roll back to version V1 or V2. The system does not need to modify the data again; it only needs to call the corresponding version's "metadata" and "redundant physical blocks" to restore the file, ensuring the traceability and flexibility of data modification and avoiding data loss caused by accidental modification.

[0160] In view of the description and exemplary embodiments of the invention disclosed herein, other embodiments of the invention will be apparent to those skilled in the art. These descriptions and embodiments are considered as examples only, and any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the scope of protection of the invention.

Claims

1. A method for generating a file hybrid chunked encryption storage supporting dynamic random reading, characterized in that, The architecture employs "double block division + independent key encryption + extraction of 'completion factor' + redundant coding + dynamic verification," and the specific steps are as follows:

1. The first, original file preprocessing and logical chunking: The original file to be stored is format-verified, a hash value is calculated, and the original file is split into n "plaintext logical chunks" {L1, L2,..., Ln} according to a given chunking strategy, and the corresponding "metadata" segment is generated; n } and the corresponding "metadata" segment is generated; This "metadata" fragment includes: file identifier and verification information, "plaintext logical block" verification information, block partitioning algorithm, and block offset mapping relationship; 2. Independent Key Generation and Encryption of "Plaintext Logic Blocks": Generate an independent encryption key for each "plaintext logic block", encrypt the "plaintext logic block" to obtain an "encrypted logic block", and generate a corresponding "metadata" fragment; the "metadata" fragment includes: encryption algorithm and parameters, key information and "encrypted logic block" verification information; 3. Extract "Completion Factor" to generate "Incomplete Encrypted Logic Block": Extract a portion of data fragments from the "Encrypted Logic Block" as "Completion Factor"; then replace the extracted fragments in the "Encrypted Logic Block" with randomly filled data to form an "Incomplete Encrypted Logic Block" and generate the corresponding "Metadata" fragment; the "Metadata" fragment includes: "Completion Factor" information and "Incomplete Encrypted Logic Block" verification information; 4. For each "incomplete encrypted logic block", generate several "redundant physical blocks" through redundant subdivision: Use a preset redundancy algorithm to perform secondary subdivision on each "incomplete encrypted logic block", generating k original data blocks and m redundant verification blocks, where k and m are the parameters set for the redundancy algorithm, representing the number of original data blocks and the number of redundant verification blocks generated, respectively; the original data blocks and redundant verification blocks together form k+m "redundant physical blocks", and generate corresponding "metadata" fragments; the "metadata" fragments include: redundancy algorithm and parameters; 5. Storage of "Redundant Physical Blocks": The "redundant physical blocks" are distributed and stored in different local or remote storage media. The storage path or identification address of each "redundant physical block" is recorded, its hash value is calculated, and a corresponding "metadata" fragment is generated. The "metadata" fragment includes: the storage path of the "redundant physical block" and the verification information of the "redundant physical block".

6. "Metadata" construction and storage: All "metadata" fragments generated in steps 1 to 5 above are summarized and constructed into complete "metadata", and the user's master key is used to encrypt and store the entire "metadata"; 7. The program execution has ended.

2. The method of claim 1, wherein, The segmentation strategies described in step 1 include, but are not limited to, segmenting by fixed size, segmenting by dynamic size, segmenting by user-defined size, segmenting by data type, or pre-segmenting by access frequency. The segmentation strategies support multi-dimensional selection and can be combined as needed.

3. The method of claim 1, wherein, The "completion factor" mentioned in step 3 is a data segment randomly selected from the "encrypted logic block". Its core function is to further enhance security and prevent data leakage caused by leakage of the "encrypted logic block" and key. Specifically, it is achieved through the following two precise and controllable extraction methods: Method (1) Fixed-length segment extraction, that is, a uniform extraction byte length threshold is set in advance. Starting from the starting position of the "encrypted logic block", the "encrypted logic block" is continuously and without intervals according to the threshold. Method (2) Fixed offset extraction, which is to extract at fixed offset intervals. That is, a fixed starting offset and extraction interval are set in advance. First, the first data segment of fixed length is extracted from the starting offset position of the "encrypted logic block". Then, after skipping some data according to the preset extraction interval, the next data segment of the same length is extracted. The extraction is completed in this way.

4. The method according to claim 1, characterized in that, The "completion factor" extraction operation described in step 3 can also be performed before the encryption in step 2. That is, after dividing the original file into blocks to generate "plaintext logical blocks", the "completion factor" extraction operation is performed first to form "incomplete logical blocks", and then the "incomplete logical blocks" are encrypted. Specifically, the "completion factor" is extracted from the "plaintext logical blocks" by means of continuous extraction of fixed length or extraction with fixed offset and interval, and the position index is recorded. Random data is filled in to form "incomplete logical blocks", and then an independent key is assigned to the "incomplete logical blocks" to complete the encryption, and "metadata" containing the extraction rules is generated simultaneously.

5. A method for reading files with hybrid block encryption storage that supports dynamic random reading, constructed based on the "generation method" described in claim 1, characterized in that, The architecture employs "double block division + independent key encryption + extraction of 'completion factor' + redundant coding + dynamic verification," and the specific steps are as follows: First, the user inputs the following parameters when reading a file: fd is the unique identifier of the file; buf is the buffer for reading data; range is the range of bytes to read. Second, the acquisition and verification of "metadata": the encrypted "metadata" is obtained based on the file's unique identifier fd, and the user's master key is used to decrypt it to obtain the original complete "metadata"; 3. Block location: Based on the block offset mapping relationship stored in the "metadata", locate the "plaintext logical block" corresponding to the file and the target byte range, and further locate the "redundant physical block"; 4. Redundant physical block reading and verification: Based on the storage path of the "redundant physical block" recorded in the "metadata", concurrently read any k "redundant physical blocks" corresponding to each target "plaintext logical block", calculate the hash value, and compare it with the "metadata" to verify the integrity, where k is the minimum number of "redundant physical blocks" corresponding to the "plaintext logical block".

5. Recovery and verification of "incomplete encrypted logic block": Using the verified redundant physical block, call the given redundancy encoding algorithm to decode and recover the "incomplete encrypted logic block", calculate the hash value of the recovered "incomplete encrypted logic block", and compare it with the hash value in the metadata to verify its integrity; 6. "Completion Factor" backfilling and decryption: Obtain the "completion factor" and its corresponding position index from the "metadata", backfill the "completion factor" into the corresponding position of the "incomplete encrypted logic block" to obtain the "encrypted logic block", and use the key recorded in the "metadata" to align and decrypt to obtain the "plaintext logic block".

7. Complete file integration and verification: Integrate and restore the "plaintext logical blocks" according to the block order defined by the "metadata" and perform the final integrity verification; 8. Return specified range of data: Copy the data of the specified byte range into the specified buffer.

6. The method according to claim 5, characterized in that, This method employs a "block-first, encryption-later" preprocessing design: the system generates multiple different keys to encrypt each of the split "plaintext logical blocks" separately, thereby achieving an organic combination of efficiency, security, and privacy protection. This design has dual advantages: firstly, in terms of security, even if a single key is leaked, it will not affect the security of the entire file; secondly, in terms of access efficiency, it supports online random access. When a user only needs to obtain a specific "plaintext logical block," there is no need to extract and decrypt all "encrypted logical blocks," but only to perform extraction and decryption operations on the target "plaintext logical block." 7. The method according to claim 5, characterized in that, This method constructs a dual linkage verification mechanism between "metadata" and files / blocks to ensure data integrity and security. First, the encrypted "metadata" is retrieved, decrypted using the account holder's key, and its own hash value is verified to complete the validity verification of the "metadata". Then, based on the block offset mapping relationship, block-level hash value, and file-level hash value stored in the "metadata", the read "redundant physical blocks" are first checked for block-level hash consistency, and then the restored "incomplete encrypted logical blocks" are checked for hash. The "complete factor" backfilling and data decryption operations are performed to merge the data into a complete original file, verify the file integrity, and achieve secure control over the entire data reading process.

8. A modified method for storing files in a hybrid block encryption manner that supports dynamic random reading, constructed based on the "generation method" described in claim 1, characterized in that, The architecture employs "double block division + independent key encryption + extraction of 'completion factor' + redundant coding + dynamic verification," and the specific steps are as follows: First, when a user modifies a file, the following parameters are input: fd is the unique identifier of the file; buf is the buffer for writing data; range is the range of bytes to be written.

2. Acquisition and verification of "metadata": Obtain the corresponding encrypted overall "metadata" based on the file identifier, and decrypt it with the user's master key to obtain the original overall "metadata"; 3. Block location: Based on the block offset mapping relationship stored in the "metadata", locate the "plaintext logical block" corresponding to the file and the target byte range; 4. Modify "plaintext logical block": Write the data to be written in buf into the specified range of the target "plaintext logical block", generate a new "plaintext logical block", and generate the corresponding "metadata" information; The metadata information includes: file identifier and verification information, plaintext logical block verification information, and block segmentation algorithm; 5. Encryption of the new "plaintext logic block": Generate a new independent encryption key for the modified "plaintext logic block", encrypt the new "plaintext logic block" to obtain the new "encrypted logic block", and generate the corresponding "metadata" information; the "metadata" information includes: encryption algorithm and parameters, key information and "encrypted logic block" verification information; 6. Extraction of "Completion Factor" after encryption: Extract a portion of data fragments from the new "encrypted logic block" as "completion factor"; then replace the extracted fragments in the new "encrypted logic block" with random padding data to form a new "incomplete encrypted logic block" and generate corresponding "metadata" information; the "metadata" information includes: "completion factor" information and "incomplete encrypted logic block" verification information; 7. Redundancy Blocking: The new "incomplete encrypted logic block" is divided into two blocks using a preset redundancy algorithm, generating k original data blocks and m redundant verification blocks, which together form new k+m "redundant physical blocks" and generate corresponding "metadata" information; the "metadata" information includes: redundancy algorithm and parameters; 8. Storage of new "redundant physical blocks": The new "redundant physical blocks" are distributed and stored in different local or remote storage media. The storage path or identification address of the new "redundant physical blocks" is recorded, and the hash value is calculated to generate the corresponding "metadata" information. The "metadata" information includes: the storage path of the "redundant physical blocks" and the verification information of the "redundant physical blocks". Step 9, "Metadata" Construction and Storage: All "metadata" fragments generated in steps 4 to 8 above are summarized and constructed into complete "metadata", and the user's master key is used to encrypt and store the entire "metadata".

9. The method according to claim 8, characterized in that, Locate the plaintext logical block to be modified, decrypt and modify it, then re-encrypt it, extract the "completeness factor", perform redundant segmentation, and replace the original "redundant physical block"; update the "metadata" and retain the old version of the "metadata" and the old version of the "redundant physical block" to ensure multi-version visibility.

Images