Decentralized data encryption, deduplication, and storage method based on encoding-matrix secret sharing

Abstract

A decentralized data encryption and deduplication storage method based on encoding-matrix secret sharing, the method comprising: performing block processing on data to be stored, and generating a plurality of data blocks (100); on the basis of content fingerprints of the plurality of data blocks, cooperatively generating an encryption encoding matrix via a plurality of decentralized key servers (110); encrypting the plurality of data blocks on the basis of the encryption encoding matrix, and generating a plurality of ciphertext blocks (120); and storing the plurality of ciphertext blocks into a distributed storage system, and eliminating duplicate data on the basis of content fingerprints respectively corresponding to the plurality of ciphertext blocks (130). The method described in the present application can improve system scalability and performance, ensure data security and storage reliability, and is applicable to large-scale distributed storage environments.

Description

Decentralized Data Encryption and Deduplication Storage Method Based on Secret Sharing of Encoding Matrix [Technical Field]

[0001] This application belongs to the field of computer storage technology, and more specifically, relates to a decentralized data encryption and deduplication storage method based on secret sharing of an encoding matrix. [Background Technology]

[0002] With the surge in data storage demands, more and more users are outsourcing their data to cloud storage systems. However, this also brings with it issues of data redundancy and security. Data encryption and deduplication technology addresses these problems by ensuring that identical plaintext blocks are encrypted into identical ciphertext blocks, thus eliminating duplicates. This technology combines encryption with deduplication, protecting data privacy while reducing redundancy overhead, thereby ensuring security and lowering storage costs.

[0003] Existing data encryption and deduplication technologies mainly include centralized and decentralized methods. Centralized methods use a centralized key management server to generate encryption keys to encrypt data and distribute the encrypted data across multiple cloud servers using code-based secret sharing. However, centralized methods heavily rely on centralized servers, which can significantly compromise system security if attacked. In contrast, decentralized key management methods involve each key server independently generating different keys, resulting in different encryption results for the same data from different users. This necessitates additional key exchange and other technologies to ensure deduplication of identical data, leading to additional computational and deduplication indexing overhead and impacting system performance. [Summary of the Invention]

[0004] To address the shortcomings of existing technologies, the purpose of this application is to provide a decentralized data encryption and deduplication storage method based on secret sharing of the encoding matrix, aiming to solve the problems of insufficient scalability, high computational overhead, and insufficient data security in existing encryption and deduplication methods.

[0005] To achieve the above objectives, in a first aspect, this application provides a decentralized data encryption and deduplication storage method based on secret sharing of an encoding matrix, comprising:

[0006] The data to be stored is divided into blocks to generate multiple data blocks;

[0007] Based on the content fingerprints of the multiple data blocks, an encryption encoding matrix is ​​generated collaboratively by multiple decentralized key servers;

[0008] The multiple data blocks are encrypted based on the encryption encoding matrix to generate multiple ciphertext blocks;

[0009] The multiple ciphertext blocks are stored in a distributed storage system, and duplicate data is deleted based on the content fingerprints corresponding to the multiple ciphertext blocks.

[0010] This application divides the data to be stored into multiple blocks and uses multiple decentralized key servers to collaboratively generate an encryption encoding matrix to encrypt and deduplicate the data blocks. This reduces the computation and communication overhead during the key exchange process, thereby improving deduplication efficiency and reducing the overall system load. It achieves efficient decentralized data encryption and deduplication storage, greatly improves the scalability and performance of the system, ensures data security and storage reliability, and is suitable for large-scale distributed storage environments.

[0011] According to the decentralized data encryption and deduplication storage method based on secret sharing of the encoding matrix provided in this application, the step of dividing the data to be stored into blocks includes:

[0012] The data to be stored is divided into blocks using a content-defined block division algorithm.

[0013] This application uses a content-defined segmentation algorithm to segment the data to be stored, which ensures that the same data block generates the same segmentation and fingerprint.

[0014] According to the decentralized data encryption and deduplication storage method based on secret sharing of the encoding matrix provided in this application, the step of generating an encryption encoding matrix collaboratively by multiple decentralized key servers based on the content fingerprints of the multiple data blocks includes:

[0015] Based on the content fingerprints of the multiple data blocks, initial key fragments are generated by multiple decentralized key servers respectively;

[0016] The cryptographic encoding matrix is ​​generated based on multiple initial key fragments.

[0017] According to the decentralized data encryption and deduplication storage method based on secret sharing of the encoding matrix provided in this application, the step of generating the encryption encoding matrix based on multiple initial key fragments includes:

[0018] The matrix elements are generated step by step based on multiple initial key fragments, and the cryptographic encoding matrix is ​​generated by combining sliding window technology for pipeline acceleration.

[0019] This application uses a sliding window-based matrix element generation method, which can ensure that multiple key servers can perform parallel computation and generate complete encoding matrices in a timely manner, thereby improving matrix generation efficiency and supporting multi-user collaborative operation.

[0020] According to the decentralized data encryption and deduplication storage method based on secret sharing of the encoding matrix provided in this application, the step of deleting duplicate data based on the content fingerprints corresponding to the multiple ciphertext blocks includes:

[0021] Obtain the content fingerprints corresponding to the existing ciphertext blocks in the distributed storage system;

[0022] The content fingerprints corresponding to the multiple ciphertext blocks are compared with the content fingerprints corresponding to the existing ciphertext blocks, and the ciphertext blocks corresponding to duplicate content fingerprints are deleted.

[0023] This application optimizes the deduplication process by mapping data content in a consistent manner, thereby improving the deduplication efficiency of the entire system.

[0024] According to the decentralized data encryption and deduplication storage method based on secret sharing of the encoding matrix provided in this application, the method further includes:

[0025] The content fingerprint information of the ciphertext blocks in the distributed storage system is updated in real time.

[0026] This application updates the stored ciphertext block information in real time to ensure efficient data retrieval and deduplication.

[0027] Secondly, this application provides a decentralized data encryption and deduplication storage system based on secret sharing of an encoding matrix, a decentralized key server cluster, a distributed storage system, and a client module, wherein:

[0028] The decentralized key server cluster is connected to the client module and includes multiple decentralized key servers. Each key server can independently generate keys and collaboratively participate in the generation of encryption encoding matrices and the data block encryption process. Multiple server clusters exchange data and cooperate with each other through a secure communication protocol.

[0029] The distributed storage system is connected to the client module and is used to store ciphertext blocks, perform deduplication based on the fingerprint of the ciphertext block content, and update the stored ciphertext block information in real time.

[0030] The client module is connected to the decentralized key server cluster and the distributed storage system, and is used to interact with the key server cluster and the distributed storage system to complete data upload, encryption, storage and data recovery operations.

[0031] Thirdly, this application provides a decentralized data encryption and deduplication storage device based on secret sharing of an encoding matrix, comprising:

[0032] The data segmentation module is used to segment the data to be stored into multiple data blocks.

[0033] The generation module is used to generate an encryption encoding matrix collaboratively by multiple decentralized key servers based on the content fingerprints of the multiple data blocks.

[0034] An encryption module is used to encrypt the plurality of data blocks based on the encryption encoding matrix to generate a plurality of ciphertext blocks;

[0035] The deduplication storage module is used to store the multiple ciphertext blocks in a distributed storage system and to delete duplicate data based on the content fingerprints corresponding to the multiple ciphertext blocks.

[0036] Fourthly, this application provides an electronic device, comprising: at least one memory for storing a program; and at least one processor for executing the program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to execute the decentralized data encryption and deduplication storage method based on secret sharing of an encoding matrix as described in the first aspect or any possible implementation thereof.

[0037] Fifthly, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the decentralized data encryption and deduplication storage method based on secret sharing of an encoding matrix as described in the first aspect or any possible implementation of the first aspect.

[0038] Sixthly, this application provides a computer program product that, when run on a processor, causes the processor to execute the decentralized data encryption and deduplication storage method based on secret sharing of an encoding matrix as described in the first aspect or any possible implementation of the first aspect.

[0039] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0040] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art:

[0041] (1) By dividing the data to be stored into multiple blocks and using multiple decentralized key servers to collaboratively generate an encryption encoding matrix to encrypt and deduplicatize the data blocks, the computation and communication overhead during the key exchange process can be reduced, thereby improving deduplication efficiency and reducing the overall system load. This achieves efficient decentralized data encryption and deduplication storage, greatly improving the system's scalability and performance, ensuring data security and storage reliability, and is suitable for large-scale distributed storage environments.

[0042] (2) Using a sliding window-based matrix element generation method can ensure that multiple key servers can compute in parallel and generate complete encoding matrices in a timely manner, thereby improving matrix generation efficiency and supporting multi-user collaborative operation. [Attached Image Description]

[0043] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 is a flowchart illustrating the decentralized data encryption and deduplication storage method based on secret sharing of the encoding matrix provided in an embodiment of this application;

[0045] Figure 2 is a schematic diagram of the process for generating an encryption encoding matrix provided in an embodiment of this application;

[0046] Figure 3 is a schematic diagram of the process of generating an optimization matrix step by step using matrix elements according to an embodiment of this application;

[0047] Figure 4 is a schematic diagram of the process for generating a parameter scalability optimization matrix according to an embodiment of this application;

[0048] Figure 5 is a schematic diagram of the structure of a decentralized data encryption and deduplication storage system based on secret sharing of the encoding matrix provided in an embodiment of this application;

[0049] Figure 6 is a schematic diagram of the structure of the decentralized data encryption and deduplication storage device based on secret sharing of the encoding matrix provided in the embodiment of this application;

[0050] Figure 7 is a schematic diagram of the structure of the electronic device provided in an embodiment of this application. 【Detailed Implementation Methods】

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0053] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0054] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0055] Next, the decentralized data encryption and deduplication storage method based on secret sharing of the encoding matrix provided in the embodiments of this application will be introduced with reference to Figures 1-4.

[0056] Figure 1 is a flowchart illustrating the decentralized data encryption and deduplication storage method based on secret sharing of the encoding matrix provided in this application embodiment. As shown in Figure 1, the method includes the following steps:

[0057] Step 100: Divide the data to be stored into blocks to generate multiple data blocks;

[0058] The decentralized data encryption and deduplication storage method based on secret sharing of the encoding matrix provided in this application embodiment can be executed by a client. The client first divides the data to be stored into several data blocks according to the content, ensuring that each data block has clear content boundaries and a fixed size for subsequent deduplication operations.

[0059] Step 110: Based on the content fingerprints of multiple data blocks, an encryption encoding matrix is ​​generated collaboratively by multiple decentralized key servers;

[0060] Content fingerprinting is a technology for identifying and tracking digital content. It creates a "fingerprint" by extracting the unique features of the content. This fingerprint can be used to identify and verify the identity of the content, similar to the role of human fingerprints in identifying personal identities.

[0061] Optionally, the content fingerprint can be a hash value.

[0062] Optionally, multiple decentralized key servers exchange data through a secure communication protocol to ensure information security during the encryption process.

[0063] The client generates a unique identifier for each data block based on the content fingerprint of the data block and sends it to multiple decentralized key servers. The servers then collaboratively generate an encrypted encoding matrix based on secret sharing technology and matrix factorization.

[0064] Step 120: Encrypt multiple data blocks based on the encryption encoding matrix to generate multiple ciphertext blocks;

[0065] The client converts data blocks into ciphertext blocks by performing matrix multiplication based on the encryption encoding matrix. The ciphertext blocks contain redundant information of the original data, so that even if part of the ciphertext blocks are lost, the data can still be recovered.

[0066] Step 130: Store multiple ciphertext blocks in a distributed storage system and perform deduplication based on the content fingerprints corresponding to each of the multiple ciphertext blocks.

[0067] The client uploads the generated ciphertext to different cloud storage servers to ensure distributed data storage, thereby improving storage security and reliability.

[0068] The client stores the encrypted ciphertext block in a distributed storage system. The system then performs deduplication based on the fingerprint of the ciphertext block content.

[0069] Optionally, when data recovery is required, the client can download a sufficient number of encrypted fragments from various cloud storage servers and use the previously generated key fragments to decrypt and recover the original data blocks.

[0070] This application provides a decentralized data encryption and deduplication storage method based on secret sharing of an encoding matrix. By dividing the data to be stored into multiple blocks and using multiple decentralized key servers to collaboratively generate an encryption encoding matrix to encrypt and deduplicate the data blocks, the computation and communication overhead during key exchange can be reduced, thereby improving deduplication efficiency and reducing the overall system load. This achieves efficient decentralized data encryption and deduplication storage, greatly improving the system's scalability and performance, ensuring data security and storage reliability, and is suitable for large-scale distributed storage environments.

[0071] In some embodiments, step 100 specifically includes:

[0072] The data to be stored is divided into blocks using a content-defined block partitioning algorithm.

[0073] Content-Defined Chunking (CDC) is a technique for data deduplication and backup. It determines the boundaries of data chunks by analyzing the content of the data itself, rather than relying on the size or location of the data. The core idea is to divide the data stream into a series of chunks, each consisting of a series of consecutive data. The boundaries of the chunks are determined by the data content. This means that two data chunks with the same content will be classified as the same chunk, even if they are in different positions in the file.

[0074] The client uses a content-defined chunking algorithm to divide the data to be stored into chunks, ensuring that each data block has clear content boundaries and a fixed size for subsequent deduplication operations.

[0075] In some embodiments, step 110 specifically includes:

[0076] Step 1101: Based on the content fingerprints of multiple data blocks, generate initial key fragments through multiple decentralized key servers respectively;

[0077] Step 1102: Generate an encryption encoding matrix based on multiple initial key fragments.

[0078] The client calculates the hash value of each data block, generates a unique identifier for each block, and sends it to multiple decentralized key servers to request key fragments. Each decentralized key server independently generates key fragments based on the client's request. These fragments are then assembled into an encryption matrix, providing the foundation for the encryption of subsequent data blocks.

[0079] In some embodiments, step 1102 specifically includes:

[0080] The matrix elements are generated step by step based on multiple initial key fragments, and the pipeline is accelerated by combining sliding window technology to generate the encrypted encoding matrix.

[0081] Figure 2 is a flowchart illustrating the process of generating an encryption encoding matrix according to an embodiment of this application. As shown in Figure 2, when multiple decentralized key servers generate key fragments to form an encryption encoding matrix, the Cauchy matrix generation method is specifically used. First, the client generates two different element sequences {x} based on the hash value of the data block. i} and {y i These sequences are calculated using a hash function (e.g., SHA-256), and the elements of each sequence are closely related to the content of the data block.

[0082] Vector-based convergent matrix generation methods cannot effectively reduce redundancy in ciphertext fragments. This paper addresses this issue by employing matrix generation optimization methods to accelerate the generation of convergent matrices. The main idea is to avoid performing invertibility checks and adjustments on the matrix and reduce communication overhead. Key optimization methods include element-wise matrix generation and parameter scalability optimization.

[0083] Figure 3 is a flowchart illustrating the step-by-step generation of an optimization matrix using matrix elements, as provided in an embodiment of this application. As shown in Figure 3, in the optimization method based on step-by-step generation of the matrix using matrix elements, the two sequences {x} generated by the client are not directly used. i} and {y jInstead of generating the Cauchy matrix using a sequence from the client and a key server respectively, the initial sequence {y} is generated on the client side using data block cryptographic hashes. j (1≤j≤k)}, and then send the elements to k key servers for encryption. The key servers return another sequence {y} to the client. j '(1≤j≤k)}, and finally the client continues to generate another sequence {x} from the cryptographic hash values ​​of the data blocks in sequence. i (1≤i≤n)}. Because the initialized sequence {y j} is not used as the target sequence, therefore its generation process does not need to guarantee that they are all different, but the final returned sequence {y} is not used as the target sequence. j '} and the regenerated sequence {x i The sequences {y} must be distinct. j '} and {x i Then the n×k encoding matrix that meets the MDS property can be calculated.

[0084] Preferably, through pipeline design, convergence matrix generation operations for three data blocks can be processed simultaneously, thereby reducing the overall time for matrix generation.

[0085] Figure 4 is a schematic diagram of the process of generating a parameter scalability optimization matrix provided in an embodiment of this application. As shown in Figure 4, for archived data with low access frequency, a medium-parameter erasure coding scheme is usually used to achieve redundant storage.

[0086] However, when the encoding parameters (n,k) and word length w meet certain specific conditions, traditional hashing methods cannot generate a sufficient number of convergent matrix elements, affecting the efficiency of encoding and the reliability of data.

[0087] For example, when using RS(14,10) erasure coding and word length w=16, a 256-bit hash value cannot generate a 384-bit matrix element.

[0088] Using a sliding window method, matrix elements are dynamically generated from hash values ​​of fixed length. By moving the window and generating elements step by step, the generated matrix elements are ensured to meet the requirements of erasure coding and the same elements are avoided from being generated repeatedly.

[0089] Specifically, the initial range of the sliding window is set to the first w bits of the hash value, generating a sequence {x}. i The first element of} is x1. The sliding window moves backward. Each time, a candidate element is generated from the current window. It checks if the generated candidate element already exists in the previous sequence; if not, it is saved as the next element in the sequence; otherwise, new candidate elements are generated. This process of moving the sliding window and generating elements continues until the sequence {x} is generated. i} and {y j All elements of}.

[0090] For {y j The sequence {y} does not need to guarantee that the elements are all different. When the key server returns a new sequence {y} j After '}, continue generating the sequence {x} from the position of the sliding window. i}, and ensure {x i} and {y j The elements in '} are all distinct.

[0091] In some embodiments, step 130 specifically includes:

[0092] Step 1301: Obtain the content fingerprints corresponding to the existing ciphertext blocks in the distributed storage system;

[0093] Step 1302: Compare the content fingerprints corresponding to the multiple ciphertext blocks with the content fingerprints corresponding to the existing ciphertext blocks, and delete the ciphertext blocks corresponding to duplicate content fingerprints.

[0094] The cloud server uses content fingerprints, or hash values, to detect whether each uploaded ciphertext block already exists in the distributed storage system. If the same fragment is found, deduplication is performed, and a unique ciphertext block is stored to reduce redundancy.

[0095] In some embodiments, the method further includes:

[0096] The content fingerprint information of the encrypted blocks in the distributed storage system is updated in real time.

[0097] Optionally, the distributed storage system may include a deduplication index module for real-time updating of stored ciphertext block information, ensuring efficient data retrieval and deduplication.

[0098] Figure 5 is a schematic diagram of the decentralized data encryption and deduplication storage system based on secret sharing of the encoding matrix provided in an embodiment of this application. As shown in Figure 5, the system includes a decentralized key server cluster 510, a distributed storage system 520, and a client module 530, wherein:

[0099] The decentralized key server cluster 510 is connected to the client module 530 and includes multiple decentralized key servers. Each key server can independently generate keys and participate collaboratively in the generation of the encryption encoding matrix and the data block encryption process. Multiple server clusters exchange data and cooperate with each other through a secure communication protocol.

[0100] The distributed storage system 520 is connected to the client module 530 to store ciphertext blocks, perform deduplication based on the fingerprint of the ciphertext block content, and update the stored ciphertext block information in real time.

[0101] The client module 530 connects to the decentralized key server cluster 510 and the distributed storage system 520 to interact with the key server cluster and the distributed storage system to complete data upload, encryption, storage and data recovery operations.

[0102] It should be understood that the above-described device is used to execute the methods in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above methods. The working process of the device can be referred to the corresponding process in the above methods, and will not be repeated here.

[0103] Figure 6 is a schematic diagram of the decentralized data encryption and deduplication storage device based on secret sharing of the encoding matrix provided in an embodiment of this application. As shown in Figure 6, the device includes a block segmentation module 610, a generation module 620, an encryption module 630, and a deduplication storage module 640, wherein:

[0104] The block segmentation module 610 is used to segment the data to be stored into blocks, generating multiple data blocks.

[0105] The generation module 620 is used to generate an encrypted encoding matrix collaboratively by multiple decentralized key servers based on the content fingerprints of multiple data blocks.

[0106] The encryption module 630 is used to encrypt multiple data blocks based on the encryption encoding matrix to generate multiple ciphertext blocks;

[0107] The deduplication storage module 640 is used to store multiple ciphertext blocks in a distributed storage system and perform duplicate data deletion based on the content fingerprints corresponding to the multiple ciphertext blocks.

[0108] It should be understood that the above-described device is used to execute the methods in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above methods. The working process of the device can be referred to the corresponding process in the above methods, and will not be repeated here.

[0109] Based on the methods in the above embodiments, Figure 7 illustrates a schematic diagram of the physical structure of an electronic device. As shown in Figure 7, this application embodiment provides an electronic device that may include: a processor 710, a communication interface 720, a memory 730, and a communication bus 740. The processor 710, communication interface 720, and memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute the decentralized data encryption and deduplication storage method based on secret sharing of the encoding matrix in the above embodiments.

[0110] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the decentralized data encryption and deduplication storage method based on secret sharing of the encoding matrix described in the various embodiments of this application.

[0111] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program. When the computer program runs on a processor, it causes the processor to execute the decentralized data encryption and deduplication storage method based on secret sharing of the encoding matrix in the above embodiments.

[0112] Based on the methods in the above embodiments, this application provides a computer program product that, when running on a processor, causes the processor to execute the decentralized data encryption and deduplication storage method based on secret sharing of the encoding matrix in the above embodiments.

[0113] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0114] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0115] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0116] It is understood that the various numerical designations used in the embodiments of this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application.

[0117] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A decentralized data encryption and deduplication storage method based on secret sharing of an encoding matrix, characterized in that, include: The data to be stored is divided into blocks to generate multiple data blocks; Based on the content fingerprints of the multiple data blocks, an encryption encoding matrix is ​​generated collaboratively by multiple decentralized key servers; The multiple data blocks are encrypted based on the encryption encoding matrix to generate multiple ciphertext blocks; The multiple ciphertext blocks are stored in a distributed storage system, and duplicate data is deleted based on the content fingerprints corresponding to the multiple ciphertext blocks.

2. The decentralized data encryption and deduplication storage method based on secret sharing of the encoding matrix according to claim 1, characterized in that, The process of dividing the data to be stored into blocks includes: The data to be stored is divided into blocks using a content-defined block division algorithm.

3. The decentralized data encryption and deduplication storage method based on secret sharing of the encoding matrix according to claim 1, characterized in that, The process of generating an encryption encoding matrix based on the content fingerprints of the multiple data blocks through collaborative efforts of multiple decentralized key servers includes: Based on the content fingerprints of the multiple data blocks, initial key fragments are generated by multiple decentralized key servers respectively; The cryptographic encoding matrix is ​​generated based on multiple initial key fragments.

4. The decentralized data encryption and deduplication storage method based on secret sharing of the encoding matrix according to claim 3, characterized in that, The process of generating the encryption encoding matrix based on multiple initial key fragments includes: The matrix elements are generated step by step based on multiple initial key fragments, and the cryptographic encoding matrix is ​​generated by combining sliding window technology for pipeline acceleration.

5. The decentralized data encryption and deduplication storage method based on secret sharing of the encoding matrix according to claim 1, characterized in that, The process of deleting duplicate data based on the content fingerprints corresponding to the multiple ciphertext blocks includes: Obtain the content fingerprints corresponding to the existing ciphertext blocks in the distributed storage system; The content fingerprints corresponding to the multiple ciphertext blocks are compared with the content fingerprints corresponding to the existing ciphertext blocks, and the ciphertext blocks corresponding to duplicate content fingerprints are deleted.

6. The decentralized data encryption and deduplication storage method based on secret sharing of the encoding matrix according to claim 1, characterized in that, The method further includes: The content fingerprint information of the ciphertext blocks in the distributed storage system is updated in real time.

7. A decentralized data encryption and deduplication storage system based on secret sharing of an encoding matrix, characterized in that, It includes a decentralized key server cluster, a distributed storage system, and a client module, among which: The decentralized key server cluster is connected to the client module and includes multiple decentralized key servers. Each key server can independently generate keys and collaboratively participate in the generation of encryption encoding matrices and the data block encryption process. Multiple server clusters exchange data and cooperate with each other through a secure communication protocol. The distributed storage system is connected to the client module and is used to store ciphertext blocks, perform deduplication based on the fingerprint of the ciphertext block content, and update the stored ciphertext block information in real time. The client module is connected to the decentralized key server cluster and the distributed storage system, and is used to interact with the key server cluster and the distributed storage system to complete data upload, encryption, storage and data recovery operations.

8. A decentralized data encryption and deduplication storage device based on secret sharing of an encoding matrix, characterized in that, include: The data segmentation module is used to segment the data to be stored into multiple data blocks. The generation module is used to generate an encryption encoding matrix collaboratively by multiple decentralized key servers based on the content fingerprints of the multiple data blocks. An encryption module is used to encrypt the plurality of data blocks based on the encryption encoding matrix to generate a plurality of ciphertext blocks; The deduplication storage module is used to store the multiple ciphertext blocks in a distributed storage system and to delete duplicate data based on the content fingerprints corresponding to the multiple ciphertext blocks.

9. An electronic device, characterized in that, include: At least one memory for storing computer programs; At least one processor is configured to execute a program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to execute the decentralized data encryption and deduplication storage method based on secret sharing of an encoding matrix as described in any one of claims 1-6.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is run on the processor, the processor performs the decentralized data encryption and deduplication storage method based on secret sharing of the encoding matrix as described in any one of claims 1-6.

Images