Distributed redundancy storage method, device and equipment and readable storage medium
By performing identity authentication and file encryption in distributed storage, the security risks in data transmission process in distributed storage are solved, the protection of file keys and data integrity verification are realized, and the security of data transmission is improved .
Patent Information
- Application Number
- CN202411293086.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2024-12-06
AI Technical Summary
Existing distributed storage technologies have security risks during data transmission and are vulnerable to the risks of attacks and data breaches theft or tampering.
By obtaining the application identifier issued by the management end and generating application passwords for identity authentication, using the second password device to generate the file key and encrypted and protected with the encryption public key of the storage server, the distributed redundant storage of files is realized and stored in the storage end During the process, file sharding encryption and integrity verification are carried out to prevent data leakage and tampering.
It effectively improves the data security of distributed storage, prevents file key leakage and data leakage, and ensures security and integrity during file transfer.
Smart Images

Figure CN119089505A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data storage, and in particular to a distributed redundant storage method, device, equipment and readable storage medium. Background Art
[0002] Distributed storage uses data sharding and replica replication technology to ensure data reliability and availability. In data sharding, data is divided into multiple parts, each of which is stored on a different node to achieve distributed storage and management of data. In replica replication, each data copy is stored on a different node to ensure that even if some nodes fail, data can still be recovered from other nodes.
[0003] When sensitive data is involved, plain text file upload is initiated directly from the SDK (Software Development Kit). If a node is attacked from the outside, has a security vulnerability, or an internal user logs in to the host, data may be leaked, stolen, or tampered with. Therefore, existing distributed storage has a large security risk. Summary of the invention
[0004] In view of this, an object of the present invention is to provide a distributed redundant storage method, apparatus, device and readable storage medium, which solves the technical problem of low data security in distributed redundant storage in the prior art.
[0005] In order to solve the above technical problems, the present invention provides a distributed redundant storage method, comprising:
[0006] Obtaining the application identifier issued by the management end, and the management end using the application password generated by the first cryptographic device to initiate an identity authentication request to the storage service end;
[0007] When the identity authentication request is passed, the second cryptographic device is called to generate a file key for the target file uploaded by the application that has passed the identity authentication, and the target encrypted file is obtained. After the file key is encrypted and protected by the encryption public key sent by the storage service client, the encrypted file key is sent to the storage service client, so that the storage service client stores it through the key database;
[0008] The storage service interface is called to send the target encrypted file to the storage service client, so that the storage service client performs distributed redundant storage of the file, and the target encrypted file is stored through the file management disk.
[0009] Optionally, before the calling of the second cryptographic device to generate a file key for the target file uploaded by the application through identity authentication to obtain the target encrypted file, the method further includes:
[0010] Using a third-party application to call the SDK file upload method to determine whether the target file is a file or a file stream;
[0011] If it is a file, calculate the file digest value and obtain the file size;
[0012] If it is a file stream, the file summary value and file size are empty;
[0013] The storage service client determines whether the file is transferred immediately based on the file summary value and the file size. When the file summary value and the file size are not empty, the storage service client determines whether the file is a duplicate based on the file summary value and the file size. If the file is a duplicate, the file is transferred immediately, and the file metadata is copied and the signature of the file summary value is recalculated. Otherwise, a file transfer order is created and an empty storage file is created.
[0014] Optionally, the calling of the second cryptographic device to generate a file key for the target file uploaded by the application through identity authentication to obtain the target encrypted file includes:
[0015] Slice the target file to obtain multiple slice files;
[0016] Circularly read each of the slice files, encrypt each of the slice files using the file key to obtain multiple slice ciphertexts, and send the slice ciphertexts to the storage service for additional storage during the encryption process;
[0017] After the encrypted transmission of the fragments is completed, the storage service obtains the target encrypted file.
[0018] Optionally, when the SDK is used to download a file, the following steps are included:
[0019] Call the SDK file download method, use the target file identifier and the encryption public key as parameters, and obtain the target download encrypted file, the target file digest value, the target file digest signature value, and the file key encrypted and protected by the encryption public key;
[0020] The target downloaded encrypted file is decrypted using the encrypted private key and the file key encrypted and protected using the encrypted public key to obtain a decrypted file; the target file summary value is the summary value calculated during the upload process;
[0021] Performing a file integrity check on the decrypted file, calculating a digest value corresponding to the decrypted file, and comparing it with the digest value of the target file;
[0022] If they are not equal, the file has been tampered with and the process ends;
[0023] If they are equal, compare the digest signature value;
[0024] If the signature values are equal, the decrypted file has not been tampered with.
[0025] Optionally, when the digest value corresponding to the decrypted file is not equal to the digest value of the target file or the digest signature value is not equal, it includes:
[0026] It prompts that the third-party application file is suspected to be tampered with. Determine whether to download the file.
[0027] Optionally, the first cryptographic device and the second key device are soft cards or cryptographic machines. When they are soft cards, the C library is introduced into the SDK; when they are cryptographic machines, the IP and port address of the cryptographic machine are configured for the SDK.
[0028] Optionally, in the process of using the SDK to perform distributed redundant storage, the process further includes:
[0029] The operation logs in the SDK storage process are reported in real time so as to audit the operation logs by the management end; wherein the SDK storage process includes the file upload process and the file download process.
[0030] The present application also provides a distributed redundant storage device, which is applied to SDK, including:
[0031] An identity authentication module, used to obtain the application identifier issued by the management end, and the management end uses the application password generated by the first cryptographic device to initiate an identity authentication request to the storage service end;
[0032] The file encryption module is used to, when an identity authentication request is passed, call the second cryptographic device to generate a file key for the target file uploaded by the application that has passed the identity authentication, obtain a target encrypted file, encrypt and protect the file key using the encryption public key sent by the storage service client, and then send the encrypted file key to the storage service client so that the storage service client stores it through the key database;
[0033] The distributed redundant storage module is used to call the storage service interface to send the target encrypted file to the storage service client so that the storage service client performs distributed redundant storage of the file. The target encrypted file is stored through the file management disk.
[0034] The present application also provides a distributed redundant storage device, including:
[0035] Memory for storing computer programs;
[0036] A processor is used to implement the steps of the above-mentioned distributed redundant storage method when executing the computer program.
[0037] The present application also provides a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned distributed redundant storage method are implemented.
[0038] It can be seen that the present invention initiates an identity authentication request to the storage service end by obtaining the application identifier issued by the management end, and the application password generated by the management end using the first password device; when the identity authentication request is passed, the second password device is called to generate a file key for the target file uploaded by the application that passes the identity authentication, and the target encrypted file is obtained. After the file key is encrypted and protected using the encryption public key sent by the storage service end, the encrypted file key is sent to the storage service end, so that the storage service end stores it through the key database; the storage service interface is called to send the target encrypted file to the storage service end, so that the storage service end performs distributed redundant storage of the file, and the target encrypted file is stored through the file management disk. Compared with the current direct initiation of plain text file upload from the SDK end, this application adopts "one text and one password" to encrypt the distributed redundantly stored files, and uses the encryption public key to protect the file key. The target encrypted file is stored through the file management disk, and the encrypted file key is stored through the key database, so as to achieve separate storage from the file, solve the risk of file key leakage, and alleviate the risk of file leakage caused by file key leakage.
[0039] In addition, the present invention also provides a distributed redundant storage device, equipment and readable storage medium, which also have the above-mentioned beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0041] Figure 1 A flowchart of a distributed redundant storage method provided by an embodiment of the present invention;
[0042] Figure 2 An example flow chart of an application management function and a log audit function implemented on a management terminal provided in an embodiment of the present invention;
[0043] Figure 3 An example diagram of a process for implementing file upload based on SDK and storage service segments provided in an embodiment of the present invention;
[0044] Figure 4 An example diagram of a process for implementing file download based on SDK and storage service segments provided in an embodiment of the present invention;
[0045] Figure 5 A schematic diagram of the structure of a distributed redundant storage device provided by an embodiment of the present invention;
[0046] Figure 6 A schematic diagram of the structure of a distributed redundant storage device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0047] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0048] Some of the terms or terms that appear in the description of the embodiments of the present application are subject to the following interpretations:
[0049] SDK: Software Development Kit is a collection of development tools used by software engineers to create application software for specific software packages, software frameworks, hardware platforms, operating systems, etc. Generally speaking, SDK is the SDK used by application programs under the development platform.
[0050] Distributed storage: A data storage technology that uses the disk space on each machine in the enterprise through the network and forms these distributed storage resources into a virtual storage device. Data is stored in various corners of the enterprise.
[0051] Redundant data storage: Redundancy is a real-time fail-safe measure in case of hard drive failure. RAID (Redundant Array of Independent Disks) is a common redundancy feature in servers and storage devices that can prevent data loss by creating multiple copies of files on multiple hard drives.
[0052] Please refer to Figure 1 , Figure 1 An example flow chart of a distributed redundant storage method provided by an embodiment of the present invention. The method may include:
[0053] S101, obtaining an application identifier issued by a management end, and the management end uses an application password generated by a first cryptographic device to initiate an identity authentication request to a storage service end.
[0054] The executor of this embodiment is SDK. The management end in this embodiment mainly implements application management, log collection and auditing. The application identifier of this embodiment refers to the application ID (identifier). This embodiment does not limit the specific first cryptographic device. For example, in this embodiment, the first cryptographic device is a general term for a device that has a certain cryptographic function or can complete a certain cryptographic work task. For example, in this embodiment, the cryptographic device can be a soft card. When it is a soft card, the C library (open source encryption and decryption library) is introduced to the SDK. When it is a cryptographic machine, the IP (domain name address) and port address of the cryptographic machine are configured for the SDK. In this embodiment, the application password generated by the first cryptographic device initiates an identity authentication request to the storage service end to improve the security of the identity authentication request. It can be understood that the SDK and the storage service side perform the same calculation operation to compare whether the calculation results are equal. If they are equal, the authentication is passed. The steps include: Step 1, using the same application password, the SDK and the storage service side call their respective cryptographic devices to derive the same key step; Step 2, the SDK and the storage service side use the same derived key to perform cryptographic operations on the same random number; Step 3, the SDK sends the cryptographic operation result to the storage service side and compares it with the storage service side's calculation result. If the cryptographic operation results of the two are the same, the authentication is passed.
[0055] S102, when the identity authentication request is passed, the second cryptographic device is called to generate a file key for the target file uploaded by the application that has passed the identity authentication, and the target encrypted file is obtained. After the file key is encrypted and protected using the encryption public key sent by the storage service client, the encrypted file key is sent to the storage service client so that the storage service client can store it through the key database.
[0056] This embodiment uses the server to send an encrypted public key to protect the file key, and sends the encrypted and protected file key to the storage service for storage. This embodiment can request the storage service to obtain the master key (encrypted public key) and signature key protected by the public key of the second cryptographic device. In this embodiment, the master key is used to encrypt and protect the file key, and the signature key is used to sign the file summary value to protect the integrity of the file summary value to prevent the file summary value from being tampered with (necessary for integrity verification). The second cryptographic device in this embodiment is a general term for devices that have certain cryptographic functions or can complete certain cryptographic tasks. When this embodiment uses the second cryptographic device to generate the file key of the target file, the file key can be used to encrypt the target file, or the file key can be used to encrypt the key corresponding to the target file, and when necessary, the encrypted target file can be decrypted using the file key. And the master key will be used in the SDK to encrypt and protect the file key. The master key in this embodiment can be an SM2 asymmetric public key or an SM4 symmetric key.
[0057] It should be further explained that in order to improve the security of file storage, before the above-mentioned calling of the second cryptographic device to generate a file key for the target file uploaded by the application through identity authentication and obtaining the target encrypted file, it can also include: using a third-party application to call the SDK file upload method to determine whether the target file is a file or a file stream; if it is a file, then calculate the file summary value and obtain the file size; if it is a file stream, the file summary value and the file size are empty; so that the storage service end determines whether the file is transmitted in seconds according to the file summary value and the file size; when the file summary value and the file size are not empty, the file is judged whether it is repeated according to the file summary value and the file size. If it is repeated, the file is transmitted in seconds and the file metadata is copied and the signature of the file summary value is recalculated; otherwise, a file transfer order is created to create an empty storage file. In this embodiment, the definition of second transmission is that when it is determined that the uploaded file already has the same file on the storage service end, there is no need to upload the file entity, the upload operation ends, and the file upload is completed. Therefore, there is no subsequent operation of uploading files during second transmission. This embodiment will determine whether the file is repeated to prevent repeated storage of files to save disk resources.
[0058] It should be further explained that in order to improve computing efficiency, the above-mentioned call to the second cryptographic device to generate a file key for the target file uploaded by the application through identity authentication to obtain the target encrypted file can include: sharding the target file to obtain multiple shard files; cyclically reading each shard file, encrypting each shard file using the file key to obtain multiple shard ciphertexts, and sending the shard ciphertexts to the storage service for additional storage during the encryption process; after the shard encryption transmission is completed, the storage service obtains the target encrypted file.
[0059] It is understandable that in the field of distributed storage, the way to read files includes files and file streams. In order to increase the reading speed, this embodiment can read the shard file stream in a loop. When it is determined that the file summary value is empty, it means that the file reading is complete. At this time, the summary value can be calculated, and the currently obtained summary value can be used as the final summary value of the plaintext. This embodiment performs summary and encryption calculations on the file shards at the same time. This embodiment can read the shard file stream in a loop, encrypt the file stream in three stages, obtain the shard ciphertext, if the file summary value is empty, calculate the summary value in three stages at the same time, complete the file reading, and obtain the final summary value of the plaintext. Since this embodiment performs shard calculations, it can improve computing efficiency and save memory resources.
[0060] S103, calling the storage service interface to send the target encrypted file to the storage service end, so that the storage service end performs distributed redundant storage of the file, and the target encrypted file is stored through the file management disk.
[0061] This embodiment will send the encrypted target encrypted file to the storage service end for distributed redundant storage.
[0062] It should be further explained that in order to improve the security of file download, when using the above SDK to download files, it can include: calling the SDK download file method, using the target file identifier and the encrypted public key as parameters, obtaining the target download encrypted file, the target file summary value, the target file summary signature value and the file key encrypted and protected by the encrypted public key; using the encrypted private key and the file key encrypted and protected by the encrypted public key to decrypt the target download encrypted file to obtain the decrypted file; the target file summary value is the summary value calculated during the upload process; the decrypted file is subjected to file integrity verification, the summary value corresponding to the decrypted file is calculated, and compared with the target file summary value; if they are not equal, the file is tampered with and the process ends; if they are equal, the summary signature value is compared; if the signature values are equal, the decrypted file has not been tampered with. This embodiment compares the file summary values when the file is stored and written and downloaded and read to solve the problem of file tampering. It can be understood that there are two situations when the file summary values are equal: 1. The summary values of the file and the database file have not changed 2. The file has changed, and the database summary value becomes the summary of the changed file. In the second case, the signature value of the digest needs to be further compared. If the signature values are equal, the file digests are the same and have not changed, and the files are the same. The signature value of the file digest is used to ensure the integrity of the database digest and prevent it from being changed.
[0063] It should be further explained that, in order to further improve the security of file downloading, when the digest value corresponding to the decrypted file is not equal to the digest value of the target file or the digest signature value is not equal, it may include: prompting the third-party application that the file is suspected to be tampered with, and determining whether to download the file. This embodiment will give prompts in time to prevent the download of tampered files, thereby affecting the use of third-party applications.
[0064] It should be further explained that, in order to improve the traceability of the storage process, the process of using the above SDK for distributed redundant storage can also include: reporting the operation logs in the SDK storage process in real time, so as to audit the operation logs using the management end; wherein the SDK storage process includes the file upload process and the file download process. This embodiment will report the operation logs in the SDK storage process in real time, audit the behavior logs, and can be used for post-event tracing.
[0065] A distributed redundant storage method provided by an embodiment of the present invention may include: S101, obtaining an application identifier issued by a management end, and the management end using an application password generated by a first cryptographic device to initiate an identity authentication request to a storage service end; S102, when the identity authentication request is passed, calling a second cryptographic device to generate a file key for a target file uploaded by an application that passes the identity authentication, obtaining a target encrypted file, encrypting and protecting the file key using an encrypted public key sent by the storage service end, and sending the encrypted file key to the storage service end, so that the storage service end stores it through a key database; S103, calling a storage service interface to send the target encrypted file to the storage service end, so that the storage service end performs distributed redundant storage of the file, and the target encrypted file is stored through a file management disk. Compared with the current method of directly initiating plaintext file upload from an SDK end, this application adopts "one file, one password" to encrypt files in distributed redundant storage, and uses a public key to encrypt the file key, and separates the file from the file for storage, thereby solving the risk of file key leakage and alleviating the risk of file leakage caused by file key leakage. In addition, this embodiment will determine whether the file is duplicated to prevent duplicate storage of the file, so as to save disk resources; and, since this embodiment will perform sharding calculations, it can improve computing efficiency and save memory resources; and, this embodiment will compare the file summary values when the file is stored and written and downloaded and read, so as to solve the problem of file tampering; and, this embodiment will give timely reminders to prevent the download of tampered files, thereby affecting the use of third-party applications; this embodiment will report the operation logs of the SDK storage process in real time, and audit the behavior logs, which can be used for retrospective tracing.
[0066] In order to make the present invention easier to understand, please refer to Figure 2 , Figure 2 An example flow chart of an application management function and a log audit function implemented on a management terminal provided in an embodiment of the present invention may specifically include:
[0067] 1. Register a new application.
[0068] The execution subject of this embodiment is the management platform. In this embodiment, the management platform can be logged in as a system administrator.
[0069] 2. Start the newly registered application and call the password device to generate an application password for the newly registered application; configure the SDK with the current application related information based on the information of the newly registered application.
[0070] 3. After the SDK plug-in reports the log, view and audit the operation log of the encrypted file system reported by the SDK plug-in.
[0071] The embodiment of the present invention uses the management terminal to audit the behavior log, which can be used for post-event tracing.
[0072] In order to make the present invention easier to understand, please refer to Figure 3 , Figure 3 An example flow chart of implementing file upload based on SDK and storage service segment provided in an embodiment of the present invention may specifically include:
[0073] 1. The SDK uses the application ID and application password issued by the security administrator to initiate an identity authentication request to the storage service. If the request is successful, the subsequent process continues.
[0074] The application ID and application password in this embodiment function similarly to the user and password. Only authorized applications can access the storage system, and the application password is the access credential granted by the storage system to the application. The SDK is configured with the storage system access address, application ID, and application password. This embodiment can use the application ID to determine whether the storage system has this application, and the corresponding application password is the authorization credential for the application to access the storage system. The SDK requests authentication, and the storage service end authenticates the authentication parameters (application ID, application password).
[0075] 2. The third-party application calls the SDK file upload method to determine whether the current uploaded file parameter is a file or a file stream; if it is a file, the file digest value is calculated and the file size is obtained; if it is a file stream, the file digest value and size are empty.
[0076] It can be understood that the summary value is calculated as a hash operation, which is to convert variable-length data into a fixed-length operation. The fixed length of the SM3 summary value is 32 bytes. The summary value of a file is closely related to the file entity. In simple terms, the summary value can be used to represent the file, that is, if the summary value is the same, the file entity is judged to be the same. The summary can be recalculated and compared with the original summary value to determine whether the file has changed. Here, the national secret SM3 hash algorithm is used to calculate the file summary value. Both files and file streams need to be calculated for summary when uploading. However, in the second transfer method, the file stream does not need to be calculated for summary. The reason is that the file stream cannot get the complete file and file size in the first time, and the file stream needs to be read until the end to get the complete file. Without a complete file, the summary calculation of the complete file cannot be performed. Without a complete file, the summary storage service end cannot determine whether the file exists, so the file stream does not perform summary calculation or second transfer operation when the file stream is transferred in seconds.
[0077] 3. The storage service determines whether the file should be transferred immediately based on the file summary value and file size. When the file summary value and file size are not empty, the storage service determines whether the file is a duplicate based on the file summary value and file size. If it is a duplicate, the file is transferred immediately, the file metadata is copied, and the signature of the file summary value is recalculated. Otherwise, a file transfer order is created and an empty storage file is created.
[0078] This embodiment determines whether to transfer the file in seconds based on the file summary value, if the file summary value is not empty. And it will determine whether the file is repeated based on the file summary value and file size. If it is repeated, it will be transferred in seconds, and the file metadata will be copied and the signature of the file summary value will be recalculated; otherwise, a file transfer order will be created, and an empty storage file will be created. It can be understood that the summary value signature is related to the application. For example, application A and application B upload the same file respectively, their file digests are the same, but the signature values of the digests are different; if the same application uploads the same file, their file digests and digest signature values are the same. In the case of duplication, it means that the storage service already has the same file. In order to save disk resources and transmission bandwidth, the SDK does not need to upload the same file again; in the case of non-duplicate, the storage service does not have this file, and it is necessary to create a file transfer order to go through the file upload process.
[0079] 4. The SDK calls the cryptographic device to generate a file key, slices the target file, and obtains multiple slice files; it reads each slice file in a loop, encrypts and calculates the summary of each slice file. When all slices are processed, multiple slice ciphertexts and multiple slice summaries are obtained. Multiple slice summaries are combined to obtain the final plaintext summary value, and the slice ciphertext is sent to the storage service for storage during the encryption process; the file key corresponding to the slice ciphertext is encrypted using the encrypted public key to obtain the target encrypted file.
[0080] It is understandable that the previous summary value is only calculated for the file. If the previous calculation is done, the file summary value is not empty and the calculation will not be repeated. If the two summary values are consistent, there is no need to calculate if there is a value in the front. The SDK processes files as follows: split the plaintext file into several small plaintext fragments; encrypt and calculate the summary of each plaintext fragment in turn, and send the encrypted fragment ciphertext to the storage service for storage; after all fragments are encrypted, the storage service also saves the complete ciphertext file assembled with fragments; after all fragments are calculated, the summary values of all fragments are assembled into a new summary. This new summary is the summary of the plaintext file, which has nothing to do with the ciphertext file and the fragment ciphertext.
[0081] 5. The SDK calls the storage service interface to apply for uploading the target encrypted file to the storage service for distributed redundant storage of the file.
[0082] 6. The storage service determines whether the file is a duplicate based on the final plaintext digest value and file size. If it is a duplicate, the relevant information uploaded this time is deleted and the old file information is assigned. If it is not a duplicate, the file-related information and the file key encrypted with the storage service encryption public key are uploaded to the storage service for key management.
[0083] This embodiment is the same as the above method for determining duplication. The purpose here is that when uploading files, there may be several applications that upload files that do not exist in the storage service at the same time. Therefore, the storage server may receive several identical files, and the purpose here is to remove duplicates.
[0084] The stored files in the embodiments of the present invention are encrypted to solve the risk of plaintext file leakage and theft, and the file keys are managed and protected using a key database and stored separately from the files to solve the risk of file key leakage and alleviate the risk of file leakage caused by file key leakage.
[0085] In order to make the present invention easier to understand, please refer to Figure 4 , Figure 4 An example flow chart of implementing file download based on SDK and storage service segment provided in an embodiment of the present invention may specifically include:
[0086] 1. The SDK uses the application ID and application password issued by the security administrator to initiate an identity authentication request to the storage service. If the request is successful, the subsequent process continues.
[0087] 2. The SDK calls the file download method, using the target file identifier and the encrypted public key as parameters to obtain the target downloaded encrypted file, the target file digest value, the target file digest signature value, and the file key encrypted and protected by the encrypted public key; the target downloaded encrypted file is decrypted using the encrypted private key and the file key encrypted and protected by the encrypted public key to obtain the decrypted file; the target file digest value is the digest value calculated during the upload process; the decrypted file is checked for file integrity, the digest value corresponding to the decrypted file is calculated, and compared with the target file digest value; if they are not equal, the file has been tampered with and the process ends; if they are equal, the digest signature value is compared; if the signature values are equal, the decrypted file has not been tampered with.
[0088] The embodiment of the present invention performs integrity calculation and verification on the stored data to solve the risk of node tampering.
[0089] A distributed redundant storage device provided by an embodiment of the present invention is introduced below. The distributed redundant storage device described below and the distributed redundant storage method described above can be referred to each other.
[0090] Please refer to Figure 5 , Figure 5 A schematic diagram of the structure of a distributed redundant storage device provided by an embodiment of the present invention may include:
[0091] The identity authentication module 100 is used to obtain the application identifier issued by the management end, and the management end uses the application password generated by the first cryptographic device to initiate an identity authentication request to the storage service end;
[0092] The file encryption module 200 is used to, when an identity authentication request is passed, call the second cryptographic device to generate a file key for the target file uploaded by the application that has passed the identity authentication, obtain a target encrypted file, encrypt and protect the file key using the encryption public key sent by the storage service client, and then send the encrypted file key to the storage service client so that the storage service client stores it through the key database;
[0093] The distributed redundant storage module 300 is used to call the storage service interface to send the target encrypted file to the storage service end so that the storage service end performs distributed redundant storage of the file. The target encrypted file is stored through the file management disk.
[0094] Further, based on the above embodiment, the above distributed redundant storage device may further include:
[0095] A file and file stream judgment module, used to use a third-party application to call the SDK file upload method to determine whether the target file is a file or a file stream;
[0096] A file summary value calculation module is used to calculate the file summary value and obtain the file size if it is a file;
[0097] The file digest value is determined to be an empty module. If it is a file stream, the file digest value and the file size are empty; so that the storage service end determines whether the file is transmitted in seconds based on the file digest value and the file size; when the file digest value and the file size are not empty, it is determined whether the file is a duplicate based on the file digest value and the file size. If it is a duplicate, the file is transmitted in seconds, and the file metadata is copied and the signature of the file digest value is recalculated; otherwise, a file transfer order is created and an empty storage file is created.
[0098] Further, based on any of the above embodiments, the above file encryption module 200 may include:
[0099] A slicing unit, used for slicing the target file to obtain a plurality of slicing files;
[0100] A slice encryption unit is used to cyclically read each of the slice files, encrypt each of the slice files using the file key, obtain multiple slice ciphertexts, and send the slice ciphertexts to the storage service for additional storage during the encryption process;
[0101] The encryption unit uses the file key to obtain the target encrypted file after the fragmented encryption transmission is completed.
[0102] Further, based on the above embodiment, the above distributed redundant storage device may further include:
[0103] If they are equal, the decrypted file has not been tampered with.
[0104] The parameter acquisition module is used to call the SDK file download method, use the target file identifier and the encryption public key as parameters, and obtain the target download encrypted file, the target file digest value, the target file digest signature value, and the file key encrypted and protected by the encryption public key;
[0105] A decryption module is used to decrypt the target downloaded encrypted file using the encryption private key and the file key encrypted and protected by the encryption public key to obtain a decrypted file; the target file summary value is the summary value calculated during the upload process;
[0106] A digest value comparison module, used to perform a file integrity check on the decrypted file, calculate a digest value corresponding to the decrypted file, and compare it with the digest value of the target file;
[0107] A tampering determination module is used to determine that if the two are not equal, the file has been tampered with and the process ends;
[0108] A digest signature value comparison module is used to compare the digest signature values if they are equal;
[0109] A tamper-proof determination module is used to determine that if the signature values are equal, the decrypted file has not been tampered with.
[0110] Further, based on the above embodiment, the above distributed redundant storage device may further include:
[0111] The tampering prompt module is used to prompt that a third-party application file is suspected to be tampered with and determine whether to download the file.
[0112] Further, based on any of the above embodiments, the first cryptographic device and the second key device are soft cards or cryptographic machines. When they are soft cards, the C library is introduced into the SDK; when they are cryptographic machines, the IP and port addresses of the cryptographic machines are configured for the SDK.
[0113] Further, based on the above embodiment, the above distributed redundant storage device may further include:
[0114] The log reporting module is used to report the operation logs in the SDK storage process in real time so that the management end can audit the operation logs; wherein, the SDK storage process includes the file upload process and the file download process.
[0115] It should be noted that the order of the modules and units in the above-mentioned distributed redundant storage device can be changed without affecting the logic.
[0116] A distributed redundant storage device provided by an embodiment of the present invention may include: an identity authentication module 100, which is used to obtain an application identifier issued by a management end, and the management end uses the application password generated by a first password device to initiate an identity authentication request to a storage service end; a file encryption module 200, when the identity authentication request is passed, calls a second password device to generate a file key for a target file uploaded by an application that passes the identity authentication, obtains a target encrypted file, encrypts and protects the file key using an encryption public key sent by the storage service end, and sends the encrypted file key to the storage service end, so that the storage service end stores it through a key database; a distributed redundant storage module 300, which is used to call a storage service interface to send the target encrypted file to the storage service end, so that the storage service end performs distributed redundant storage of files, and the target encrypted file is stored through a file management disk. Compared with the current direct initiation of plain text file upload from the SDK end, this application adopts "one file, one password" to encrypt files in distributed redundant storage, and uses the file key encryption public key protection, and separates the file storage, so as to solve the risk of file key leakage and alleviate the risk of file leakage caused by file key leakage. In addition, this embodiment will determine whether the file is duplicated, prevent duplicate storage of files, and save disk resources; and, since this embodiment will perform sharding calculations, it can improve computing efficiency and save memory resources; and, this embodiment will compare the file summary values when the file is stored and written and downloaded and read, to solve the problem of file tampering; and, this embodiment will give timely reminders to prevent the download of tampered files, thereby affecting the use of third-party applications; this embodiment will report the operation logs of the SDK storage process in real time, audit the behavior logs, and can be used for retrospective tracing.
[0117] A distributed redundant storage device provided by an embodiment of the present invention is introduced below. The distributed redundant storage device described below and the distributed redundant storage method described above can be referred to each other.
[0118] Please refer to Figure 6 , Figure 6 A schematic diagram of the structure of a distributed redundant storage device provided in an embodiment of the present invention may include:
[0119] A memory 10, used for storing computer programs;
[0120] The processor 20 is used to execute a computer program to implement the above-mentioned distributed redundant storage method.
[0121] The memory 10 , the processor 20 , and the communication interface 30 all communicate with each other via a communication bus 40 .
[0122] In the embodiment of the present invention, the memory 10 is used to store one or more programs, and the program may include program code, and the program code includes computer operation instructions. In the embodiment of the present invention, the memory 10 may store programs for implementing the following functions:
[0123] Obtaining the application identifier issued by the management end, and the management end using the application password generated by the first cryptographic device to initiate an identity authentication request to the storage service end;
[0124] When the identity authentication request is passed, the second cryptographic device is called to generate a file key for the target file uploaded by the application that has passed the identity authentication, and the target encrypted file is obtained. After the file key is encrypted and protected using the encryption public key sent by the storage service client, the encrypted file key is sent to the storage service client, so that the storage service client stores it through the key database;
[0125] The storage service interface is called to send the target encrypted file to the storage service end, so that the storage service end performs distributed redundant storage of the file, and the target encrypted file is stored through the file management disk.
[0126] In a possible implementation, the memory 10 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function, etc.; the data storage area may store data created during use.
[0127] In addition, the memory 10 may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include an NVRAM. The memory stores an operating system and operating instructions, executable modules or data structures, or a subset thereof, or an extended set thereof, wherein the operating instructions may include various operating instructions for implementing various operations. The operating system may include various system programs for implementing various basic tasks and processing hardware-based tasks.
[0128] The processor 20 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic device, a microprocessor or any conventional processor, etc. The processor 20 may call a program stored in the memory 10 .
[0129] The communication interface 30 may be an interface of a communication module, and is used to connect to other devices or systems.
[0130] Of course, it should be noted that Figure 6 The structure shown does not constitute a limitation on the distributed redundant storage device in the embodiment of the present invention. In actual applications, the distributed redundant storage device may include Figure 6 More or fewer components than shown, or combinations of certain components.
[0131] The following is an introduction to a readable storage medium provided in an embodiment of the present invention. The readable storage medium described below and the distributed redundant storage method described above can be referenced to each other.
[0132] The present invention also provides a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned distributed redundant storage method are implemented.
[0133] The readable storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.
[0134] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0135] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0136] Finally, it should be noted that, in this article, relationships such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0137] The above is a detailed introduction to a distributed redundant storage method, device, equipment and readable storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.
Claims
1. A distributed redundant storage method, characterized in that: Applied to SDK, including: Obtaining the application identifier issued by the management end, and the management end using the application password generated by the first cryptographic device to initiate an identity authentication request to the storage service end; When the identity authentication request is passed, the second cryptographic device is called to generate a file key for the target file uploaded by the application that has passed the identity authentication, and the target encrypted file is obtained. After the file key is encrypted and protected by the encryption public key sent by the storage service client, the encrypted file key is sent to the storage service client, so that the storage service client stores it through the key database; The storage service interface is called to send the target encrypted file to the storage service client, so that the storage service client performs distributed redundant storage of the file, and the target encrypted file is stored through the file management disk.
2. The distributed redundant storage method according to claim 1, characterized in that: The calling of the second cryptographic device to generate a file key for the target file uploaded by the application through identity authentication, before obtaining the target encrypted file, also includes: Using a third-party application to call the SDK file upload method to determine whether the target file is a file or a file stream; If it is a file, calculate the file digest value and obtain the file size; If it is a file stream, the file summary value and file size are empty; The storage service client determines whether the file is transferred immediately based on the file summary value and the file size. When the file summary value and the file size are not empty, the storage service client determines whether the file is a duplicate based on the file summary value and the file size. If the file is a duplicate, the file is transferred immediately, and the file metadata is copied and the signature of the file summary value is recalculated. Otherwise, a file transfer order is created and an empty storage file is created.
3. The distributed redundant storage method according to claim 1, characterized in that: The calling of the second cryptographic device to generate a file key for the target file uploaded by the application through identity authentication to obtain the target encrypted file includes: Slice the target file to obtain multiple slice files; Circularly read each of the slice files, encrypt each of the slice files using the file key to obtain multiple slice ciphertexts, and send the slice ciphertexts to the storage service for additional storage during the encryption process; After the encrypted transmission of the fragments is completed, the storage service obtains the target encrypted file.
4. The distributed redundant storage method according to claim 1, characterized in that: When using the SDK to download a file, it includes: Call the SDK file download method, use the target file identifier and the encryption public key as parameters, and obtain the target download encrypted file, the target file digest value, the target file digest signature value, and the file key encrypted and protected by the encryption public key; The target downloaded encrypted file is decrypted using the encrypted private key and the file key encrypted and protected using the encrypted public key to obtain a decrypted file; the target file summary value is the summary value calculated during the upload process; Performing a file integrity check on the decrypted file, calculating a digest value corresponding to the decrypted file, and comparing it with the digest value of the target file; If they are not equal, the file has been tampered with and the process ends; If they are equal, compare the digest signature value; If the signature values are equal, the decrypted file has not been tampered with.
5. The distributed redundant storage method according to claim 4, characterized in that: When the digest value corresponding to the decrypted file is not equal to the digest value of the target file or the digest signature value is not equal, it includes: It prompts that the third-party application file is suspected to be tampered with. Determine whether to download the file.
6. The distributed redundant storage method according to claim 1, characterized in that: The first cryptographic device and the second key device are soft cards or cryptographic machines. When they are soft cards, the C library is introduced into the SDK. When they are cryptographic machines, the IP and port address of the cryptographic machine are configured for the SDK.
7. The distributed redundant storage method according to claim 1, characterized in that: The process of using the SDK to perform distributed redundant storage also includes: The operation logs in the SDK storage process are reported in real time so as to audit the operation logs by the management end; wherein the SDK storage process includes the file upload process and the file download process.
8. A distributed redundant storage device, characterized in that: Applied to SDK, including: An identity authentication module, used to obtain the application identifier issued by the management end, and the management end uses the application password generated by the first cryptographic device to initiate an identity authentication request to the storage service end; The file encryption module is used to, when an identity authentication request is passed, call the second cryptographic device to generate a file key for the target file uploaded by the application that has passed the identity authentication, obtain a target encrypted file, encrypt and protect the file key using the encryption public key sent by the storage service client, and then send the encrypted file key to the storage service client so that the storage service client stores it through the key database; The distributed redundant storage module is used to call the storage service interface to send the target encrypted file to the storage service client so that the storage service client performs distributed redundant storage of the file. The target encrypted file is stored through the file management disk.
9. A distributed redundant storage device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the distributed redundant storage method according to any one of claims 1 to 7 when executing the computer program.
10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the distributed redundant storage method according to any one of claims 1 to 7 are implemented.
