A cloud-edge collaborative deduplication method, system, device and medium

Abstract

The application provides a cloud-edge collaborative deduplication method, system, device and medium, which comprises the following steps: performing trusted authentication between the client and the edge server, and between the edge server and the cloud server; performing communication encryption and uploading on the to-be-uploaded data; performing hash calculation on the data blocks to obtain corresponding fingerprint information; dividing the data blocks according to the file label, the fingerprint information and a two-level index strategy; if the type is a hot block, performing edge deduplication; if the type is a cold block, performing cloud deduplication. The application creates a cloud security zone and an edge security zone, realizes effective isolation of communication, and guarantees the safety of the communication process. The shared key is obtained while performing trusted authentication, which reduces the management and storage overhead of the key. The two-level index strategy is used to realize edge deduplication and cloud deduplication, which reduces the pressure of cloud storage and improves the storage efficiency and data management capability of the system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data storage and computer technology, and particularly relates to a cloud-edge collaborative deduplication method, system, device and medium. BACKGROUND

[0002] Edge computing is a distributed computing architecture in which data processing and application running are closer to the "edge" devices or local networks of data. The cloud-edge collaborative mode aims to reduce the delay of data transmission to the cloud server for processing, improve the real-time performance of data processing, reduce the computational burden of the cloud server, and enhance the security and privacy protection of data.

[0003] On the other hand, secure deduplication is an attractive data redundancy reduction technique that stores only one copy of the duplicated data and provides the owner with a link to the copy. At the same time, it also uses encryption to achieve data security, taking into account the storage efficiency and security requirements of data.

[0004] Secure deduplication in edge computing improves the cost-effectiveness and data security of cloud storage. However, existing solutions are built on expensive encryption operations, which result in high communication and computation overheads. SUMMARY

[0005] The present application provides a cloud-edge collaborative deduplication method, system, device and medium, which solves the problem of high encryption cost and large communication and computation overheads in existing secure deduplication solutions in edge computing when processing data.

[0006] To solve the above technical problems, the technical solution adopted by the present application is to provide a cloud-edge collaborative deduplication method, comprising the steps of: performing trusted authentication between the client and the edge server, and performing trusted authentication between the edge server and the cloud server; performing communication encryption on the data to be uploaded to obtain encrypted data and upload; wherein the encrypted data includes multiple data blocks of a file and a file tag corresponding to the file; performing communication decryption on the uploaded encrypted data, and performing hash calculation on each data block to obtain fingerprint information corresponding to each data block; dividing the multiple data blocks into types according to the file tag, the fingerprint information corresponding to each data block, and a two-level index strategy; if the type of the data block is hot block, performing edge deduplication on the data block; if the type of the data block is cold block, performing cloud deduplication on the data block.

[0007] In some embodiments, the method of trusted authentication between the client and the edge server includes the steps of: the edge server creating an edge security zone, the edge security zone generating a first attestation report; wherein the first attestation report includes identity information and state measurement values of the edge security zone; the edge security zone sending the first attestation report to an authentication unit, the authentication unit verifying the first attestation report to obtain a first signed report; the authentication unit sending the first signed report to the client through the edge server, and the client verifying the first signed report to determine the trustworthiness of the edge security zone.

[0008] In some embodiments, the method of trusted authentication between the edge server and the cloud server includes the steps of: the cloud server creating a cloud security zone and a key security zone, and multiple edge servers creating corresponding multiple edge security zones; the key security zone verifying the trustworthiness of the multiple edge security zones and generating a shared key and multiple private keys; wherein the private keys correspond one-to-one to the edge security zones; the key security zone generating a second attestation report; wherein the second attestation report includes fingerprint information of the shared key, fingerprint information of the private keys, and identity information of the key security zone; the key security zone sending the second attestation report to an authentication unit, the authentication unit verifying the second attestation report to obtain a second signed report; and the authentication unit sending the second signed report to each edge server through the key security zone, the edge server verifying the second signed report to determine the trustworthiness of the key security zone and obtain the shared key and the corresponding private key.

[0009] In some embodiments, the method of dividing multiple data blocks into types according to the file label, the fingerprint information corresponding to each data block, and a two-level index strategy specifically includes: identifying the version type of the file corresponding to the data block according to the file label corresponding to each data block; searching for the corresponding version type in a first-level index entry and locating to a corresponding second-level index table according to the version type; wherein the second-level index table includes a fingerprint information list, an address list, and a type list; querying the fingerprint information list according to the fingerprint information corresponding to the data block, and updating the type of the corresponding data block in the type list; dividing the type of the data block corresponding to the updated part in the type list into hot blocks, and dividing the type of the data block corresponding to the un-updated part in the type list into cold blocks.

[0010] In some embodiments, the edge deduplication method comprises the steps that: the hot block comprises a first type of hot block and a second type of hot block; if the type of the data block is the first type of hot block, the data block is encrypted and saved; and if the type of the data block is the second type of hot block, the data block is deleted.

[0011] In some embodiments, the cloud deduplication method comprises the steps that: the fingerprint information corresponding to the data block of a cold block type is sent; wherein the cold block comprises a first type of cold block and a second type of cold block; the type of the data block is determined by secondary comparison according to the fingerprint information, and a secondary comparison result is fed back; if the type of the data block is the first type of cold block, the data block is encrypted and saved after uploading; and if the type of the data block is the second type of cold block, no processing is performed.

[0012] In some embodiments, the subsequent steps further comprise: sending the file label corresponding to the file to be downloaded; finding the corresponding recipe information according to the file label; obtaining the plurality of data blocks corresponding to the file according to the recipe information, and decrypting the plurality of data blocks to obtain the file and download the file.

[0013] The application further provides a cloud-edge collaborative deduplication system, comprising: an authentication unit configured to perform trusted authentication between a client and an edge server, and trusted authentication between the edge server and a cloud server; a client configured to perform communication encryption on to-be-uploaded data to obtain encrypted data and upload the encrypted data; wherein the encrypted data comprises a plurality of data blocks of a file and a file label corresponding to the file; an edge server configured to perform communication decryption on the uploaded encrypted data, perform hash calculation on each data block to obtain fingerprint information corresponding to each data block, perform type division on the plurality of data blocks according to the file label, the fingerprint information corresponding to each data block, and a two-level index strategy, perform edge deduplication on the data blocks of a hot block type, and perform cloud deduplication on the data blocks of a cold block type.

[0014] The application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described above when executing the computer program.

[0015] The application further provides a computer-readable storage medium storing a computer program, wherein the computer program is executable on a processor to implement the steps of the method described above.

[0016] The beneficial effects of this invention are as follows: This invention discloses a cloud-edge collaborative deduplication method, system, device, and medium. The method includes the following steps: performing trusted authentication between the client and the edge server, and trusted authentication between the edge server and the cloud server; encrypting the data to be uploaded to obtain encrypted data and uploading it; wherein, the encrypted data includes multiple data blocks of the file and the file tag corresponding to the file; after decrypting the uploaded encrypted data, performing hash calculation on each data block to obtain fingerprint information corresponding to each data block; classifying the multiple data blocks according to the file tag, the fingerprint information corresponding to each data block, and a two-level indexing strategy; if the data block is a hot block, edge deduplication is performed on the data block; if the data block is a cold block, cloud deduplication is performed on the data block. This application achieves effective isolation of communication by creating a cloud security zone and an edge security zone, ensuring the security of the communication process. It also obtains a shared key while performing trusted authentication, reducing the management and storage overhead of the key. Furthermore, the two-level indexing strategy for edge deduplication and cloud deduplication not only reduces the pressure on cloud storage but also improves the storage efficiency and data management capabilities of the entire system. This application not only improves data communication and storage efficiency, but also further reduces communication, encryption, and storage costs, achieving a highly efficient balance between system performance and data security. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the cloud-edge collaborative deduplication method of this application;

[0018] Figure 2 This is a schematic diagram of the cloud-edge collaborative deduplication system of this application;

[0019] Figure 3 This is a schematic diagram of the specific structure of the cloud-edge collaborative deduplication system of this application;

[0020] Figure 4 This is a schematic diagram of the two-level indexing strategy in the cloud-edge collaborative deduplication method of this application;

[0021] Figure 5 This is a schematic diagram of the architecture of an embodiment of an electronic device according to this application;

[0022] Figure 6 This is a schematic block diagram of an embodiment of a computer-readable storage medium according to this application. Detailed Implementation

[0023] For the purpose of promoting the understanding of the present application, the present application will be described in further detail below with reference to the drawings and specific embodiments. The preferred embodiments of the present application are shown in the drawings. However, the present application can be realized in many different forms and is not limited to the embodiments described in this specification. On the contrary, these embodiments are provided for the purpose of making the disclosure of the present application more thorough and comprehensive.

[0024] It should be noted that, unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. The term "and / or" used in this specification includes any and all combinations of one or more related listed items.

[0025] Figure 1 The implementation flowchart of the cloud-edge collaborative deduplication method provided by the present application is shown, which includes the following steps:

[0026] S1: respectively performing trusted authentication between the client and the edge server, and trusted authentication between the edge server and the cloud server.

[0027] S2: performing communication encryption on the to-be-uploaded data to obtain encrypted data and uploading; wherein the encrypted data includes a plurality of data blocks of a file and a file tag corresponding to the file.

[0028] S3: performing communication decryption on the uploaded encrypted data, and performing hash calculation on each data block to obtain fingerprint information corresponding to each data block.

[0029] S4: dividing the plurality of data blocks by type according to the file tag, the fingerprint information corresponding to each data block, and a two-level index strategy.

[0030] S5: if the type of the data block is a hot block, performing edge deduplication on the data block; if the type of the data block is a cold block, performing cloud deduplication on the data block.

[0031] The present application realizes effective isolation of communication by creating a cloud security zone and an edge security zone, thereby ensuring the security of the communication process. The shared key is obtained while performing trusted authentication, thereby reducing the management and storage overhead of the key. Further, the edge deduplication and cloud deduplication are realized by the two-level index strategy, thereby not only reducing the pressure on cloud storage, but also improving the storage efficiency and data management capability of the entire system. The present application not only improves the data communication efficiency and storage efficiency, but also further reduces the communication, encryption and storage costs, thereby realizing efficient balance of system performance and data security.

[0032] The present application will be described in further detail below with reference to the drawings and specific embodiments. Figures 1 to 6The application is further described in detail with specific embodiments.

[0033] As shown in the Figure 1 The cloud-edge collaborative deduplication method provided by the embodiments of the application is described in detail as follows.

[0034] S1: respectively performing trusted authentication between the client and the edge server, and trusted authentication between the edge server and the cloud server.

[0035] As shown in the Figure 2 The deduplication system of the application includes one cloud server 1, multiple edge servers 2, and multiple clients 3. One cloud server 1 can respectively communicate with multiple edge servers 2 without interference; each edge server 2 can communicate with multiple clients 3 without interference.

[0036] Among them, the cloud server 1 is a physical device located in the network center, with strong computing and storage capabilities, which is responsible for scheduling global ciphertext data storage and deduplication work. The edge server 2 is distributed around the client 3, which provides outsourcing storage and computing services for users. The client 3 is the legal owner of the data, each client 3 has specific access rights, and the edge server 2 manages the access rights of each user according to these access rights.

[0037] As shown in the Figure 2 In operation, the system of the application first needs to be initialized, that is, trusted authentication needs to be performed between each part (such as between the client 3 and the edge server 2, or between the edge server 2 and the cloud server 1) to ensure the security of the communication process. The trusted authentication process is mainly completed by the authentication unit 4, which specifically refers to the Intel Attestation Service (IAS) in this embodiment.

[0038] Specifically, as shown in the Figure 3 The method of trusted authentication between each client 3 and the edge server 2 includes the following steps:

[0039] The edge server 2 first creates an edge security area 22, and calls the SGX (Software Guard Extensions, processor extension) code to make the edge security area 22 generate a first proof report. The first proof report includes the identity information and state measurement value of the edge security area 22, and the state measurement value is the fingerprint information in the edge security area 22.

[0040] Furthermore, the edge security zone 22 sends the first verification report to the authentication unit 4. The authentication unit 4 verifies the authenticity of the first verification report and signs it to generate a first signature report 29. The authentication unit 4 returns the first signature report 29 to the edge server 2, and the edge server 2 sends the first signature report 29 to the client 3. At the same time, the edge server 2 saves the first signature report 29 to the edge storage zone 23.

[0041] After receiving the first signature report 29, client 3 uses the public key of authentication unit 4 to verify the signature in the first signature report 29 and further determine the trustworthiness of edge security zone 22.

[0042] Furthermore, in combination Figure 2 , Figure 3 As shown, the method for trusted authentication between multiple edge servers 2 and cloud server 1 includes the following steps:

[0043] Cloud server 1 first creates cloud security zone 12 and key security zone 11, and each edge server 2 creates its own corresponding edge security zone 22.

[0044] Furthermore, the trustworthiness of multiple edge security zones 22 is verified through the key security zone 11. The key security zone 11 uses a random function from the KDC (Key Distribution Center) to generate high-security encryption keys (including a shared key and multiple private keys), such as AES (Advanced Encryption Standard) in GCM (Galois counter mode), which provides data confidentiality and integrity. Each private key corresponds one-to-one with an edge security zone 22; that is, each edge security zone 22 corresponds to a unique private key.

[0045] Furthermore, the key security zone 11 generates a second proof report. This second proof report includes the fingerprint information of the shared key, the fingerprint information of each private key, and the identity information of the key security zone 11. The key security zone 11 sends the second proof report to the authentication unit 4. The authentication unit 4 verifies the authenticity of the second proof report and signs it, generating a second signature report 17. The authentication unit 4 returns the second signature report 17 to the key security zone 11 and sends it to each edge server 2 via the key security zone 11. Simultaneously, the cloud server 1 saves the second signature report 17 to the cloud storage area 13.

[0046] After each edge server 2 receives the second signature report 17, the second signature report 17 is verified respectively, the trustworthiness of the key security area 11 is judged, and the shared key and the respective corresponding different private key are obtained. Each edge server 2 saves the obtained shared key and private key into the respective edge security area 22 for subsequent use.

[0047] S2: communication encryption is performed on the to-be-uploaded data, encrypted data is obtained, and uploading is performed; wherein the encrypted data includes a plurality of data blocks of a file and a file tag corresponding to the file.

[0048] Specifically, as shown in Figure 2 the step is mainly completed by the client 3.

[0049] When the client 3 needs to upload a file to the edge server 2 or the cloud server 1, the client 3 first needs to perform communication encryption on the to-be-uploaded data to avoid data leakage during uploading.

[0050] In this embodiment, an ECDH (Elliptic Curve Diffie-Hellman) protocol is used to generate a one-time session key to perform communication encryption on the to-be-uploaded data. The one-time session key calculation step is not repeated here. By using the ECDH protocol, the client 3 and the edge server 2 can negotiate a common session key on an insecure channel, thereby ensuring the security of the communication process.

[0051] It should be noted that one file can be composed of a plurality of different data blocks, and when one of the files needs to be uploaded, a plurality of data blocks of the file can be selected for uploading to facilitate subsequent duplicate checking and deletion. The file tag represents the specific version type of the file.

[0052] Further, the encrypted data after communication encryption is uploaded to the corresponding edge server 2 for further processing.

[0053] S3: After communication decryption is performed on the uploaded encrypted data, hash calculation is performed on each data block to obtain fingerprint information corresponding to each data block.

[0054] Specifically, as shown in Figure 3 the step is mainly completed by the edge server 2.

[0055] The client 3 uploads the encrypted data to the edge server 2 through the file transmission interface 21 and transmits the encrypted data to the edge security area 22 of the edge server 2. The encrypted data is communication-decrypted by the encryption and decryption module 24 of the edge security area 22 using the one-time session key generated in S2.

[0056] The decrypted data is then transmitted to the deduplication module 25. The hash calculation submodule 201 calculates the fingerprint information corresponding to each data block in the data and records the recipe information 27 corresponding to the data. At the same time, the recipe information 27 is stored in the edge storage area 23 for use in the subsequent download stage.

[0057] S4: Based on file tags, fingerprint information corresponding to each data block, and a two-level indexing strategy, classify multiple data blocks by type.

[0058] Specifically, such as Figure 3 As shown, this step is mainly accomplished through edge server 2.

[0059] The fingerprint information corresponding to each data block calculated by the hash calculation submodule 201 in S3 is transmitted to the classification submodule 202 to classify each data block. In this application, the data block types are divided into two categories: cold blocks (i.e., data that is not frequently used) and hot blocks (i.e., data that is needed in the short term or data that is frequently used in the long term).

[0060] In this embodiment, combined with Figure 4 As shown, the specific classification method includes the following steps:

[0061] Based on the file tag corresponding to each data block, the version type of the file corresponding to the data block is identified. The corresponding version type is searched in the primary index entries, and the corresponding secondary index table is located based on the version type. The secondary index table includes a fingerprint information list, an address list, and a type list. Based on the fingerprint information corresponding to the data block, a query is performed in the fingerprint information list, and the type of the corresponding data block is updated in the type list. Data blocks whose types are updated in the type list are classified as hot blocks, and data blocks whose types are not updated in the type list are classified as cold blocks.

[0062] by Figure 4 Taking the data as an example (where blue represents newly added content and red represents modified content), the above process will be further explained with an example. First, the entire type list in the secondary index table is filled with "cold blocks". Assuming that based on the file tag of the current data block, the version type of the file corresponding to the current data block is identified as Linux, then in... Figure 4 The system searches for the corresponding version type (i.e., Linux version type) in the first-level index entry and then locates the corresponding second-level index table (i.e., Linux second-level index table) based on the version type.

[0063] If the fingerprint information corresponding to the current data block is A, the same fingerprint information (A) is found in the fingerprint information list in the Linux secondary index table, and the type in the type list corresponding to the fingerprint information (A) is modified as "hot block"; if the fingerprint information corresponding to the current data block is D (already exists in the secondary index table), the same reasoning applies.

[0064] If the fingerprint information corresponding to the current data block is B, the same fingerprint information is not found in the fingerprint information list in the Linux secondary index table, the fingerprint information B is added to the Linux secondary index table, and the type corresponding thereto is filled as "hot block".

[0065] If the fingerprint information of all data blocks is queried, and there are two fingerprint information (C and E) that have not been queried in the Linux secondary index table, the type of the data block corresponding to the fingerprint information (C and E) is classified as "cold block", and is not changed.

[0066] If the version type of the file corresponding to the current data block is FSL according to the file tag of the current data block, and the corresponding version type (i.e. FSL version type) is not found in the primary index entry in Figure 4 , the FSL version type is added in the primary index entry, and the corresponding secondary index table (i.e. FSL secondary index table) is newly created according to the FSL version type. And if the fingerprint information corresponding to the current data block is F, the fingerprint information F is added in the fingerprint information list in the newly created FSL secondary index table, and the type corresponding thereto is filled as "hot block".

[0067] The classification of the type of the data block can effectively improve the storage efficiency, make the storage of useful data more convenient, and save the storage space of the remaining repeated or invalid data in the edge server 2.

[0068] S5: If the type of the data block is hot block, the edge deduplication is performed on the data block; if the type of the data block is cold block, the cloud deduplication is performed on the data block.

[0069] Among them, the hot block includes a first hot block and a second hot block; the cold block includes a first cold block and a second cold block. The first hot block refers to a newly uploaded non-repeated data block, and the second hot block refers to a repeated data block. The first cold block refers to a data block not stored in the cloud server 1, and the second cold block refers to a data block stored in the cloud server 1.

[0070] In this embodiment, the method of edge deduplication includes the steps of: if the type of the data block is a first hot block, the data block is encrypted and saved; if the type of the data block is a second hot block, the data block is deleted.

[0071] Specifically, then, in the case of Figure 4For example, combined with the analysis in S4, it can be known that the data blocks with fingerprint information A and D are of type two hot blocks, the data blocks with fingerprint information B and F are of type one hot block, and the data blocks with fingerprint information C and E are of cold block.

[0072] Further, the uploaded data block of type one hot block is encrypted by the private key of the edge server 2 and saved in the edge data block storage module 28 of the edge storage area 23. The uploaded data block of type two hot block is directly deleted. The uploaded data block of type cold block is saved in the data communication buffer sub-module 203 in the buffer module 26 of the edge security area 22 for further processing.

[0073] Further, combined with the analysis in S4, it can be known that the data blocks with fingerprint information A and D are of type two hot blocks, the data blocks with fingerprint information B and F are of type one hot block, and the data blocks with fingerprint information C and E are of cold block. Figure 3 As shown in the figure, in the embodiment, the cloud-side deduplication method includes the following steps:

[0074] First, the fingerprint information corresponding to the data block of type cold block is sent to the cloud-side security area 12 in the cloud server 1 through the data communication buffer sub-module 203 in the edge server 2, and the corresponding fingerprint information is queried in the security index table 14.

[0075] If the same fingerprint information is queried in the security index table 14, it means that the type of the data block is type two cold block. If the same fingerprint information is not queried in the security index table 14, further query is performed in the global index table 15 in the cloud-side storage area 13. If the same fingerprint information is queried in the global index table 15, it means that the type of the data block is type two cold block. If the same fingerprint information is still not queried in the global index table 15, it means that the type of the data block is type one cold block.

[0076] Further, the above query result is fed back to the edge server 2. According to the result, the data communication buffer sub-module 203 encrypts the data block of type one cold block stored in it by the shared key and uploads it to the cloud data block storage module 16 in the cloud-side storage area 13 in the cloud server 1 for saving. The shared key is the same key shared by all, so it does not need to be further decrypted and can be directly saved.

[0077] Further, when downloading the file, the present application further includes the following steps:

[0078] Referring to Figure 3 , the client 3 sends the file tag corresponding to the file to be downloaded to the edge server 2. The edge server 2 finds the recipe information 27 of the corresponding file in the edge storage area 23 according to the file tag. The recipe information 27 records the data block composition content of the corresponding file, that is, which data blocks constitute the file.

[0079] Further, according to the recipe information 27, a plurality of data blocks corresponding to the file are found and obtained from the edge server 2 or the cloud server 1, and the plurality of data blocks are decrypted by the data recovery buffer sub-module 204 of the edge server 2 to reconstruct and obtain the file. The client 3 can further download the file from the edge server 2.

[0080] Corresponding to the cloud-edge collaborative deduplication method of the above embodiment, the embodiment of the present application also provides a cloud-edge collaborative deduplication system. For the sake of convenience, only the parts related to the embodiment of the present application are shown.

[0081] As shown in Figure 2 , Figure 3 The cloud-edge collaborative deduplication system includes:

[0082] The authentication unit 4 is configured to perform trusted authentication between the client 3 and the edge server 2, and trusted authentication between the edge server 2 and the cloud server 1.

[0083] The client 3 is configured to perform communication encryption on the to-be-uploaded data to obtain encrypted data and upload the encrypted data; wherein the encrypted data includes a plurality of data blocks of a file and a file tag corresponding to the file.

[0084] The edge server 2 is configured to perform communication decryption on the uploaded encrypted data, perform hash calculation on each data block to obtain fingerprint information corresponding to each data block, and perform type division on the plurality of data blocks according to the file tag, the fingerprint information corresponding to each data block, and a two-level index strategy; and the edge server 2 is further configured to perform edge deduplication on data blocks of a hot type.

[0085] The cloud server 1 is configured to perform cloud deduplication on data blocks of a cold type.

[0086] It should be noted that other technical features in the cloud-edge collaborative deduplication system described above are the same as the features disclosed in the above-mentioned method embodiments, and the above-mentioned corresponding method embodiments can be referred to for explanation, which will not be repeated here.

[0087] Based on the same inventive concept, the present application also provides an electronic device including a processor, a memory and a communication circuit, the processor being connected to the memory and the communication circuit respectively; wherein the communication circuit is configured to be communicatively connected, the memory is configured to store a computer program, and the processor is configured to execute the computer program to implement the cloud-edge collaborative deduplication method described above.

[0088] Referring to Figure 5 , the electronic device described in the embodiment of the present application can specifically include a processor 210 and a memory 220. The memory 220 is coupled to the processor 210.

[0089] The processor 210 is configured to control an operation of the electronic device, and can also be referred to as a CPU (Central Processing Unit). The processor 210 can be an integrated circuit chip having a processing capability of signals. The processor 210 can also be a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application-Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The general-purpose processor can be a microprocessor or the processor 210 can also be any conventional processor or the like.

[0090] The memory 220 is configured to store a computer program, and can be a RAM or a ROM, or other types of storage terminal. Specifically, the memory 220 can include one or more computer-readable storage media, which can be non-transitory or transitory. The memory 220 can also include a high-speed random access memory and a nonvolatile memory such as one or more disk storage terminals, flash memory terminals. In some embodiments, the non-transitory computer-readable storage medium in the memory 220 is configured to store at least one program code.

[0091] The processor 210 is configured to execute the computer program stored in the memory 220 to implement the method described in the method embodiments of the present application.

[0092] In some embodiments, the electronic device can further include a peripheral terminal interface 230 and at least one peripheral terminal. The processor 210, the memory 220 and the peripheral terminal interface 230 can be connected through a bus or a signal line. Each peripheral terminal can be connected to the peripheral terminal interface 230 through a bus, a signal line or a circuit board. Specifically, the peripheral terminal includes at least one of a radio frequency circuit 240, a display screen 250, an audio circuit 260 and a power supply 270.

[0093] The peripheral terminal interface 230 can be used to connect at least one I / O (Input / output) related peripheral terminal to the processor 210 and the memory 220. In some embodiments, the processor 210, the memory 220 and the peripheral terminal interface 230 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 210, the memory 220 and the peripheral terminal interface 230 can be implemented on a separate chip or circuit board, and the present embodiment is not limited in this regard.

[0094] The radio frequency circuit 240 is configured to receive and send RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 240 communicates with communication networks and other communication devices through electromagnetic signals. The radio frequency circuit 240 is the communication circuit of the electronic device. The radio frequency circuit 240 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 240 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, an operating personnel identity module, and the like. The radio frequency circuit 240 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network, and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 240 can also include NFC (Near Field Communication) related circuitry, which is not limited in this application.

[0095] The display screen 250 is configured to display a UI (User Interface). The UI can include graphics, text, icons, video, and any combination thereof. When the display screen 250 is a touch display screen, the display screen 250 also has the ability to collect touch signals on or above the surface of the display screen 250. The touch signals can be input as control signals to the processor 210 for processing. At this time, the display screen 250 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 250 can be one, arranged on the front panel of the electronic device; in other embodiments, the display screen 250 can be at least two, arranged on different surfaces of the electronic device or in a folding design; in other embodiments, the display screen 250 can be a flexible display screen, arranged on a curved surface or a folding surface of the electronic device. Even, the display screen 250 can also be arranged in an irregular shape, that is, a special-shaped screen. The display screen 250 can be made of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), and the like.

[0096] The audio circuit 260 can include a microphone and a speaker. The microphone is used to collect sound waves of the operator and the environment, and convert the sound waves into an electrical signal input to the processor 210 for processing, or input to the radio frequency circuit 240 to realize voice communication. For the purpose of stereo sound collection or noise reduction, the microphone can be multiple, respectively arranged at different parts of the electronic device. The microphone can also be an array microphone or an omnidirectional collection type microphone. The speaker is used to convert the electrical signal from the processor 210 or the radio frequency circuit 240 into sound waves. The speaker can be a traditional diaphragm speaker, or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, not only can the electrical signal be converted into a sound wave that humans can hear, but also can be converted into a sound wave that humans cannot hear for ranging purposes. In some embodiments, the audio circuit 260 can also include a headphone jack.

[0097] The power supply 270 is used to supply power to each component in the electronic device. The power supply 270 can be alternating current, direct current, disposable battery or rechargeable battery. When the power supply 270 includes a rechargeable battery, the rechargeable battery can be a wired charging battery or a wireless charging battery. The wired charging battery is a battery charged through a wired line, and the wireless charging battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0098] For the detailed description of the functions and execution processes of the functional modules or components in the electronic device embodiments of the present application, reference can be made to the description of the method embodiments of the present application described above, which will not be repeated here.

[0099] In several embodiments provided in the present application, it should be understood that the disclosed electronic device and method can be implemented in other ways. For example, the above-described electronic device embodiments are only schematic, for example, the division of modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0100] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment scheme.

[0101] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0102] Based on the same inventive concept, the present application further provides a computer readable storage medium storing a computer program, the computer program being executable by a processor to implement the cloud-edge collaborative deduplication method.

[0103] Referring to Figure 6 The integrated unit, if realized in the form of a software function unit and sold or used as an independent product, can be stored in the computer readable storage medium 300. Based on such understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions, can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions / computer programs for enabling an Internet of Things device (which can be a personal computer, a server, or a network terminal, etc.) or a processor to perform all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media having the above storage medium, and an electronic terminal such as a computer, a mobile phone, a notebook computer, a tablet computer, a camera, etc.

[0104] The execution process of the program data in the computer readable storage medium can be referred to the description of the above-mentioned method embodiments of the present application, and will not be repeated here.

[0105] It can be seen that the application discloses a cloud-edge collaborative data deduplication method, system, device and medium, and the method comprises the following steps: respectively performing trusted authentication between a client and an edge server, and trusted authentication between the edge server and a cloud server; performing communication encryption on to-be-uploaded data to obtain encrypted data and perform uploading; wherein the encrypted data comprises a plurality of data blocks of a file and a file tag corresponding to the file; after performing communication decryption on the uploaded encrypted data, performing hash calculation on each data block to obtain fingerprint information corresponding to each data block; according to the file tag, the fingerprint information corresponding to each data block and a two-level index strategy, dividing the plurality of data blocks by type; if the type of the data block is a hot block, performing edge deduplication on the data block; if the type of the data block is a cold block, performing cloud deduplication on the data block. The application realizes effective isolation of communication by creating a cloud security zone and an edge security zone, and guarantees the safety of the communication process. The shared key is obtained while performing trusted authentication, and the management and storage overhead of the key is reduced. Further, the two-level index strategy is used to realize edge deduplication and cloud deduplication, which not only reduces the pressure of cloud storage, but also improves the storage efficiency and data management capability of the whole system. The application not only improves the data communication efficiency and storage efficiency, but also further reduces the communication, encryption and storage costs, and realizes efficient balance of system performance and data safety.

[0106] The above is only an embodiment of the application, and does not limit the patent range of the application, and any equivalent structural transformation or direct or indirect application in other related technical fields by using the content of the specification and drawings of the application is also included in the patent protection range of the application.

Claims

1. A cloud-edge collaborative deduplication method, characterized in that, The method comprises the steps of: respectively performing trusted authentication between the client and the edge server, and trusted authentication between the edge server and the cloud server; communicating encryption is performed on the data to be uploaded, encrypted data is obtained, and the encrypted data is uploaded; wherein the encrypted data comprises a plurality of data blocks of a file and a file tag corresponding to the file; after the encrypted data uploaded is decrypted, hash calculation is performed on each data block to obtain fingerprint information corresponding to each data block; according to the file tag, the fingerprint information corresponding to each data block, and a two-level index strategy, the plurality of data blocks are divided by type, specifically comprising: according to the file tag corresponding to each data block, the version type of the file corresponding to the data block is identified; the corresponding version type is searched in a first index entry, and the corresponding second index table is located according to the version type; wherein the second index table comprises a fingerprint information list, an address list and a type list; according to the fingerprint information corresponding to the data block, the fingerprint information list is queried, and the type of the corresponding data block in the type list is updated; the types of the data blocks corresponding to the updated part in the type list are divided into hot blocks, and the types of the data blocks corresponding to the non-updated part in the type list are divided into cold blocks; if the type of the data block is a hot block, edge deduplication is performed on the data block; if the type of the data block is a cold block, cloud deduplication is performed on the data block.

2. The cloud-edge collaborative deduplication method of claim 1, wherein, The method for trusted authentication between the client and the edge server comprises the steps of: The edge server creates an edge security zone, and the edge security zone generates a first proof report; wherein the first proof report comprises identity information and state measurement value of the edge security zone; The edge security zone sends the first proof report to an authentication unit, and the authentication unit verifies the first proof report to obtain a first signature report; The authentication unit sends the first signature report to the client through the edge server, and the client verifies the first signature report to determine the trustworthiness of the edge security zone. 3.The cloud-edge collaborative deduplication method of claim 1, wherein, The method for trusted authentication between the edge server and the cloud server comprises the steps of: The cloud server creates a cloud security zone and a key security zone, and a plurality of edge security zones corresponding to the plurality of edge servers are created; The key security zone verifies the trustworthiness of the plurality of edge security zones and generates a shared key and a plurality of private keys; wherein the private key corresponds to the edge security zone one by one; The key security zone generates a second proof report; wherein the second proof report comprises fingerprint information of the shared key, fingerprint information of the private key, and identity information of the key security zone; The key security zone sends the second proof report to an authentication unit, and the authentication unit verifies the second proof report to obtain a second signature report; The authentication unit sends the second signature report to each edge server through the key security area, the edge server verifies the second signature report, judges the trustworthiness of the key security area, and obtains the shared key and the corresponding private key.

4. The cloud-edge collaborative deduplication method of claim 1, wherein, The edge deduplication method comprises the following steps: The hot block comprises a first type of hot block and a second type of hot block; If the type of the data block is the first type of hot block, the data block is encrypted and saved; If the type of the data block is the second type of hot block, the data block is deleted.

5. The cloud-edge collaborative deduplication method of claim 1, wherein, The cloud deduplication method comprises the following steps: The fingerprint information corresponding to the data block of the cold block type is sent; wherein the cold block comprises a first type of cold block and a second type of cold block; According to the fingerprint information, the type of the data block is compared and judged again, and the second comparison and judgment result is fed back; If the type of the data block is the first type of cold block, the data block is encrypted and saved after uploading; If the type of the data block is the second type of cold block, no processing is performed.

6. The cloud-edge collaborative deduplication method of claim 1, wherein, Further comprising the following steps: The file tag corresponding to the file to be downloaded is sent; According to the file tag, the corresponding formula information is found; According to the formula information, the data blocks corresponding to the file are obtained, and the data blocks are decrypted to reconstruct the file and download.

7. A cloud-edge collaborative deduplication system, comprising: Comprise: An authentication unit is configured to perform trusted authentication between a client and an edge server, and trusted authentication between the edge server and a cloud server; A client is configured to perform communication encryption on to-be-uploaded data, obtain encrypted data, and upload the encrypted data; wherein the encrypted data comprises a plurality of data blocks of a file and a file tag corresponding to the file; An edge server is configured to perform communication decryption on the uploaded encrypted data, perform hash calculation on each data block, and obtain fingerprint information corresponding to each data block; Further configured to divide the plurality of data blocks into different types according to the file tag, the fingerprint information corresponding to each data block, and a two-level index strategy, specifically comprising: Identify the version type of the file corresponding to each data block according to the file tag corresponding to each data block; Find the corresponding version type in a first index entry, and locate a corresponding second index table according to the version type; wherein the second index table comprises a fingerprint information list, an address list, and a type list; Query the fingerprint information list according to the fingerprint information corresponding to the data block, and update the type of the corresponding data block in the type list; Divide the types of the data blocks corresponding to the updated part in the type list into hot blocks, and divide the types of the data blocks corresponding to the un-updated part in the type list into cold blocks; Further configured to perform edge deduplication on the data blocks of the hot block type; A cloud server is configured to perform cloud deduplication on the data blocks of the cold block type.

8. An electronic device, comprising: A computer program product comprising a storage medium to store the program code of a computer program, wherein the program code is executable by a processor to cause the processor to perform the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program is executable by a processor to cause the processor to perform the steps of the method according to any one of claims 1 to 6.

Images