A secure storage system for cloud-edge integration

Abstract

The present invention discloses a cloud-edge integrated secure storage system, which belongs to the field of data storage security. It includes: the system is composed of multiple edge terminals and a central cloud storage terminal; the edge terminal performs efficient filtering and processing on large-scale data, and then performs secure processing and encryption based on edge-end integrated homomorphic encryption, and then stores it; the edge terminal deduplicates the key data in the stored data on the central cloud storage terminal and then securely stores it through active encryption, and the central cloud storage terminal provides the same performance guarantee for different edge terminals through performance isolation perceived by file system behavior; the edge terminal and the central cloud storage terminal cooperate to perform cloud-edge secure deduplication, cloud-edge secure synchronization and cloud-edge data tiered storage tasks, and the execution of cloud-edge tasks is scheduled by the intelligent agent maintained by the central cloud storage terminal according to the requirements of different cloud-edge tasks and the status of the edge terminal. The present invention can take into account the performance and security of the cloud-edge storage system, optimize storage performance, reduce system latency, and ensure data security and reliability.

Description

Technical Field

[0001] The present invention belongs to the field of data storage security technology, and more specifically, relates to a secure storage system in a cloud-edge fusion scenario. Background Art

[0002] Cloud-edge convergence refers to the integration of cloud computing and edge computing, leveraging their respective strengths to achieve more efficient, intelligent, and convenient data processing and application services. In cloud-edge convergence, edge computing, as an extension of cloud computing, plays a key role in connectivity, processing, and applications. Cloud computing provides supporting services such as storage, computing, and security, enabling data to flow freely between the edge and cloud, enabling intelligent applications.

[0003] Current cloud-edge converged secure storage systems face a series of challenges: 1) How to ensure secure data storage. In cloud-edge converged scenarios, edge computing devices are typically deployed geographically in an open environment, making them vulnerable to attacks. Data integrity and availability cannot be guaranteed, and thus data storage cannot be secure. 2) How to ensure secure and efficient data transmission. Data transmission faces the risk of data leakage. Furthermore, the transmission of massive amounts of data is constrained by issues such as limited bandwidth and network connection latency. 3) How to ensure trusted computing. In a cloud-edge converged system, the entire system is often composed of multiple organizations that do not trust each other. When the system as a whole provides external services or performs collaborative computing between various parties, transparency of the computing process between the system and all parties must be ensured. However, current cloud-edge systems lack a transparent and trusted computing platform, making it impossible to guarantee transparency and security of the computing process within the system and among all parties. 4) How to ensure trusted management of the system. Edge scenarios involve various edge and terminal devices. These devices are dynamic and may join or exit the network at any time. They may also engage in harmful behaviors for profit reasons. A secure device supervision platform is needed to monitor device behavior. However, the current cloud-edge fusion system lacks a trusted system supervision and auditing platform.

[0004] In a cloud-edge converged environment, cloud and edge devices often exhibit significant heterogeneity, reflected in significant disparities in computing power and storage capacity. Edge devices have weak computing power and small storage capacity, but are located closer to the data source. Faced with massive amounts of data, they must possess efficient methods for filtering and processing. Cloud devices, however, possess strong computing power and large storage capacity, but are located further from the data source. They can complement the edge's computing and storage capabilities. Since the cloud serves multiple edge devices, it must provide fair storage services and ensure performance isolation. The cloud and edge not only exhibit significant heterogeneity but also face different security risks. Edge devices are closer to users and geographically distributed, making them more vulnerable to attacks and requiring various measures to ensure data security. The cloud and edge must collaborate to process and store data, requiring protection of data security during transmission. The cloud offers stronger security, storing large amounts of data from different edge devices. Some data may be encrypted using weak encryption algorithms during upload due to limited edge computing power. Therefore, strong encryption of this data is required in the cloud.

[0005] To address these existing problems, researchers at home and abroad have proposed a variety of cloud-edge integrated secure storage architectures, but they all focus only on one aspect of security and performance, without considering the security, performance, and communication delay of the entire system at the same time, and are unable to balance security and performance. Summary of the Invention

[0006] In response to the above defects or improvement needs of the existing technology, the present invention provides a cloud-edge integrated security storage system, the purpose of which is to take into account the performance and security of the cloud-edge storage system, optimize storage performance, reduce system latency, and ensure data security and reliability.

[0007] To achieve the above objectives, according to one aspect of the present invention, a cloud-edge integrated secure storage system is provided, comprising:

[0008] Multiple edge terminals and central cloud storage terminals;

[0009] The edge performs efficient filtering and processing on large-scale data, and then securely processes and encrypts the data based on edge-integrated homomorphic encryption, and then stores it;

[0010] The edge end deduplicates the key data in the stored data on the central cloud storage end and then securely stores it through active encryption; the central cloud storage end provides the same performance guarantee for different edge ends through performance isolation perceived by file system behavior;

[0011] The edge end and the central cloud storage end cooperate with each other to schedule the operation of cloud-edge tasks related to deduplication, synchronization and data tiered storage; wherein, when any of the edge ends initiates a synchronization request to the central cloud storage end as a synchronization request direction, the central cloud storage end determines all synchronization participants according to the pre-stored synchronization rules, and the synchronization requesting party determines whether all synchronization participants are in the same local area network. If they are in the same local area network, the cloud-edge secure storage synchronization process is directly carried out; if they are not in the same local area network, the synchronization requesting party and the synchronization participants are controlled to establish a direct and confidential communication tunnel with the assistance of the central storage end, and the cloud-edge secure storage synchronization process is carried out in the communication tunnel.

[0012] Furthermore, when the size of the file to be synchronized between the synchronization requester and the synchronization participant is lower than the transmission volume threshold, full synchronization is performed; wherein the synchronization process satisfies causal consistency; and the transmission volume threshold is calculated based on the current network bandwidth.

[0013] The transmission volume threshold is calculated based on the current network bandwidth.

[0014] Furthermore, when the size between the synchronization requester and the synchronization participant is greater than the transmission volume threshold, incremental synchronization is performed; wherein, the synchronization process satisfies causal consistency, and the incremental synchronization uses an improved content-defined blocking technology with guess jumps to accelerate the synchronization process.

[0015] Furthermore, the central cloud storage end uses an intelligent agent to online decide whether to create a data copy of the original data at the edge end and store it in layers at each moment according to the current environmental factors of the original data;

[0016] The current environmental factors include at least one of a cost model, a request volume, an access delay, and an access frequency.

[0017] Furthermore, the intelligent agent maintained by the central cloud storage end is used to schedule tasks, synchronize cloud-edge tasks, deduplicate and stratify data according to the memory requirements, CPU requirements, delay thresholds, cloud-edge network status and edge end status of different cloud-edge tasks.

[0018] Furthermore, a maximum acceptable response time is set for each of the cloud-edge tasks, and whether the scheduling is successful is determined based on the actual response time. Different reward functions are set for successful scheduling and failed scheduling to obtain the difference in the impact of different scheduling decisions on the results.

[0019] Furthermore, if the data copy created by the edge end is modified, the edge end initiates a synchronization request to the central cloud storage end to synchronize the original data in the central cloud storage end with the data copy of the edge end.

[0020] Furthermore, if the original data corresponding to the edge data copy in the central cloud storage end is modified, the central cloud storage end initiates a synchronization request to synchronize with the edge end storing the data copy to ensure data consistency.

[0021] Furthermore, the edge end encrypts the stored data through edge-integrated homomorphic encryption technology to obtain secure data, and some key data is transmitted to the central cloud storage end, and the central cloud storage end can operate the secure data without decryption.

[0022] Furthermore, the central cloud storage terminal selects an encryption algorithm according to the current system status and resource usage information to perform strong encryption processing on the weakly encrypted data stored in the terminal.

[0023] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0024] (1) The cloud-edge fusion secure storage system of the present invention addresses the strong heterogeneity and varying security risks of cloud-edge resources. It balances the performance and security of the cloud-edge fusion storage system, provides a unified level of security assurance for heterogeneous cloud-edge resources with varying security risks, and achieves integrated cloud-edge fusion security. It also provides a unified perspective of performance assurance for heterogeneous cloud-edge resources, achieving performance integration. The system offers advantages such as high performance, low latency, and reliable security assurance.

[0025] (2) This solution provides an efficient and secure cloud-edge storage synchronization method. The synchronization process first determines the file size threshold based on the network bandwidth of the communication link between the synchronization participant and the synchronization requester. When the file to be synchronized is smaller than the file size threshold, full synchronization is used to avoid the additional overhead introduced by the block process in the incremental update. When the file size to be synchronized is larger than the threshold, incremental update is used. By dividing the file into blocks and transmitting only the modified part, the bandwidth waste caused by transmitting the entire file can be avoided, and the synchronization process delay can be reduced. This method takes into account both computational overhead and network transmission overhead, and has the advantages of low overhead and low latency.

[0026] (3) This solution provides an efficient data tiered storage method. Edge storage has the advantages of low latency and low cost, but the edge storage space is small and the security risk is high; the central cloud storage has the advantages of large storage capacity and high security, but the central cloud storage has high storage cost and is far away from the user, resulting in the disadvantage of high latency. Therefore, the respective advantages of the edge and central cloud storage can be combined to set a cost model. Based on factors such as the cost model, request volume, access latency, and access frequency, the intelligent agent can decide online whether to create a data copy at the edge and the layer (hot layer, cold layer) in the cloud center at each moment. At the same time, the synchronization method is used to ensure the consistency of the original data and the copy data. By storing data in layers, this method can reduce data storage costs, reduce data access and operation latency, and improve storage system performance.

[0027] (4) This solution provides an efficient scheduling method for cloud-edge tasks. Tasks are scheduled based on their memory requirements, CPU requirements, latency thresholds, cloud-edge network status, and edge status. Each cloud-edge task is assigned a maximum acceptable response time. The success of the scheduling is determined based on the actual response time, and the agent is adjusted accordingly. This method can adjust the execution of numerous cloud-edge transmission tasks, such as synchronization, deduplication, and data tiered storage, based on the current cloud-edge status, so that various cloud-edge transmission tasks are executed in a reasonable order, improving the overall performance of the system.

[0028] (5) This solution provides an edge-integrated homomorphic encryption technology that can encrypt local data on the edge, allowing the cloud to operate edge data without decryption. It can also take advantage of the large number of edge servers and fully utilize the computing power of edge servers to collaboratively and securely process data.

[0029] (6) This solution provides a method for secure data storage with active encryption. It selects a strong encryption algorithm to encrypt weakly encrypted data uploaded by the edge to ensure security. Due to the characteristics of the edge, such as the large number of tasks it performs and limited performance, the encryption of data may be weak. The cloud can ensure data security by strongly encrypting some weakly encrypted data. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Illustration of the cloud-edge integrated secure storage system module designed for the present invention.

[0031] Figure 2 The cloud-edge integrated secure storage system architecture and flow chart designed for this invention.

[0032] Figure 3 A relationship diagram of the cloud-edge modules of the cloud-edge integrated secure storage system designed for the present invention. DETAILED DESCRIPTION

[0033] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0034] like Figure 1-3 As shown, the cloud-edge integrated secure storage system of the present invention includes multiple edge terminals and a central cloud storage terminal; the edge terminal performs efficient filtering and processing on large-scale data, and then encrypts the data based on edge-integrated homomorphic encryption for storage; the edge terminal deduplicates the key data in the stored data on the central cloud storage terminal and then securely stores it through active encryption; the central cloud storage terminal provides the same performance guarantee for different edge terminals through performance isolation perceived by file system behavior; the edge terminal and the central cloud storage terminal cooperate to schedule the operation of cloud-edge tasks related to deduplication, synchronization and data tiered storage; wherein, when any edge terminal initiates a synchronization request to the central cloud storage terminal as a synchronization request direction, the central cloud storage terminal determines all synchronization participants according to the pre-stored synchronization rules, and the synchronization request party determines whether all synchronization participants are in the same local area network. If they are in the same local area network, the cloud-edge secure storage synchronization process is directly carried out; if they are not in the same local area network, the synchronization request party and the synchronization participant are controlled to establish a direct and confidential communication tunnel with the assistance of the central storage terminal, and the cloud-edge secure storage synchronization process is carried out in the communication tunnel.

[0035] The working process of the cloud-edge integrated secure storage system involves three aspects:

[0036] (1) By designing various modules at the edge, between the cloud and edge, and the central cloud storage end, the performance integration of cloud-edge fusion is achieved. The performance integration of cloud-edge fusion refers to providing a unified perspective of performance guarantee for heterogeneous cloud-edge resources. Specifically, at the edge, an application-aware near-data processing method is used to achieve efficient filtering and processing of large-scale data; between the cloud and edge, an efficient cloud-edge federated learning method is used to train models on edge data; data stored in the central cloud is stored in layers using an efficient cloud-edge data tiering method, and data copies are created at the edge to reduce data access latency. The efficient cloud-edge task scheduling method is used to efficiently process cross-cloud-edge transmission tasks in real time; and at the central cloud storage end, a file system behavior-aware performance isolation method is used to further achieve performance isolation of resources for different tenants.

[0037] (2) By designing various modules at the edge, between the cloud and edge, and the central cloud storage end, the security integration of cloud and edge is achieved. The security integration of cloud and edge refers to providing a unified level of security protection for heterogeneous cloud and edge resources with different security risks. Specifically, at the edge, the edge data is securely processed through edge-integrated homomorphic encryption technology; between the cloud and edge, the data uploaded by edge users is deduplicated through the cloud-edge security deduplication method, and the consistency and synchronization of data between the edge and cloud are securely and efficiently guaranteed through the cloud-edge security storage synchronization method; at the central cloud storage end, the encryption algorithm in the resource-limited scenario is actively selected for weakly encrypted data through the active encryption data security storage method, and the deduplicated data is strongly encrypted using the encryption method.

[0038] (3) Optimize cloud-edge storage efficiency and reliability by designing various modules at the edge and central cloud storage. Specifically, various modules are designed at the edge and central cloud storage to optimize cloud-edge storage efficiency and reliability, including optimizing storage efficiency and security at the edge through a joint optimization method of storage efficiency and security; and ensuring data storage reliability at the central cloud storage through a storage reliability assurance method based on super-fault domains.

[0039] In this embodiment, the application-aware near-data processing method in (1) adds modules such as data filtering, encryption and decryption, and hash calculation to the main control of the memory, processes data at the source of data storage, solves the problem of mismatch between processor processing speed and storage I / O speed, and performs efficient filtering and processing on large-scale data at the edge. The efficient federated learning method in (1) can adopt a federated learning framework based on lottery theory and adaptive differential privacy protection, including three processes: client grouping, iterative pruning based on lottery theory, and fine-tuning training. The cloud-edge efficient task scheduling method in (1) can use a cloud-edge collaborative task real-time scheduling method based on DDQN and a resource allocation method for multi-tenant application delay awareness. The cloud-edge efficient data tiering method in (1) can utilize the complementary advantages of cloud storage and edge storage, comprehensively consider factors such as cost, request volume, access delay, and access frequency, and decide online through the intelligent agent whether to create a data copy at the edge at each moment, as well as the data tier (hot tier, cold tier) in the cloud storage. The file system behavior-aware performance isolation method in (1) can record the performance requirements of the edge end in the metadata of the virtual machine image file, reuse the data path to propagate the edge end performance requirements, and build a logically centralized I / O scheduler that is aware of the file system access behavior on the data plane to allocate memory and disk resources according to the edge end performance requirements.

[0040] In this embodiment, the edge-integrated homomorphic encryption in (2) uses a homomorphic encryption algorithm to encrypt and process data, and establishes a common key through key negotiation; the cloud-edge security deduplication algorithm in (2) can use a low-overhead key management method based on cloud-edge collaboration, a duplicate data removal method for a multi-control storage system, and a deduplication storage and query method for cluster internal network communication traffic logs; the cloud-edge secure storage synchronization method in (2) can use an adaptive synchronization method to select incremental synchronization and full synchronization according to different network environments and file sizes, and use a CDC technology based on content-defined blocks combined with guess jumps to optimize incremental synchronization, and use encryption technology such as AES to ensure data security during the synchronization process. This method can be combined with kubeedge for cloud-edge scenarios. Kubuedge sinks the container orchestration capabilities of Kubernetes to the edge, facilitating the deployment of edge containers. By deploying the synchronization request processing module, edge resource monitoring module and communication coordination module in the cloud controller, it is possible to process edge synchronization requests, assist the edge in establishing communication tunnels, and monitor the resource usage information of the edge. At the same time, a Pod is deployed on the edge to carry out the subsequent synchronization process. The Pod contains various containers that implement the change monitoring module, network evaluation module, file and resource evaluation module, synchronization module (including full synchronization and incremental synchronization), network communication module and encryption module.

[0041] In this embodiment, the storage efficiency and security joint optimization method in (3) can ensure the efficiency and security of edge data operations by implementing a distributed key-value database, a multimedia data query algorithm based on a searchable encryption strategy, and a security threat perception and defense method for edge devices (including adversarial samples and tampered content detection for video image classification, backdoor attack and defense methods for images, and memory leak detection methods based on memory utilization curves, etc.). The super-fault domain storage guarantee method in (3) can be implemented using a super-fault domain multi-channel video data processing method. The traditional erasure strip storage method adopts a random write mode. In the video storage system, when facing the super-fault domain data loss problem, the traditional random write mode is likely to cause the image group (Group of Pictures, abbreviated as GOP) packets in the hard disk to be too correlated, resulting in difficulty in video data recovery after a hard disk failure. The super-fault domain multi-channel video data processing method collects and pre-processes video data through different channels, splits it into GOPs, and performs erasure coding; calculates the difference between GOP packets between different channels based on the generation time of each GOP packet; and stores the GOP packets with the largest difference in each channel on the same hard disk based on the maximum difference distribution principle. This method can reduce the risk of data loss under super fault domain and improve recovery accuracy.

[0042] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A cloud-edge integrated secure storage system, characterized by: include: Multiple edge terminals and central cloud storage terminals; The edge performs efficient filtering and processing on large-scale data, and then securely processes and encrypts the data based on edge-integrated homomorphic encryption, and then stores it; The edge end deduplicates the key data in the stored data on the central cloud storage end and stores it securely through active encryption; The central cloud storage end provides the same performance guarantee for different edge ends through performance isolation of file system behavior awareness; The edge end and the central cloud storage end cooperate with each other to schedule the operation of cloud-edge tasks related to deduplication, synchronization and data tiered storage; wherein, when any of the edge end initiates a synchronization request to the central cloud storage end as a synchronization request direction, the central cloud storage end determines all synchronization participants according to the pre-stored synchronization rules, and the synchronization requesting party determines whether all synchronization participants are in the same local area network. If they are in the same local area network, the cloud-edge secure storage synchronization process is directly carried out; If the synchronization requester and the synchronization participant are not controlled in the same local area network, a direct and confidential communication tunnel is established with the assistance of the central storage end, and a cloud-edge secure storage synchronization process is performed in the communication tunnel.

2. The cloud-edge fusion secure storage system according to claim 1, characterized in that: When the size of the file to be synchronized between the synchronization requester and the synchronization participant is lower than the transmission volume threshold, full synchronization is performed; wherein the synchronization process satisfies causal consistency; the transmission volume threshold is calculated based on the current network bandwidth.

3. The cloud-edge fusion secure storage system according to claim 2, characterized in that: When the size between the synchronization requester and the synchronization participant is greater than the transmission volume threshold, incremental synchronization is performed; wherein the synchronization process satisfies causal consistency, and the incremental synchronization uses an improved content-defined blocking technology with guess jumps to accelerate the synchronization process.

4. The cloud-edge fusion secure storage system according to claim 1, characterized in that: The central cloud storage end uses an intelligent agent to decide online whether to create a data copy of the original data at each moment on the edge end and store it in layers according to the current environmental factors of the original data; The current environmental factors include at least one of a cost model, a request volume, an access delay, and an access frequency.

5. The cloud-edge fusion secure storage system according to claim 4, characterized in that: The intelligent agent maintained by the central cloud storage end is used to schedule tasks according to the memory requirements, CPU requirements, delay thresholds, cloud-edge network status and edge end status of different cloud-edge tasks. Cloud-edge tasks include synchronization, deduplication and data stratification.

6. The cloud-edge fusion secure storage system according to claim 5, characterized in that: A maximum acceptable response time is set for each cloud-edge task, and whether the scheduling is successful is determined based on the actual response time. Different reward functions are set for successful and failed scheduling to obtain the difference in the impact of different scheduling decisions on the results.

7. The cloud-edge fusion secure storage system according to claim 4, characterized in that: If the data copy created by the edge end is modified, the edge end initiates a synchronization request to the central cloud storage end to synchronize the original data in the central cloud storage end with the data copy of the edge end.

8. The cloud-edge fusion secure storage system according to claim 7, characterized in that: If the original data corresponding to the edge data copy in the central cloud storage end is modified, the central cloud storage end initiates a synchronization request to synchronize with the edge end storing the data copy to ensure data consistency.

9. The cloud-edge fusion secure storage system according to claim 1, characterized in that: The edge end encrypts the stored data through edge-end integrated homomorphic encryption technology to obtain secure data, and transmits some key data to the central cloud storage end. The central cloud storage end can operate the secure data without decryption.

10. The cloud-edge fusion secure storage system according to claim 9, characterized in that: The central cloud storage terminal selects an encryption algorithm according to the current system status and resource usage information to perform strong encryption on the weakly encrypted data stored in the terminal.

Images