Cloud-edge fused secure storage system

Abstract

The invention discloses a secure storage system for cloud edge fusion, and belongs to the field of data storage security. Comprising a plurality of edge ends and a central cloud storage end; the edge end carries out efficient filtering processing on large-scale data, then homomorphic encryption security processing and encryption based on edge end integration are carried out, and then storage is carried out; key data in the stored data is safely stored through active encryption after deduplication at the central cloud storage end by the edge ends, and the central cloud storage end provides the same performance guarantee for different edge ends through performance isolation of file system behavior perception; the edge end and the central cloud storage end cooperate with each other to execute cloud-side security de-duplication, cloud-side security synchronization and cloud-side data hierarchical storage tasks, and execution of the cloud-side tasks is scheduled by an intelligent agent maintained by the central cloud storage end according to requirements of different cloud-side tasks and the state of the edge end. Performance and safety of the cloud side storage system can be considered, storage performance is optimized, system delay is reduced, and data safety and reliability are ensured.

Description

Technical Field

[0001] The present invention belongs to the field of data storage security technology, and more specifically, relates to a secure storage system in a cloud-edge fusion scenario. Background Art

[0002] Cloud-edge fusion refers to the integration of cloud computing and edge computing, using the advantages of cloud computing and edge computing to achieve more efficient, smarter and more convenient data processing and application services. In cloud-edge fusion, edge computing, as an extension of cloud computing, plays an important role in connection, processing and application, while cloud computing provides support services such as storage, computing and security, allowing data to flow freely between the edge and the cloud to realize intelligent applications.

[0003] The current cloud-edge fusion security storage system faces a series of challenges: 1) How to ensure the secure storage of data. In the cloud-edge fusion scenario, edge computing devices are generally deployed geographically in a distributed manner and in an open environment. Edge devices are vulnerable to attacks, and the integrity and availability of data cannot be guaranteed, and the secure storage of data cannot be guaranteed. 2) How to ensure the secure and efficient transmission of data. In the process of data transmission, there is a risk of data leakage. At the same time, due to problems such as limited transmission bandwidth and network connection delay, the transmission of massive data restricts the development of cloud-edge fusion. 3) How to ensure trusted computing. In a cloud-edge fusion system, the entire system is often composed of multiple organizations that do not trust each other. When the system provides services to the outside world as a whole or when all parties perform collaborative computing, it is necessary to ensure the transparency of the computing process between the system and all parties. However, the current cloud-edge system lacks a transparent and trusted computing platform, and cannot guarantee the transparency and security of the computing process of the system and all parties. 4) How to ensure the trusted management of the system. Edge scenarios involve various types of edge and terminal devices. These devices are somewhat dynamic and may join or exit the network at any time. These devices may also engage in harmful behavior for profit reasons. A secure device supervision platform is needed to supervise device behavior. However, the current cloud-edge fusion system lacks a trusted system supervision and auditing platform.

[0004] In the cloud-edge fusion environment, cloud devices and edge devices often have strong heterogeneity, which is reflected in the huge difference in computing power and storage capacity. Edge devices have poor computing power and small storage capacity, but are close to the source of data generation. In the face of massive data, they must have an efficient method to filter and process data; cloud devices have strong computing power and large storage capacity, but are far away from the source of data generation. They can serve as a supplement to the edge in computing and storage. Since the cloud needs to serve multiple edge terminals, it is necessary to provide fair storage services to ensure performance isolation. The cloud and edge terminals not only have strong heterogeneity, but also face different security risks. Edge devices are closer to users and are geographically distributed. They are more vulnerable to attacks and require various means to ensure data security; the cloud and edge terminals need to cooperate in processing and storing data, and need to protect the security of data during transmission; the cloud has strong security and stores a large amount of data from different edge terminals. Some data are encrypted using weak encryption algorithms when uploading due to the limitation of edge computing power. In this case, such data needs to be strongly encrypted in the cloud.

[0005] In response to these existing problems, researchers at home and abroad have proposed a variety of cloud-edge integrated secure storage architectures, but they all only focus on one aspect of security and performance, and do not consider the security, performance, and communication delay of the entire system at the same time, and are unable to take into account both security and performance. Summary of the invention

[0006] In response to the above defects or improvement needs of the prior art, the present invention provides a cloud-edge integrated secure storage system, which aims to take into account the performance and security of the cloud-edge storage system, optimize storage performance, reduce system latency, and ensure data security and reliability.

[0007] To achieve the above object, according to one aspect of the present invention, a cloud-edge integrated secure storage system is provided, comprising:

[0008] Multiple edge terminals and central cloud storage terminals;

[0009] The edge performs efficient filtering and processing on large-scale data, and then securely processes and encrypts the data based on edge-integrated homomorphic encryption, and then stores it;

[0010] The edge end stores the key data in the stored data securely through active encryption after deduplication at the central cloud storage end; the central cloud storage end provides the same performance guarantee for different edge ends through performance isolation perceived by file system behavior;

[0011] The edge end and the central cloud storage end cooperate with each other to schedule the running of cloud-edge tasks related to deduplication, synchronization and data tiered storage; wherein, when any of the edge ends initiates a synchronization request to the central cloud storage end as a synchronization requester, the central cloud storage end determines all synchronization participants according to pre-stored synchronization rules, and the synchronization requester determines whether all synchronization participants are in the same local area network. If they are in the same local area network, the cloud-edge secure storage synchronization process is directly carried out; if they are not in the same local area network, the synchronization requester and the synchronization participants are controlled to establish a direct and confidential communication tunnel with the assistance of the central storage end, and the cloud-edge secure storage synchronization process is carried out in the communication tunnel.

[0012] Furthermore, when the size of the file to be synchronized between the synchronization requester and the synchronization participant is lower than the transmission volume threshold, full synchronization is performed; wherein the synchronization process satisfies causal consistency; and the transmission volume threshold is calculated based on the current network bandwidth.

[0013] The transmission volume threshold is calculated according to the current network bandwidth.

[0014] Furthermore, when the size between the synchronization requester and the synchronization participant is greater than the transmission volume threshold, incremental synchronization is performed; wherein the synchronization process satisfies causal consistency, and the incremental synchronization uses an improved content-defined block technology with guess jumps to accelerate the synchronization process.

[0015] Furthermore, the central cloud storage end uses an intelligent agent to online decide whether to create a data copy of the original data at the edge end and store it in layers at each moment according to the current environmental factors of the original data;

[0016] The current environmental factors include at least one of a cost model, a request volume, an access delay, and an access frequency.

[0017] Furthermore, the intelligent agent maintained by the central cloud storage end is used to schedule tasks according to the memory requirements, CPU requirements, delay thresholds, cloud-edge network status and edge terminal status of different cloud-edge tasks, and to synchronize, deduplicate and stratify cloud-edge tasks.

[0018] Furthermore, a maximum acceptable response time is set for each of the cloud-edge tasks, and whether the scheduling is successful is determined based on the actual response time. Different reward functions are set for successful scheduling and failed scheduling to obtain the difference in the impact of different scheduling decisions on the results.

[0019] Furthermore, if the data copy created by the edge end is modified, the edge end initiates a synchronization request to the central cloud storage end to synchronize the original data in the central cloud storage end with the data copy of the edge end.

[0020] Furthermore, if the original data in the central cloud storage terminal and corresponding to the data copy at the edge terminal is modified, the central cloud storage terminal initiates a synchronization request to synchronize with the edge terminal storing the data copy to ensure data consistency.

[0021] Furthermore, the edge end encrypts the stored data through edge-end integrated homomorphic encryption technology to obtain secure data, and some key data is transmitted to the central cloud storage end, and the central cloud storage end can operate the secure data without decryption.

[0022] Furthermore, the central cloud storage terminal selects an encryption algorithm according to the current system status and resource usage information to perform strong encryption processing on the weakly encrypted data stored in the terminal.

[0023] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0024] (1) The cloud-edge fusion security storage system of the present invention is proposed in view of the characteristics of cloud-edge resources with strong heterogeneity and different security risks. It can take into account the performance and security of the cloud-edge fusion storage system, provide a unified level of security protection for heterogeneous cloud-edge resources with different security risks, and realize cloud-edge fusion security integration; it also provides a unified perspective of performance protection for heterogeneous cloud-edge resources to achieve performance integration. The system has the advantages of high performance, low latency and reliable security guarantee.

[0025] (2) This solution provides an efficient and secure cloud-edge storage synchronization method. The synchronization process first determines the file size threshold based on the network bandwidth of the communication link between the synchronization participant and the synchronization requester. When the file to be synchronized is smaller than the file size threshold, full synchronization is used to avoid the additional overhead introduced by the block process in the incremental update. When the file size to be synchronized is larger than the threshold, incremental update is used. By dividing the file into blocks and transmitting only the modified part, the waste of bandwidth caused by transmitting the entire file can be avoided, and the synchronization process delay can be reduced. This method takes into account both the computational overhead and the network transmission overhead, and has the advantages of low overhead and low latency.

[0026] (3) This solution provides an efficient data tiered storage method. Edge storage has the advantages of low latency and low cost, but the edge storage space is small and the security risk is high; the central cloud storage has the advantages of large storage capacity and high security, but the central cloud storage has high storage costs and is far away from users, resulting in the disadvantage of high latency. Therefore, the advantages of the edge and central cloud storage can be combined to set a cost model. Based on the cost model, request volume, access latency, access frequency and other factors, the intelligent agent can decide online whether to create a data copy at the edge and the level (hot layer, cold layer) in the cloud center at each moment. At the same time, the synchronization method is used to ensure the consistency of the original data and the copy data. This method can reduce data storage costs, reduce data access and operation latency, and improve storage system performance by storing data in layers.

[0027] (4) This solution provides an efficient scheduling method for cloud-edge tasks. Tasks are scheduled according to the memory requirements, CPU requirements, latency thresholds, cloud-edge network status, and edge status of different cloud-edge tasks. Each cloud-edge task is set with a maximum acceptable response time. The scheduling success is determined based on the actual response time, and the agent is adjusted accordingly. Through this method, the execution of many cloud-edge transmission tasks such as synchronization, deduplication, and data tiered storage can be adjusted according to the current cloud-edge status, so that various cloud-edge transmission tasks can be executed in a reasonable order, improving the overall performance of the system.

[0028] (5) This solution provides an edge-integrated homomorphic encryption technology that can encrypt local data on the edge, allowing the cloud to operate edge data without decryption. It can also take advantage of the large number of edge servers and make full use of the computing power of edge servers to collaboratively and securely process data.

[0029] (6) This solution provides an active encrypted data security storage method, which selects a strong encryption algorithm to encrypt the weakly encrypted data uploaded by the edge to ensure security. Due to the characteristics of the edge, such as the large number of tasks it performs and limited performance, the encryption of data may be weak. The cloud can ensure data security by strongly encrypting some weakly encrypted data. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Illustration of the cloud-edge integrated secure storage system module designed for the present invention.

[0031] Figure 2 The cloud-edge integrated secure storage system architecture and flow chart designed for the present invention.

[0032] Figure 3 A relationship diagram of the cloud-edge modules of the cloud-edge integrated secure storage system designed for the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0034] like Figure 1-3 As shown, the cloud-edge integrated secure storage system of the present invention includes multiple edge terminals and a central cloud storage terminal; the edge terminal performs efficient filtering and processing on large-scale data, and then encrypts the data based on edge-end integrated homomorphic encryption for storage; the edge terminal deduplicates the key data in the stored data at the central cloud storage terminal and then stores it securely through active encryption; the central cloud storage terminal provides the same performance guarantee for different edge terminals through performance isolation perceived by file system behavior; the edge terminal and the central cloud storage terminal cooperate to schedule the operation of cloud-edge tasks related to deduplication, synchronization and data tiered storage; wherein, when any edge terminal initiates a synchronization request to the central cloud storage terminal as a synchronization request direction, the central cloud storage terminal determines all synchronization participants according to the pre-stored synchronization rules, and the synchronization request party determines whether all synchronization participants are in the same local area network. If they are in the same local area network, the cloud-edge secure storage synchronization process is directly performed; if they are not in the same local area network, the synchronization request party and the synchronization participant are controlled to establish a direct and confidential communication tunnel with the assistance of the central storage terminal, and the cloud-edge secure storage synchronization process is performed in the communication tunnel.

[0035] The working process of the cloud-edge integrated secure storage system involves three aspects:

[0036] (1) The performance integration of cloud-edge fusion is achieved by designing various modules at the edge, between cloud and edge, and at the central cloud storage end. The performance integration of cloud-edge fusion refers to providing performance guarantees from a unified perspective for heterogeneous cloud-edge resources. Specifically, at the edge, efficient filtering and processing of large-scale data is achieved through application-aware near-data processing methods; between the cloud and edge, model training of edge data is performed through cloud-edge efficient federated learning methods; data stored in the central cloud is stored in layers through cloud-edge efficient data tiering methods, and data copies are created at the edge to reduce data access latency; efficient task scheduling methods are used to efficiently process cross-cloud-edge transmission tasks in real time; and at the central cloud storage end, performance isolation of resources is further achieved for different tenants through file system behavior-aware performance isolation methods.

[0037] (2) The security integration of cloud-edge fusion is achieved by designing various modules at the edge, between cloud and edge, and at the central cloud storage end. The security integration of cloud and edge fusion refers to providing a unified level of security protection for heterogeneous cloud and edge resources with different security risks. Specifically, at the edge, the edge data is securely processed through edge-integrated homomorphic encryption technology; between the cloud and edge, the data uploaded by edge users is deduplicated through the cloud-edge secure deduplication method, and the consistency and synchronization of data between the edge and cloud are securely and efficiently guaranteed through the cloud-edge secure storage synchronization method; at the central cloud storage end, the encryption algorithm in the resource-limited scenario is actively selected for weakly encrypted data through the active encryption data security storage method, and the deduplicated data is strongly encrypted using the encryption method.

[0038] (3) Optimize cloud-edge storage efficiency and reliability by designing various modules at the edge and central cloud storage. Specifically, various modules are designed at the edge and central cloud storage to optimize cloud-edge storage efficiency and reliability, including optimizing storage efficiency and security at the edge through a storage efficiency and security joint optimization method; at the central cloud storage end, the reliability of data storage is ensured through a storage reliability assurance method based on a super fault domain.

[0039] In this embodiment, the application-aware near data processing method in (1) adds data filtering, encryption and decryption, and hash calculation modules to the main control of the memory, processes data at the source of data storage, solves the problem of processor processing speed and storage I / O speed mismatch, and efficiently filters and processes large-scale data at the edge. The efficient federated learning method in (1) can adopt a federated learning framework based on lottery theory and adaptive differential privacy protection, including three processes: client grouping, iterative pruning based on lottery theory, and fine-tuning training. The cloud-edge efficient task scheduling method in (1) can use a cloud-edge collaborative task real-time scheduling method based on DDQN and a resource allocation method for multi-tenant application delay awareness. The cloud-edge efficient data tiering method in (1) can utilize the complementary advantages of cloud storage and edge storage, comprehensively consider factors such as cost, request volume, access delay, and access frequency, and use an intelligent agent to online decide whether to create a data copy at the edge at each moment, as well as the data tier (hot tier, cold tier) in cloud storage. The file system behavior-aware performance isolation method in (1) can record the performance requirements of the edge in the metadata of the virtual machine image file, reuse the data path to propagate the edge performance requirements, and build a logically centralized I / O scheduler that is aware of the file system access behavior on the data plane to allocate memory and disk resources according to the edge performance requirements.

[0040] In this embodiment, the edge-integrated homomorphic encryption in (2) uses a homomorphic encryption algorithm to encrypt and process data, and establishes a common key through key negotiation; the cloud-edge secure deduplication algorithm in (2) can use a low-overhead key management method based on cloud-edge collaboration, a duplicate data removal method for a multi-control storage system, and a deduplication storage and query method for cluster internal network communication traffic logs; the cloud-edge secure storage synchronization method in (2) can use an adaptive synchronization mode selection method to select incremental synchronization and full synchronization according to different network environments and file sizes, and use the CDC technology based on content-defined blocks combined with guessing jumps to optimize incremental synchronization, and use encryption technology such as AES to ensure the data security of the synchronization process. This method can be combined with kubeedge for cloud-edge scenarios. Kubuedge sinks the container orchestration capabilities of Kubernetes to the edge, facilitating the deployment of edge containers. By deploying the synchronization request processing module, edge resource monitoring module and communication coordination module in the cloud controller, it is possible to process the synchronization requests of the edge, assist the edge in establishing communication tunnels, and monitor the resource usage information of the edge. At the same time, a Pod is deployed on the edge to carry out the subsequent synchronization process. The Pod contains various containers that implement the change monitoring module, network evaluation module, file and resource evaluation module, synchronization module (including full synchronization and incremental synchronization), network communication module and encryption module.

[0041] In this embodiment, the storage efficiency and security joint optimization method in (3) can ensure the efficiency and security of edge data operations by implementing a distributed key-value database, a multimedia data query algorithm based on a searchable encryption strategy, and a security threat perception and defense method for edge devices (including adversarial samples and tampered content detection for video image classification, backdoor attack and defense methods for images, and memory leak detection methods based on memory utilization curves, etc.). The super-fault domain storage guarantee method in (3) can be implemented using a super-fault domain multi-channel video data processing method. The traditional erasure strip storage method adopts a random write mode. In the video storage system, when facing the problem of super-fault domain data loss, the traditional random write mode is likely to cause the image group (Group of Pictures, referred to as GOP) package in the hard disk to be too correlated, resulting in difficulty in video data recovery after hard disk failure. The super-fault domain multi-channel video data processing method collects and pre-processes video data through different channels, splits it into GOPs and performs erasure coding; according to the generation time of each GOP packet, calculates the difference between GOP packets in different channels; according to the maximum difference distribution principle, the GOP packets with the largest difference in each channel are stored on the same hard disk. This method can reduce the risk of data loss under super fault domain and improve recovery accuracy.

[0042] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A cloud-edge integrated secure storage system, characterized in that: include: Multiple edge terminals and central cloud storage terminals; The edge performs efficient filtering and processing on large-scale data, and then securely processes and encrypts the data based on edge-integrated homomorphic encryption, and then stores it; The edge end stores the key data in the stored data securely through active encryption after deduplication at the central cloud storage end; The central cloud storage end provides the same performance guarantee for different edge ends through performance isolation of file system behavior awareness; The edge end and the central cloud storage end cooperate with each other to schedule the running of cloud-edge tasks related to deduplication, synchronization and data tiered storage; wherein, when any of the edge ends initiates a synchronization request to the central cloud storage end as a synchronization request direction, the central cloud storage end determines all synchronization participants according to the pre-stored synchronization rules, and the synchronization requesting party determines whether all synchronization participants are in the same local area network. If they are in the same local area network, the cloud-edge secure storage synchronization process is directly carried out; If the synchronization requester and the synchronization participant are not in the same local area network, a direct and confidential communication tunnel is established with the assistance of the central storage end, and a cloud-edge secure storage synchronization process is performed in the communication tunnel.

2. The cloud-edge fusion secure storage system according to claim 1, characterized in that: When the size of the file to be synchronized between the synchronization requester and the synchronization participant is lower than the transmission volume threshold, full synchronization is performed; wherein the synchronization process satisfies causal consistency; and the transmission volume threshold is calculated based on the current network bandwidth.

3. The cloud-edge fusion secure storage system according to claim 2, characterized in that: When the size between the synchronization requester and the synchronization participant is greater than the transmission volume threshold, incremental synchronization is performed; wherein the synchronization process satisfies causal consistency, and the incremental synchronization uses an improved content-defined block technology with guess jumps to accelerate the synchronization process.

4. The cloud-edge fusion secure storage system according to claim 1, characterized in that: The central cloud storage end uses an intelligent agent to decide online whether to create a data copy of the original data at the edge end and store it in layers at each moment according to the current environmental factors of the original data; The current environmental factors include at least one of a cost model, a request volume, an access delay, and an access frequency.

5. The cloud-edge fusion secure storage system according to claim 4, characterized in that: The intelligent agent maintained by the central cloud storage end is used to schedule tasks according to the memory requirements, CPU requirements, delay thresholds, cloud-edge network status and edge status of different cloud-edge tasks. Cloud-edge tasks include synchronization, deduplication and data stratification.

6. The cloud-edge fusion secure storage system according to claim 5, characterized in that: A maximum acceptable response time is set for each cloud-edge task, and whether the scheduling is successful is determined based on the actual response time. Different reward functions are set for successful and failed scheduling to obtain the difference in the impact of different scheduling decisions on the results.

7. The cloud-edge fusion secure storage system according to claim 4, characterized in that: If the data copy created by the edge end is modified, the edge end initiates a synchronization request to the central cloud storage end to synchronize the original data in the central cloud storage end with the data copy of the edge end.

8. The cloud-edge fusion secure storage system according to claim 7, characterized in that: If the original data in the central cloud storage terminal and corresponding to the data copy at the edge terminal is modified, the central cloud storage terminal initiates a synchronization request to synchronize with the edge terminal storing the data copy to ensure data consistency.

9. The cloud-edge fusion secure storage system according to claim 1, characterized in that: The edge end encrypts the stored data through edge-end integrated homomorphic encryption technology to obtain secure data, and transmits some key data to the central cloud storage end. The central cloud storage end can operate the secure data without decryption.

10. The cloud-edge fusion secure storage system according to claim 9, characterized in that: The central cloud storage terminal selects an encryption algorithm according to the current system status and resource usage information to perform strong encryption processing on the weakly encrypted data stored in the terminal.

Images