Distributed secure edge heterogeneous storage network with Byzantine attack resilience

Abstract

The invention discloses a distributed secure edge heterogeneous storage network with Byzantine attack resilience. Files are stored in an edge storage device which is elastic to eavesdroppers and Byzantine opponents in a distributed mode. The method includes a cost-effective method in which an authorized user is able to access the content of all edge storage nodes. In this method, key blocks and file blocks masked by key blocks are stored in the edge storage node. In addition, redundant data for error correction is stored. In turn, errors introduced by the Byzantine opponent may be corrected when retrieving all chunks. In a loss resilient approach, redundant data is stored together with masked file partitions. When retrieving blocks from the edge storage nodes, a unique approach to address unknown file partition values applies to identify corrupted nodes based on average residual values for each storage node.

Description

[0001] This invention patent application is a divisional application of the invention patent application with application number 202110974263.2 filed on August 24, 2021, and named “Distributed Secure Edge Heterogeneous Storage Network with Byzantine Attack Resilience”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application is related to U.S. patent application No. 16 / 669,696 [Docket No. STL 074555], entitled “DISTRIBUTED SECURE EDGE STORAGE NETWORK UTILIZING COST FUNCTION TO ALLOCATE HETEROGENEOUS STORAGE,” filed on October 31, 2019, and U.S. patent application No. 16 / 669,712 [Docket No. STL 074556], entitled “DISTRIBUTED SECURE EDGE STORAGE NETWORK UTILIZING REDUNDANT HETEROGENEOUS STORAGE,” filed on October 31, 2019, all contents disclosed and taught in these patent applications are specifically incorporated herein by reference. Background Art

[0004] The present disclosure as a whole relates to distributed data storage systems. Data generation is growing exponentially due to, among other reasons, the widespread adoption of mobile devices and the "Internet of Things" (IoT). One source estimates that by 2025, data creation will grow to 163 zettabytes, equivalent to ten times the amount of data created in 2017. The stored data may include large amounts of automatically generated data, such as data generated by sensors. Sensor data may include raw data captured by terminal devices as well as data generated as a result of analyzing the raw data. One solution for storing and analyzing large amounts of data involves transferring it to large data centers, often referred to as cloud storage.

[0005] Assuming that network traffic is growing exponentially, it will become increasingly difficult to send all the created data to the cloud for storage, especially for time-critical applications. In some emerging technologies, such as smart cities and self-driving cars, data may need to be analyzed and stored in real time, which may be difficult to achieve in the cloud. In addition, cloud computing may suffer from relatively high latency (e.g., cloud storage facilities can be far away from the location where the data is generated) and due to the unpredictability of network connections (e.g., due to demand surges, outages, etc.). Summary of the invention

[0006] The present disclosure presents a secure distributed heterogeneous edge storage method that provides resilience against eavesdropping attacks and Byzantine attacks for storing files in distributed edge storage nodes. In one aspect, the file is partitioned, a key is generated, and packets including file information masked by the key are stored in the distributed nodes. In addition, additional data is generated that can be used for error correction in the event that one of the nodes is damaged by a Byzantine adversary. The method includes selecting a value Z e , enabling access to Z e An eavesdropping attacker at a storage node cannot decode any part of the information of a file of size |F| stored in the network, which stores the file in a distributed manner on Z e The method also includes selecting a value Z b , so that the information from Z can be corrected by applying the error correction code to the information stored in the storage node b The method includes selecting N* storage nodes that minimize a cost function among the storage nodes, the cost function including |F|, Z e , Z b , initial data access cost C T , and the transmission and download costs C d . Error correction codes can be based on Z b and |F| for correcting errors from the retrieved information about the file F. Initially, equal memory sizes are allocated from N* storage nodes in the storage nodes to store the file, redundant error correction data, and a set of linear code keys. Subsequently, the method includes iteratively determining a first cost of adding more storage nodes to the N* storage nodes and a second cost of allocating more memory from a subset of the N* storage nodes. The method includes constructing redundant data groups based on applying an error correction code to the set of linear code keys and a file partition masked with the set of linear code keys, and based on a minimum cost determined according to the iterative determination, N≥N among the storage nodes. * The storage nodes store files, linear code keys and redundant data packets. The storage nodes are sorted from large to small according to storage capacity, so that the largest Z e The storage node stores the linear code key, the minimum NZ e -2Z b The storage nodes store the file partitions, and the redundant data packets are stored in the remaining 2Z b Storage node.

[0007] In another method, the file information is stored with sufficient redundancy such that when retrieving packets from the distributed edge storage nodes, an overconfigured system for solving the equations of the file information is created. Subsequently, the unique solution, herein referred to as the packet Kaczmarz method, is used to identify corrupted blocks from one or more of the edge storage nodes. Subsequently, blocks from the edge storage nodes can be omitted, and an iterative solution of the system of equations can be constructed to replicate the file based on the information retrieved from the distributed edge repository. Specifically, the method includes selecting N storage nodes coupled via a network to store a file of size |F| and redundant data of size |F red |. While the method can be used with homogeneous storage nodes, in one example, at least two of the N storage nodes are assigned different sizes of memory to store the file, and the N storage nodes are sorted from the maximum storage capacity |s1| at the first storage node to the minimum capacity |s N | at the Nth storage node. The number of storage nodes N can be calculated based on the average error for the number of t random nodes selected from among the N nodes of the maximum number of Z b corrupted nodes. This approach also includes selecting a value of Z e <N such that an attacker with access to Z e storage nodes cannot decode the file. The method includes partitioning the file into file partitions, thereby generating keys stored in the first Z e of the N storage nodes, and creating storage blocks that include independent linear combinations of the keys and the file partitions. Subsequently, the method includes storing storage packets that include independent linear combinations of the keys and the file partitions in the Z e +1 to the Nth storage nodes.

[0008] This summary is provided to introduce in simplified form some concepts that are further described below in the Detailed Description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0009] Other embodiments are also described and recited herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a block diagram of an edge storage system according to one exemplary embodiment.

[0011] Figure 2 is a block diagram showing a wiretapping adversary and a Byzantine adversary according to one exemplary embodiment.

[0012] Figure 3 is a block diagram showing the availability of node storage capacity according to one exemplary embodiment.

[0013] Figure 4 is a block diagram illustrating a secure distributed edge storage scheme that provides resilience against one or more eavesdropping adversaries and one or more Byzantine adversaries.

[0014] Figure 5 is a flow chart illustrating an example of node selection.

[0015] Figure 6 is a block diagram of an apparatus according to an exemplary embodiment.

[0016] Figure 7 is an exemplary algorithm of the grouped Kaczmarz method for identifying corrupted data from edge storage nodes.

[0017] Figure 8 is an exemplary flow chart for executing the packet Kaczmarz method to recover file information from packets retrieved from an edge storage node.

[0018] Fig. 9 is an example of a graph of mean absolute error regarding the number of random edge storage nodes that an authorized user can access in an edge storage system.

[0019] Fig.10 is an exemplary algorithm for retrieving data from a distributed edge storage system that identifies corrupted nodes to provide resiliency against Byzantine adversaries.

[0020] Fig.11 is an exemplary flow chart for retrieving file partition values ​​from an edge storage node that is resilient to Byzantine adversaries.

[0021] Fig.12 is an exemplary flow chart for storing data in a distributed edge storage system to provide resilience to one or more Byzantine adversaries.

[0022] Fig.13 An exemplary processing device for performing aspects of the present disclosure is depicted. DETAILED DESCRIPTION

[0023] In the applications incorporated by reference above, methods for storing data in heterogeneous edge nodes are provided. These methods allow a file F to be distributed and stored in multiple storage edge nodes, so that the system can access a given number of storage nodes (Z e ) is resilient to eavesdropping adversaries (i.e., the eavesdropping adversary cannot obtain any information about file F). The present disclosure proposes modified methods to those discussed in these previous applications to provide protection against Byzantine attackers (Z b). Byzantine attacks are based on the Byzantine error problem that one or more nodes providing data may be corrupted. In turn, without countermeasures, the corrupted data may lead to system failure. In the context of a distributed edge storage system, this may mean that a file F distributedly stored in multiple edge nodes may not be copied by authorized users in view of data from edge nodes that are corrupted by a Byzantine adversary.

[0024] In general, the previous methods for secure distributed storage between heterogeneous nodes to resist eavesdropping adversaries are modified so that in addition to the key grouping and masked file grouping of the previous methods, additional redundant data is also stored in the distributed edge nodes. Although the additional redundant data provides additional computational complexity to the system, the benefit is that the system can be designed and deployed to be immune to Byzantine attackers (Z b ) provides robust, secure edge node storage for files affected by the system, and provides protection against eavesdroppers (Z e )’s elasticity.

[0025] One method of storing a file F in a distributed edge storage repository involves a scenario where an authorized user has access to all edge storage nodes in a given system. The present disclosure is generally different from the prior art methods of the incorporated applications in that the present disclosure modifies the storage selection process to optimize the selected storage in view of the need to store additional data to correct errors in packets retrieved from edge storage nodes. In addition, the manner in which files are partitioned for distributed storage is modified to allow for partitioning and storage of additional redundant data. Thus, the following discussion generally describes a method for securely distributing storage of a file F among multiple edge storage nodes, including modified storage selection and file partitioning to provide for Z b Resilience to damaged nodes. Additionally, the present disclosure includes a discussion on the construction and grouping generation of linear encoding keys.

[0026] exist Figure 1 , a block diagram illustrates an example edge storage system 100. A plurality of storage nodes 102 store data from sensors 104 (e.g., cameras) or some other data generators. In summary, at least the storage nodes 102 form an edge layer 106 that is physically close to the sensors 104 and may also send some or all of the data to cloud storage 108. Note that the sensors 104 may also be considered part of the edge layer 106, and in some cases, some or all of the sensors 104 may be integrated with the storage nodes 102, for example, within the same device.

[0027] Distributed edge storage can provide data availability closer to where the data is needed and reduce latency. For example, a user terminal 110 adjacent to the edge layer 106 can actively monitor the sensors 104 and / or storage nodes 102, looking for patterns and / or events that occur in near real time. However, storing data on edge devices can compromise data security. For example, edge devices may have limited capabilities (e.g., computing, memory), and therefore may not be able to implement multiple layers of security without overly limiting performance. Edge devices may also not be controlled by a single entity, which can make it difficult to enforce security policies. The present disclosure describes a security solution that addresses security challenges that are unique to edge devices but applicable to other devices, such as central distributed storage systems.

[0028] For a distributed edge storage setup, a suitable attack model is a situation where many edge devices are damaged. More specifically, a potential attack mode is an eavesdropping attack in which an attacker (eavesdropper) controls a group of edge devices and monitors the data stored therein. In another attack mode, an attacker (Byzantine attacker) may damage one or more edge devices so that data retrieved from such damaged devices may be damaged. This second attack mode is sometimes referred to as a Byzantine attack based on a Byzantine failure of a distributed computing system in which damaged data from an attacker causes system failure. Therefore, the present disclosure presents a system in which the goal is to keep data confidential from the attacked device and to address damaged data from devices used as distributed storage nodes.

[0029] According to an exemplary embodiment, the scenario where the eavesdropping attacker 206 and the Byzantine attacker 210 are actively attacking the edge device 202 is also Figure 2 Data 200 (e.g., file F) is stored in a distributed manner in a set of edge storage nodes 202 that can be accessed via a network. An end user may wish to access a computer device connected via a network, such as Figure 1 The user terminal 110 shown independently in the figure stores the file F. For the purposes of this disclosure, a storage node is an independently operable, self-contained computing unit that includes at least some available storage. This independence means that an eavesdropping attacker 206 or a Byzantine attacker 210 that has access to one storage node does not necessarily have access to other nodes that interoperate to store portions of the data 200. This is different from other distributed storage schemes, such as disk arrays, where portions of data files can be distributed between different disks. In general, an attacker who can access one disk in an array (e.g., by accessing a host computer or a storage controller card) can typically access all disks. It should also be noted that the edge storage nodes 202 do not necessarily need to communicate with each other to implement the distributed storage described herein, but all storage can typically be accessed by a single entity, such as an authorized user terminal.

[0030] In this example, the first subset 204 of edge storage nodes 202 is accessible to an eavesdropping attacker 206, such that the eavesdropping attacker 206 can view the data of interest stored on the first subset 204. For purposes of this disclosure, the value Z e represents the maximum number of nodes with system protection that the eavesdropping attacker 206 can access. In addition, the second subset 208 of the edge storage nodes 202 is accessible to the Byzantine attacker 210, so that the Byzantine attacker 210 can damage the data of interest stored on the second subset 208. For the purpose of this disclosure, the value Z b represents the maximum number of nodes that a Byzantine attacker 210 can corrupt while maintaining system operability. The system is designed so that an eavesdropping attacker 206 cannot read data that can only access Z e In addition, the system is designed so that a Byzantine attacker 210 cannot prevent an authorized user from retrieving any part of the data file 200 of a node. b The authorized user will be able to access the Z file of node 202. e The above nodes, and therefore can read data file 200. In addition, the authorized user will be able to correct the Z in the slave node 202. b In this example, the authorized user will need to access all edge nodes 202 to read data file 200. In addition, it is understood that the authorized user may not be able to distinguish between nodes that have been compromised by eavesdropping attacker 206 or Byzantine attacker 210. In addition, it may be the case that Z e nodes and Z b The nodes may share at least one node that is compromised by both the eavesdropping attacker 206 and the Byzantine attacker 210.

[0031] The eavesdropping attack is addressed by using a secret sharing scheme using a linear encoding key, where the data is divided into shares of equal size and each share is masked with a linear encoding key and stored in one of the available storage nodes. For example, suppose there is of Available storage nodes. Data F is first divided into two equal shares f1 and f2, and keys k1 and k2 are generated.

[0032] Then, create Four groups and store them in Note that the values ​​of the groups are arithmetically combined as indicated and truncated to fit into the groups. The system is set up so that the two shares f1 and f2 are vulnerable to any Z that may be attacked by eavesdropping. e =2 Storage nodes are kept confidential.

[0033] To provide resilience against Byzantine attackers, additional information about the file F is redundantly stored. The additional information can be used for error correction methods (e.g., (n, k) error correction codes). Subsequently, when data is retrieved from the storage node, the error correction code can be applied to the retrieved data so that any corrupted data received from a storage node that is corrupted by a Byzantine attacker can be corrected. Specifically, for a given (n, k) error correction code that is capable of correcting e errors, nk number of additional redundant data can be stored in the distributed storage nodes for every k pieces of information.

[0034] The edge device storage nodes 202 can be heterogeneous with different memory, computing, bandwidth, power, etc. Directly applying existing secret sharing schemes leads to poor performance of distributed edge storage because they do not take into account the heterogeneity of storage nodes. For example, if four storage nodes The allocated storage capacity availability is different, then and The storage should be grouped into different sizes. For the purposes of this disclosure, the term "storage capacity availability" is used to describe the ability of a storage node to satisfy requests that meet some minimum requirements. A storage node should not only have the available capacity to store data, but should also have the performance to write to (including network communications) and read from storage that meets the minimum standards defined for the system. Figure 3 , it is shown how the storage capacity availability of different storage nodes varies according to an exemplary embodiment.

[0035] exist Figure 3 , rectangles 300 and 302 represent data transfers over time by two different heterogeneous storage nodes s1 and s2. Node s1 has a higher transfer rate r1 than node s2, which transfers data at rate r2. A partition of size |f| is available if the data storage node can transmit and receive the partition within time t. Block 300 and block 302 both have an area size of |f|, so both nodes have storage capacity availability for at least one partition. As shown in box 304, assuming that node s1 also has sufficient storage capacity when capacity is available, node s1 can theoretically have storage capacity availability for four partitions of size |f|. In this example, even though node s2 has a much higher storage capacity than node s2, it still has one-quarter of the storage capacity availability.

[0036] In order for an entity to utilize heterogeneous edge storage nodes for secure storage, the entity may also define (i) how to select a storage node from all candidates, (ii) how to partition files, (iii) how to generate keys, and (iv) how to create groups to be stored in the selected storage nodes, etc. These issues are addressed in the present disclosure, as well as how to optimize storage allocation for cost.

[0037] exist Figure 4 , a block diagram shows a distributed edge storage solution according to an exemplary embodiment. File 400 is processed by a storage processing device 402, which may include conventional computing hardware, such as a processor, memory, input / output bus, etc., and special hardware, such as an application-specific integrated circuit, a digital signal processor, etc. The storage processing unit includes functional units (e.g., software modules, logic circuits), such as a file partitioner 404, which can be configured to query an edge network 406 of edge storage nodes to determine the storage capacity availability of a single storage node. Using the techniques discussed below, the file partitioner 404 selects a subset 408 of nodes and creates a definition 410 of a storage configuration for the file 400. The definition 410 includes identification of the selected storage nodes, the partition / group size on each node, the data and keys arranged in each partition, etc.

[0038] The key generation part 412 includes a module 414 for generating a key set for the file 400 and a module 416 for linearly combining the keys of the set into a linear encoding key. i The packet generation module 418 uses the definition 410 and the linear encoding key to generate the file 400 and store it on the network 406. A similar set of modules can be used to read data from the network 406 based on the definition 410, download the partitions / packets, unlock the partitions / packets, and reassemble the file.

[0039] Each group is created using the file partition and the generated key. The file partitioner 404 effectively uses the available heterogeneous resources so that the designed distributed edge storage system is targeted at up to Z e The eavesdropper adversary 420 attacks at the storage node and at most Z b The Byzantine adversary 422 that attacks the storage node is safe. Consider a system model where there are The set of all candidate storage nodes in network 406 is composed of Indicates. Select All A subset 408 of available storage nodes is used to store data F securely.

[0040] The set of selected storage nodes is given by Indicate that, among which Then, create groups of sets to be stored in the set 408 of selected storage nodes, among which Stored in the storage at

[0041] Assume that the system is vulnerable, among which the ability of the eavesdropping attacker 420 is characterized according to the parameter Z e <N. More specifically, the eavesdropping attacker 420 can access the data stored in at most Z e storage nodes. However, the defender does not know the information about which storage nodes are attacked. Therefore, the goal is to keep the data F confidential for any Z e storage nodes. The value of Z e can also be regarded as a security parameter, among which a higher value of Z e can provide higher security for the system, and a smaller value of Z e makes the system more vulnerable to attacks

[0042] It is also assumed that the system is vulnerable, among which the ability of the Byzantine attacker 422 is characterized according to the parameter Z b <N. More specifically, the Byzantine attacker 422 can corrupt the data stored in at most Z b storage nodes. Similar to the eavesdropping attacker 420, the defender also does not know the information about which storage nodes are attacked by the Byzantine attacker 422. Therefore, the goal is to be able to correct the error of the data F when the damaged data is provided to the authorized user 424 at any Z b storage nodes. The value of Z b can also be regarded as a security parameter, among which a higher value of Z b can provide higher security for the system, and a smaller value of Z b makes the system more vulnerable to attacks

[0043] From the perspective of the defender, the storage usage of a more robust system with larger values of Z e and Z b increases (the number of distributed storage nodes increases) and the complexity of designing a secure system also increases. In other words, by considering the vulnerability state of the system to attacks, the parameters Z e and Z b can be regarded as a compromise between providing higher security and increasing system complexity

[0044] One goal is to store data so that the attacker cannot obtain any meaningful information about the data. More specifically, the proposed solution provides information-theoretic confidentiality defined as where H() is the information-theoretic entropy, and is stored in any size Storage set In one application of information theoretic confidentiality, linear combinations of data partitions can reveal some meaningful information about the entire dataset. In the proposed method, the number of pairs of any linear combination of data partitions is Z e Any subset of the storage nodes in the storage set is kept confidential. Authorized users can access the storage nodes stored in the storage set to extract data F from all the groups in .

[0045] In this disclosure, features are described that help reduce computational complexity at the design stage and reduce computational complexity and communication costs for authorized users at the retrieval data stage. Figure 4 , select a subset of all candidate storage nodes to distribute the file F. The selected storage set should meet the minimum requirements for providing security. In the second step, the file F is divided into partitions. In the third step, a key is generated and g i 's is constructed as a linear combination of the generated keys, where g i The value is used to mask the file partition. In the fourth step, the file partition and the constructed g i 's is used to generate a packet and store it in a set of selected storage nodes. The details of each step and complementary examples are described below.

[0046] Storage options

[0047] All keys and file partitions need to be retrieved to recover the file according to this method. Therefore, a set of storage is selected so that the total storage size of all storage nodes is equal to |F| (the size of file F) and |K| (the size of data pair Z). e The sum of the key sizes required to store confidentiality). That is, the size of the key can be described as Or the number of nodes that an eavesdropping attacker can access multiplied by the packet size of the first node that exceeds the number of nodes that an eavesdropping attacker can access. Therefore, for the purpose of error correction codes, the size of the message length k = |K| + |F| is the data required to retrieve and restore the file.

[0048] The present disclosure also considers that data from Z b The probability of a node being damaged, where Z bAgain, this becomes a design parameter of the system. The method described herein makes the system robust against Byzantine attacks by using an error correction code for the retrieved file groups. For example, an error correction code can be used, which is an (n,k) error correction code, as described above k = |K| + |F|. That is, for a given message length k, a block length n can be provided for correcting errors in the received block. The message length and block length include additional information in the form of bits for error correction. The message length can be a function of the size of the key K and the file F. Therefore, the total size of the data stored in the storage node can be increased by nk, which represents the additional information required for error correction.

[0049] (n,k) error-correcting codes may be able to correct Error, where is the largest grouping stored in all stores. The reason is that the maximum number of damaged stores is Z b , and thus the maximum size of the corrupted information is a multiple of the maximum packet size in all storages multiplied by the number of nodes corrupted by the Byzantine attacker. For example, a Reed-Solomon code may be used as an (n, k) error correction code. In this example, nk is equal to 2e, which can be used in the cost function described below to determine the amount of storage required to store a file of size |F|. Other error correction codes may have different error correction capabilities, and it will be understood based on the present disclosure to modify the cost function appropriately.

[0050] To provide security, file partitions are masked with keys (including redundant data for error correction), so some available memory resources should be allocated to store the keys. The minimum requirement for information-theoretic privacy of storing a file partition f in a storage node is to mask it with a random key of the same size as f, e.g., f+k, where |k|=|f|. In addition, to make the data private to any Z e Storage nodes are kept confidential and stored in any Z e The groups in the storage nodes should be linearly independent. To satisfy this constraint, one requirement is that for any size Storage Group (Stored in storage i In the middle), there should be at least Z e other storage partitions (each of which is stored in another storage) are greater than or equal in size In other words, if the storage nodes are sorted in descending order by the allocated storage size, such that A minimum requirement is to store the e +1 The groups in the storage nodes have the same size. In addition, any Z e Storage nodes should all contain linearly independent groups.

[0051] Therefore, the first requirement for storage selection is where |F| is the size of file F. Another requirement is that the total storage size allocated on all storage nodes should be equal to This means that in order to distribute the data F, we need Additional storage is needed to store the key and keep the data safe. However, in order for an authorized user to retrieve data F, it should be possible to access the data stored in all storage nodes and download them, then the key part should be subtracted and the useful information F should be extracted.

[0052] Furthermore, as mentioned above, the use of error correction codes requires the storage of additional information for performing error correction. e The next 2Z after storage b Maximum storage is used to store additional information for error correction, or Finally, the rest of NZ e -2Z b Minimal storage is used to store file partitions masked with a key. The largest portion of file F masked with the key corresponds to the portion stored in storage Please note that the files stored in the largest Z e +2Z b +1 All packets in a repository may have the same size.

[0053] Authorized users from The cost of retrieving data from a storage node is a function of the number of storage nodes and other parameters such as storage type, bandwidth, power, etc., which are different for different storage nodes. The set of selected storage nodes should be chosen to minimize the cost. Even if the storage nodes are homogeneous in terms of storage type, bandwidth, and power, but heterogeneous only in terms of available storage size, minimizing the cost is still important. In the following, a set of selected optimized storage nodes is found by focusing on optimizing the cost of this simplified scenario using the following equation (1), which takes into account the Z cost of protecting data from eavesdropping attacks. e The cost of affecting the node, and providing enough information to correct the Z damaged by the Byzantine attacker b Error in node:

[0054]

[0055] As shown in equation (1), the cost consists of two parts: (i) NC T , where C T is the initial cost of accessing the data stored in each storage node, and NC T is the total cost of all storage nodes, and (ii) Among them C dis the cost of transmitting and downloading a piece of information, and the total cost of all data is Generally speaking, C T This includes network discovery, setting up and releasing connections, etc. Note that in the simplified model, for each candidate storage node, it can be considered that C T and C d is constant. In some variations, for each node, C T and C d can be different so that the cost value in equation (1) is the average value. More specifically, for C d , is the weighted average And for C T , is the average value in and C T,i are the download cost value of the i-th storage node and the initial cost of accessing the data stored at storage node i, respectively.

[0056] The goal is to select a set of storage nodes To minimize the cost defined in equation (1). To solve this problem, determine The impact on the cost, and then find the lower bound of the cost and the corresponding optimal value of N. = equal to masking all packets stored at all storage nodes with the key and storing them in storage The largest part of file F at . To reduce the cost, On the other hand, as will be explained below in the "File Partitioning" section, the file partitions are distributed in the last NZ e -2Z b storage nodes, so the minimum size of the largest part is equal to |F| / (NZ e -2Z b ), that is, the file partitions are evenly distributed in the last NZ e -2Z b Again, this differs from the eavesdropping-only security approach because additional information for error correction will be stored in the 2Z b Storage node.

[0057] The optimal cost solution may not always be possible because the storage nodes containing different parts of the file partition are heterogeneous and have different |s i |, and thus have different However, this can provide a lower bound on the cost, as shown in equation (2) below.

[0058]

[0059] As shown in equation (2), by increasing N, the first part of the lower limit increases and the second part decreases. Therefore, there is an optimal value of N that minimizes the lower limit, which is calculated as shown in equation (3) below, where N * Rounded to an optimal integer. Specifically, N1 is N * Round to the nearest smaller integer, and N2 is N * Round to the nearest higher integer.

[0060]

[0061] When the edge devices are heterogeneous, the optimized number of storage nodes N is calculated * The cost of is greater than the lower bound calculated in Equation (2). However, this clue can be used to select a set of The strategy is to initially use * The same memory size of the storage nodes (the storage nodes with the largest available storage size) is used, and if more memory is needed to store F, it is iteratively decided to (i) add more storage nodes or (ii) use more memory from each available storage. The decision between these two options is based on minimizing the cost. The details are provided below and in Figure 5 , which is a flowchart illustrating node selection according to an exemplary embodiment. Note that |s i | indicates storage i The maximum available memory size at Indicates that from storage s i Used to store part of file F, i.e. group The memory size.

[0062] Initialization 500 of storage selection involves selecting the first N nodes after arranging them in descending order of available storage size. * Storage nodes as The maximum size of file F that can be evenly distributed and stored in these storage nodes is (NZ e -2Z b )|s N |. Therefore, if it is determined that the size of 501 file F is larger than this, it enters the "modify" phase, in which more 510 storage nodes are added or more storage nodes are used from available storage nodes. If the size of file F is smaller than the first N * The available size in the storage nodes, the set of storage nodes initially selected is finally determined 502 as And which is stored in each storage s i The size of each group in is equal to

[0063] The modification phase involves determining one of these options 510, 512, where the option with the minimum cost is selected. For the modification phase, additional info=(NZ e -2Z b -n)(|s N-n |-|s N-n+1 |) Information: (i) Increase the size of each packet stored in the first Nn storage nodes by 503 |s N-n |-|s N-n+1 |, updated to Or (ii) add 504 followed by N add Storage nodes, so Note that n is initialized to 1 at step 500, and N add should be as small as possible to reduce the increase in cost. Each option 503, 504 increases a different cost. More specifically, the first option increases the cost by add,1 =(Z e +2Z b )(|s N-n |-|s N-n+1 |)C d , because in equation (1) The size of |s N-n |-|s N-n+1 |, and the second option 504 increases the cost by cost add,2 =N add C T , because the number of storage nodes in equation (1) increases by N add Based on the system parameters, the lower cost option is selected (see operation 508).

[0064] Note that if info = (NZ e -2Z b -n)(|s N-n |-|s N-n+1 |) is equal to 0, the first option 503 is not available, and therefore the second option 504 is selected by adding as many additional storage nodes as needed, so that and After this selection, the variables are updated, such as Figure 5 The operations are detailed in .

[0065] After a modification phase involving modification of the memory size used from each storage or set of selected storage nodes, the parameters should be updated. If the first option 503 was selected in the modification phase, the update 505 involves changing the parameters and parameter n is updated to n=n+1. If the second option 504 is selected in the modification phase, the N add The storage node is added to the selected storage node set N=N+N add To update the selected storage node set 506. In addition, Should be set to Please note that depending on the amount of information, you may choose So that storage is not used N All available sizes, that is, In addition, the parameter n should be updated to n=n+N add .

[0066] After making the required updates, if the condition 507 is still not met for the lowest cost option The phases "Modify" and "Update" may then be repeated sequentially until the condition is met. When the condition 507 is met, the storage configuration is finalized 514 similarly to finalization 502, but using the nodes and grouping sizes determined in the "Modify" and "Update" phases.

[0067] File Partitioning

[0068] Front Z e Storage nodes are assigned to store only keys, the next 2Z b Storage nodes are allocated for storage redundant information for error correction of the key and the file partition masked with the key, and the remaining NZ e -2Z b Storage nodes store file partitions masked with a key.

[0069] Therefore, the maximum Z e Storage and the smallest NZ e -2Z b The packets in storage are allocated as described in the file partitioning section of US Patent Application No. 16 / 669,696, incorporated by reference above.

[0070] Store in the remaining 2Z b The packets stored are the redundant information needed by the error correction code to detect and correct damaged packets. These packets are constructed using the generator matrix of the error correction code multiplied by a vector consisting of the key and the file partition masked with the key, i.e., all the information is stored together in the first Z e And the last NZ e -2Z b Storing.

[0071] In addition, key generation and construction g i 's and group generation are also based on the disclosure in US patent application No. 16 / 669,696. That is, the key information is stored in the largest Z e The storage node and the file partitions masked with the key are stored in the smallest NZ e -2Z b In the storage node, the remaining 2Z b The storage node stores a packet constructed by multiplying the error correction code generation matrix by a vector including the key and the file partition masked with the key.

[0072] In this regard, to retrieve file F, an authorized user retrieves all stored packets from the distributed repository. The file partitions may be unmasked using a key, and any errors in the data may be corrected by applying an error correction code.

[0073] exist Figure 6 , a block diagram shows a device 600 according to an exemplary embodiment. The device 600 can be configured as a user terminal as described above, but storage nodes can also have similar hardware. The device includes a processor 602 (e.g., a CPU), a memory 604, and an input / output circuit 606. The memory 604 may include both volatile memory (e.g., RAM) and non-volatile instruction memory (e.g., flash memory, magnetic storage device), and may store instructions 608, in which case this facilitates distributed storage via a network 612. The device 600 has a network interface 610 for accessing the network 612. The network 612 includes a plurality of storage nodes 614, such as edge storage nodes as described above.

[0074] Instructions 608 are operable to cause processor 602 to select a value Z e (eg, by a system designer via device selection) to enable access to Z of storage node 614 e The attacker cannot decode the file 616 of size |F| stored in the network 612, which stores the files in a distributed manner on Z e Instructions 608 are also operable to cause processor 602 to select value Z b (e.g., by a system designer via device selection) to enable authorized users to correct Z from storage node 614 b Instruction 608 causes processor 602 to select N storage nodes that minimize the cost function * , the cost function includes |F|, Z e , Z b , initial data access cost C T , and the transmission and download costs C dInstruction 608 causes the processor 602 to initially allocate equal memory sizes from the largest (with the largest available capacity) N * storage nodes to store the set 618 of files and linear code keys.

[0075] Instruction 608 causes the processor 602 to iteratively determine a first cost of adding more storage nodes to the N * storage nodes and a second cost of allocating more memory from each of the N * storage nodes. Based on the minimum cost determined according to the iterative determination, instruction 608 causes the processor 602 to store the file 616 and the linear code key 618 in N≥N * of the storage nodes, the key 618 is stored in the first to Z e th storage nodes, and the partition of the file 616 is masked with independent linear combinations of the key 618 stored in the Z e +2Z b +1 th to N th storage nodes. In addition, redundant data generated by multiplying the generating matrix of the error correction code by a vector including the key and the file partition masked with the key is stored in the remaining 2Z b storage.

[0076] Another method for securely storing data in edge storage nodes provides data redundancy such that data can be retrieved even when an authorized user can access less than all of the available storage nodes in the system. For example, such a method is described in U.S. Patent Application No. 16 / 669,712, which is incorporated herein by reference. This method reflects the fact that the storage nodes in the system can change over time (e.g., due to the loss and / or addition of storage nodes in the system). That is, the system can be designed such that data F can be retrieved by accessing data stored in t devices, where Z e <t≤N. The reason behind this consideration is that edge devices are mobile and the encounter time of an authorized user with a storage node can vary over time. For example, storage nodes can go offline from time to time. In addition, edge devices are not designed for enterprise purposes, so their tolerance threshold for failures may be low. Therefore, the goal is to provide the ability to retrieve data by accessing less than all N storage nodes in case some storage nodes become unavailable due to mobility or failure.

[0077] As described in the previous method for providing secure distributed file storage with edge node loss, a redundant amount of data |F is stored among the available storage nodes redThis provides redundancy and allows file F to be reconstructed from retrieved data packets even if the identities of edge nodes storing packets related to file F change, for example because one or more edge nodes storing the data packets become unavailable after storing the packets.

[0078] In general, when a packet is retrieved from a distributed storage node, the result is a linear system of equations of the form Ax=b, where each row of A is a coefficient vector of keys and file partitions corresponding to the associated packet block, x is a vector of keys and file partitions, and b is a vector of values ​​of the packet blocks collected from storage nodes that the authorized user has access to. In a scenario where a Byzantine attacker is corrupting data retrieved from an edge storage node, an error parameter e is added to the system of equations to create a system of equations described as Ax=b+e. For storage nodes that have not yet been corrupted by a Byzantine attacker, the error parameter e will be zero. However, the authorized user does not have a priori knowledge of which nodes may be attacked.

[0079] Then, the present disclosure presents a method in which a system of linear equations created by packets retrieved from storage nodes is iteratively solved using a method referred to herein as the grouped Kaczmarz (GK) method, which is a modification of the windowed Kaczmarz (WK) method. The randomized Kaczmarz (RK) method converges to the optimal solution exponentially, expecting to obtain a system of linear equations without damaged equations. If some equations are noisy, i.e., their results deviate from the correct results and there are some small errors, such as by introducing damaged data from a Byzantine attacker who modifies the packets with a small margin, RK converges to a least squares solution. The WK method has been proposed for situations where some equations are damaged (i.e., their result values ​​deviate from the correct result values ​​with a larger margin, such as the case of data retrieved from a damaged storage node). However, the GK method presented herein modifies the WK method and then applies RK. In the context of distributed storage systems, the GK method has been found to be superior to the WK method. In the following, the GK method is described, and then an algorithm for using GK to make a secure distributed edge node storage system more robust against Byzantine attacks is described.

[0080] Following the previous example, a linear equation system of the form Ax=b+e can be constructed to recreate file F from the retrieved groupings of key information and file information, A can be the coefficient matrix of the linear equation system, and x is a vector of unknowns. In addition, b is a vector of results. In this regard, if Ax-bb is not damaged, the equation is equal to a zero vector. Otherwise, that is, in the case of at least one damaged node, b is equal to a non-zero vector. In a method related to the storage of information groups of file F in a distributed edge storage node system, x represents a vector of keys and file partitions, A is a coefficient matrix of keys and file partitions for generating groups to be stored in the storage node, and b is a vector of values ​​retrieved from the storage node by an authorized user. For a malicious storage node s i ,satisfy Inequality, where A j is the j of matrix A th OK, b j is the j of vector x th elements, and is the storage node s i The index set of the rows of the matrix A storing the grouped blocks in . In the GK method, if Figure 7 As shown in the algorithm definition in , a malicious node is detected. Then, the corresponding corrupted blocks from the corrupted node are omitted from the linear equation system, and the remaining packets from the detected uncorrupted nodes are used to retrieve the keys and file partitions.

[0081] The WK algorithm is based on applying RK on A and b for a number (k) of iterations to obtain the matrix x k . Then calculate from the vector Ax k The residuals of the elements of vector b are subtracted from , and d maximum residuals are identified. That is, the WK method performs iterations of the solution to determine the residuals of the converged solution relative to each input in the system of equations. Subsequently, the equation corresponding to the d maximum residuals is identified as corrupted, and in subsequent iterations of the solution of the linear system of equations, the remaining equations that were not identified as corrupted are used to (i) further iterate the solution and identify additional corrupted equations or (ii) extract unknowns as solutions (e.g., with sufficiently low residuals in the system). However, system performance can be improved by using GK compared to the brute force application of WK on all equations obtained from all grouped blocks stored in all storage nodes.

[0082] In the GK approach, computations are stored in specific storage nodes s i The residual average of all blocks in the group is calculated for all storage nodes (i.e., all nodes s i, i∈N), the d storage nodes with the largest average residuals are identified as malicious nodes. That is, in contrast to the WK approach where a corrupted equation (i.e., a single input to a system of equations) is identified as corrupted, the GK approach exploits the understanding of the context of the distributed edge storage system that if a given storage node is corrupted, then all packets retrieved from that corrupted storage node may also be corrupted, or at least suspicious. Therefore, the calculation of the average residual for a given node (which may contribute many blocks, thereby creating multiple inputs or equations for the system) allows the identification of the corrupted node, so that all information contributing to the system of equations from that node can be omitted from the system due to corruption. Therefore, identifying corrupted nodes based on the average residual can be used to omit many equations from the system, with the system inputs all retrieved from the d nodes identified as corrupted based on the average residual values.

[0083] Then, in subsequent iterations of the GK method, the system omits all blocks corresponding to the detected malicious storage nodes. Thus, the process of detecting corrupted nodes using the GK method continues to iterate until k b Z b The packet is detected as damaged or equivalent k b Z b Until the node is detected as malicious. Note that according to the Byzantine attack model, at most Z b The packet is damaged, and k b is a system parameter determined to ensure a margin to detect more packets as damaged if there are some correct packets among the packets detected as damaged.

[0084] Finally, all detected corrupted blocks are removed from the system of linear equations and RK is applied to the remaining equations to obtain a vector of unknowns, which are the keys and file partitions that can be used to reconstruct file F. Note that using RK helps capture possible small corruptions applied by malicious nodes that may not have been captured by calculating the average residual. Figure 7 The algorithm shown in summarizes the steps of GK.

[0085] Figure 8 An exemplary operation 800 of applying the GK method in a distributed edge storage system is depicted. At initialization operation 802, the number of iterations is set to zero. At iteration counter operation 804, it is determined whether the current number of iterations is less than the guaranteed margin parameter k b Multiply by Z b If the number of iterations is less than the guaranteed margin parameter multiplied by Z b, then define operation 806 to define a system of linear equations to obtain a solution to the above-retrieved grouping in the form of Ax=be. Solving operation 808 attempts to solve the system of linear equations. As described above, the use of redundant information can allow an over-configuration system to be created for linear equations. Therefore, parsing operation 808 may include iterating using an over-configuration system so that the residual value of each data block from a storage node can be calculated. Average operation 810 averages the residual of each storage node based on a single residual of each block from a given storage node. That is, the average residual of a given storage node can be determined based on the residual of each single block retrieved from a given storage node. Then, removal operation 812 removes all blocks in the system of linear equations from a given number (d) of storage nodes with the highest average residual. Parameter d can be a design parameter of a system selected to be greater than 1.

[0086] Operation 800 may be added to the number of iterations after each removal operation 812 and continued until the number of iterations equals the guaranteed margin parameter multiplied by Z b At this point, solve operation 814 returns a solution from the linear equation using the remaining blocks from the storage nodes that were not removed by remove operation 812.

[0087] As mentioned above, k b is a design parameter that can be adjusted by the system designer to balance system performance with robustness against corrupted information in the system. Fig. 9 It is shown that the choice of guarantee margin parameter k compared with WK b Impact on WK performance and optimal solutions for exemplary scenarios.

[0088] Specifically, Fig. 9 The performance of GK compared with WK and the optimal solution (where the corrupted blocks are known a priori) are shown. Fig. 9 The system model in the figure is for the number of storage nodes N = 50 and the maximum number of nodes that an eavesdropping adversary can access Z e =3 design. Each point in the plot is obtained by an authorized user who can access t storage nodes, where Z b = 3 is malicious, i.e., a damaged packet. The logarithm of the sum of absolute errors of all unknowns is calculated and the average is calculated through 100 random experiments. In each random experiment, Z b = 3 random nodes are selected as malicious and each packet stored at the malicious node is corrupted by adding random errors to the block. The error ranges between subtracting 50% from the actual value to adding 1000% to the actual value. The GK method used for simulation follows Figure 7 The algorithm details are shown in Figure 2, with parameters k = 20, k d =3 and d=1.

[0089] The performance of GK is evaluated using many other baselines drawn, including WK1, WK2, ideal GK, and the optimal solution. WK1 applies the WK method to the blocks received from the packet of the node selected by t until a packet from Z is detected. b One or more blocks of a node are damaged. Then, the blocks where damage is detected are omitted, and RK with 20,000 iterations is applied to the remaining blocks.

[0090] WK2 applies the WK method to the blocks received from the packet of the node selected by t until it detects the block from Z b One or more blocks of a node are damaged. Then, any node with at least one damaged block is detected to be malicious. In contrast to WK1, only all blocks of nodes detected as malicious are omitted, and RK with 20,000 iterations is applied to all blocks from the remaining nodes.

[0091] Ideal GK graph applications have a guaranteed metric parameter k in the range of two to seventeen b different values ​​of GK and select the guaranteed metric parameter k that has the best performance for each point in the graph b .

[0092] The best solution is an unrealistic solution that assumes that the authorized user knows the malicious nodes a priori and applies RK with 20,000 iterations on blocks received from non-malicious nodes. This is not a valid assumption for a practical solution because prior knowledge of the corrupted nodes is not available in real-world environments, however, this can be used as a reflection of the lower bound on the error when applying RK to uncorrupted blocks.

[0093] like Fig. 9 As shown, WK2 slightly outperforms WK1 with a negligible performance improvement achieved by omitting all blocks of packets where at least one block is detected as damaged instead of discarding only damaged packets. This slight improvement means that by exploiting the correlation about the state of damage or not in the blocks of a particular packet, the probability of detecting damaged packets and malicious nodes will increase, however, the brute force application of WK is still not an appropriate method to capture this correlation. Therefore, GK effectively captures this correlation and thus significantly outperforms WK1 and WK2. This proves that calculating the average of the residuals over all blocks of each packet instead of calculating the residuals for each single block and using this average as a metric for detecting malicious nodes provides a significant performance improvement for the WK method. In other words, in GK, the average distance between the intermediate solution and the blocks associated with the node is a metric for identifying malicious nodes, while in WK, the distance between the intermediate solution and each block is a metric for identifying damaged packets for WK1 or malicious nodes for WK2.

[0094] In order to ensure the metric parameter k bFor larger values ​​of , GK is more likely to detect malicious nodes (i.e., increase the true positive rate), and there is a risk of detecting honest nodes as malicious (i.e., increase the true negative rate) and ignoring valid packets from the packet set, which reduces the performance of RK applied in the last step of GK. Therefore, as another baseline, different k b ∈{2,3,...,17} value, and the best k b For the ideal GK diagram. Select the best k from b It can be seen that this allows reducing the logarithmic error by up to 5 points. Note that GK uses a fixed value k for the metric parameter for different values ​​of t. b = 3. Therefore, although the ideal GK is not practical, the graph of the ideal GK does show that with the k selected by the system designer b There is little difference between the performance gap of GK with the static value of WK1 and the ideal GK solution. Another useful baseline for evaluating the performance of GK is the optimal solution, where RK is applied to undamaged packets. As shown, the performance of GK is close to the performance bound of the ideal GK and the optimal solution, especially when considering the performance of WK1 and WK2.

[0095] Referring back to the system design for processing a file F for storage in multiple distributed edge nodes, Fig.12 The flowchart of includes the following stages: storage node selection 1200, file partitioning and building file blocks 1201, key generation and building key blocks 1202, and group generation 1203. Compared with previous methods that are not robust against Byzantine attackers, in the method of the present invention, storage selection is different to accommodate a system that is robust against Byzantine attackers. Specifically, since it is assumed that some blocks retrieved from distributed storage nodes will be corrupted, the initial distribution of blocks used to rebuild the file requires the distribution of additional redundant data. Therefore, the minimum number of storage nodes required depends on the acceptable errors of the application.

[0096] In the first stage 1200, a subset of all candidate storage nodes is selected to distribute the file F. To this end, a graph of the error versus the value of t (the number of storage nodes that the authorized user can access) is obtained by calculating 1204 the average error for each value of t, such as in Fig. 9 That is, for the selected storage node set At each iteration, t random storage nodes are selected, and the average error of the iteration is calculated. The process is repeated for different values ​​of t, and a graph is drawn based on the calculated errors relative to different values ​​of t. Then a set of storage nodes is selected 1206 so that the graph of the selected set obtained satisfies the system requirement related to the average error for a specific value of t being less than or equal to the acceptable error. After selecting 1206 the set of storage nodes, in a second stage 1201, the file F is divided into partitions 1208, and file blocks are constructed 1210 as linear combinations of the file partitions, where the file blocks are used to increase redundancy. In a third stage 1202, a key 1212 is generated, and a key block is constructed 1214 as a linear combination of the generated keys, where the key block is used to mask the file partitions. In a fourth stage 1203, the constructed file blocks and key blocks are used to generate 1216 storage groups, and the storage groups are stored 1218 in the set of selected storage nodes.

[0097] Further references Fig.10 , an algorithm for retrieving data from distributed edge storage nodes is shown. The method systematically recovers from corrupted packets and extracts keys and file partitions within an acceptable error range. Fig.11 An exemplary application of such an algorithm to recover files in a distributed edge storage system is shown in the exemplary operation 1100 in FIG. In the operation 1100 of recovering data, a subset of random storage nodes (e.g., t-∈ storage nodes) less than t is first randomly selected 1102. The GK method (e.g., as described above with respect to Figure 7 and Figure 8 1104) is applied to packets retrieved from a subset of storage nodes. Note that t is a system parameter determined according to system requirements. For example, a distributed storage system may be designed such that the average error in t is less than or equal to an acceptable error determined by a system administrator (e.g., using Fig. 9 ). Fig.11In the retrieval algorithm, for the first round, a storage node less than t is selected, that is, selecting t–∈ allows determining whether less than t nodes can be used to decode the file partition by accessing less than t nodes. For the randomly selected node, the mean absolute value (MAF1) of the current iteration is determined 1106. Then, a selection of an additional storage node is randomly made 1108. That is, the method includes iteratively adding another storage node, and includes a group received from the storage node added to the grouping set to which GK is applied. Therefore, operation 1100 includes applying 1110 the GK method to a new subset of storage nodes with additional random nodes. Determine a new mean absolute value (MAF2) 1114. Comparison operation 1116 compares the difference between the mean absolute value (MAF1) of the previous iteration and the mean absolute value (MAF2) of the current iteration. If the difference between MAF1 and MAF2 is greater than the iteration parameter (δ), the operation is iterated until it is determined that there is no further improvement based on the difference between MAF1 and MAF2 being less than δ (a parameter selectable by the system designer). Once such a condition is met, operation 1100 includes returning 1118 the latest value of the file partition retrieved from the application of the GK method to the latest subset of storage nodes.

[0098] Fig.13 An exemplary schematic diagram of a processing device 1300 suitable for implementing various aspects of the disclosed technology is shown. For example, the processing device 1300 may include a storage processing module 1350, which includes a storage processing device as described above. Thus, the processing device 1300 may include a storage processing device for the purpose of processing files according to any of the foregoing discussions. The processing system 1300 includes one or more processor units 1302, a memory 1304, a display 1306, and other interfaces 1308 (e.g., buttons). The memory 1304 typically includes both volatile memory (e.g., RAM) and non-volatile memory (e.g., flash memory). An operating system 1310, such as Microsoft An operating system, Apple macOS operating system, or Linux operating system, resides in memory 1304 and is executed by processor unit 1302, but it will be appreciated that other operating systems may be employed.

[0099] One or more application programs 1312 are loaded into the memory 1304 and executed by the processor unit 1302 on the operating system 1310. The application programs 1312 can receive input from various input local devices, such as a microphone 1334, an input accessory 1335 (e.g., a keypad, a mouse, a stylus, a touch pad, a joystick, an input device mounted on the instrument, etc.). In addition, the application programs 1312 can provide network connections (e.g., mobile phone networks, wireless networks, etc.) by using more communication transceivers 1330 and antennas 1338. ), communicating with such devices via a wired or wireless network to receive input from one or more remote devices, such as a remotely located smart device. The processing device 1300 may also include various other components, such as a positioning system (e.g., a global positioning satellite transceiver), one or more accelerometers, one or more cameras, an audio interface (e.g., a microphone 1334, an audio amplifier and speaker, and / or an audio jack), and a storage device 1328. Other configurations may also be used.

[0100] Processing device 1300 also includes a power supply 1316 that is powered by one or more batteries or other power sources and provides power to other components of processing device 1300. Power supply 1316 may also be connected to an external power source (not shown) that overwrites or recharges the internal batteries or other power sources.

[0101] Example implementations may include hardware and / or software embodied by instructions stored in memory 1304 and / or storage 1328 and processed by processor unit 1302. Memory 1304 may be memory of a host device or an accessory coupled to a host.

[0102] The processing system 1300 may include various tangible processor-readable storage media and intangible processor-readable communication signals. Tangible processor-readable storage may be embodied by any available media accessible by the processing system 1300 and includes both volatile and non-volatile storage media, removable and non-removable storage media. Tangible processor-readable storage media do not include intangible communication signals and include volatile and non-volatile, removable and non-removable storage media implemented in any method or technology for storing information such as processor-readable instructions, data structures, program modules or other data. Tangible processor-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CDROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage, or any other tangible media that can be used to store the desired information and can be accessed by the processing system 1300. In contrast to tangible processor-readable storage media, intangible processor-readable communication signals may embody processor-readable instructions, data structures, program modules or other data residing in a modulated data signal such as a carrier wave or other signal transmission mechanism. The term "modulated data signal" refers to an intangible communication signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, intangible communication signals include signals propagated through wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media.

[0103] Some embodiments may include articles of manufacture. Articles of manufacture may include tangible storage media for storing logic. Examples of storage media may include one or more types of processor-readable storage media capable of storing electronic data, including volatile or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writable or rewritable memory, etc. Examples of logic may include various software elements, such as software components, programs, applications, computer programs, applications, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, operating segments, methods, processes, software interfaces, application program interfaces (APIs), instruction sets, computing codes, computer codes, code segments, computer code segments, words, values, symbols, or any combination thereof. In one embodiment, for example, articles of manufacture may store executable computer program instructions that, when executed by a computer, cause the computer to perform methods and / or operations in accordance with the described embodiments. Executable computer program instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, etc. Executable computer program instructions can be implemented according to a predefined computer language, manner or syntax to instruct a computer to perform a certain operation segment. Instructions can be implemented using any suitable high-level, low-level, object-oriented, visual, compiled and / or interpreted programming language.

[0104] One general aspect of the present disclosure includes a method for secure distributed storage of information about a file in a plurality of distributed edge storage nodes. The method includes selecting a value Z e , enabling access to Z e An eavesdropping attacker at a storage node cannot decode any part of the information of a file of size |F| stored in the network, which stores the file in a distributed manner on Z e The method also includes selecting a value Z b , so that the information from Z can be corrected by applying the error correction code to the information retrieved from the storage node b The method includes selecting N* of the storage nodes that minimizes a cost function, the cost function including |F|, Z e , Z b , initial data access cost C T , and the transmission and download costs C dThe method includes initially allocating equal memory sizes from N* of the storage nodes to store a file, redundant error correction data, and a set of linear code keys. The method also includes iteratively determining a first cost of adding more storage nodes to the N* storage nodes and a second cost of allocating more memory from a subset of the N* storage nodes. The method includes constructing redundant data groups based on applying an error correction code to the linear code key set and a file partition masked with the linear code key set, and based on a minimum cost determined according to the iterative determination, allocating equal memory sizes from the N* of the storage nodes to store a file, redundant error correction data, and a set of linear code keys. * The files, linear code keys and redundant data packets are stored in the storage nodes. The storage nodes are sorted from large to small according to the storage capacity, so that the largest Z e The storage node stores the linear code key, the minimum NZ e -2Z b The storage node stores the file partitions, and the remaining 2Z b The storage nodes store the redundant packets.

[0105] Implementations may include one or more of the following features. For example, at least two of the N storage nodes may allocate memory of different sizes to store files. The N storage nodes may be ordered from the largest storage capacity of the first storage node to the smallest capacity of the Nth storage node.

[0106] In one example, the cost function includes the equation In one example, according to Round to an integer to select N * .

[0107] In one example, the redundant amount of data Stored in 2Z b The redundant data may be generated by multiplying a generator matrix of the error correction code by a vector including a linear code key and a file partition masked with the linear code key.

[0108] In one example, the method may further include retrieving the data packet from the storage node and applying the (n, k) error correction code to the data packet.

[0109] Another general aspect of the present disclosure includes a distributed secure edge storage system. The system includes a network interface operable to access a network having a plurality of edge storage nodes. A storage processing device is coupled to the network. The storage processing device is operable to select a value Z e , enabling access to Z e An eavesdropping attacker on a storage node cannot decode any part of the information of a file of size |F| stored in the network, which stores the file in a distributed manner on Z e In the above storage nodes. The storage processing device also selects the value Z b, so that the information from Z can be corrected by applying an error correction code to the information retrieved from the storage node b The storage processing device is operable to select N* storage nodes from the storage nodes that minimize a cost function, the cost function including |F|, Z e , Z b Initial data access cost C T , and the transmission and download costs C d . The storage processing device initially allocates equal memory sizes from N* storage nodes of the storage nodes to store files, redundant error correction data, and a set of linear code keys. Then, the storage processing device iteratively determines a first cost of adding more storage nodes to the N* storage nodes and a second cost of allocating more memory from a subset of the N* storage nodes, and constructs redundant data groups based on applying error correction codes to the linear code key set and file partitions masked with the linear code key set. The storage processing device also stores the files, linear code keys, and redundant data groups in the N≥N storage nodes based on a minimum cost determined according to the iterative determination. * In the storage node. The storage nodes are sorted from large to small according to storage capacity, so that the largest Z e The storage node stores the linear code key, the minimum NZ e -2Z b The storage node stores the file partitions, and the remaining 2Z b The storage nodes store the redundant packets.

[0110] Implementations may include one or more of the following features. For example, at least two of the N storage nodes may allocate memory of different sizes to store files. In this regard, the N storage nodes may be ordered from the largest storage capacity at the first storage node to the smallest capacity at the Nth storage node.

[0111] In one example, the cost function includes the equation N * The value can be based on Round to an integer to select. Redundant data Can be stored in the remaining 2Z b The redundant data may be generated by multiplying a generator matrix of the error correction code by a vector including a linear code key and a file partition masked with the linear code key.

[0112] In one example, the storage processing device further operates to retrieve the data packet from the storage node and apply the (n, k) error correction code to the data packet.

[0113] Another general aspect of the present disclosure includes one or more tangible processor-readable storage media embodied with instructions for executing, on one or more processors and circuits of a device, a process for securely distributing storage of information about a file in a plurality of distributed edge storage nodes. The process includes selecting a value Z e , enabling access to Z e An eavesdropping attacker on a storage node cannot decode any part of the information of a file of size |F| stored in the network, which stores the file in a distributed manner on Z e The process also includes selecting the value Z b , so that the information from Z can be corrected by applying the error correction code to the information retrieved from the storage node b The process also includes selecting N* storage nodes that minimize the cost function among the storage nodes, where the cost function includes |F|, Z e , Z b Initial data access cost C T , and the transmission and download costs C d . Then, the process includes initially allocating equal memory sizes of N* storage nodes from the storage nodes to store the file, redundant error correction data, and a set of linear code keys, and iteratively determining a first cost of adding more storage nodes to the N* storage nodes, and a second cost of allocating more memory from a subset of the N* storage nodes. Construction of redundant data groups is based on applying error correction codes to the set of linear code keys and partitioning of files masked with the set of linear code keys. Based on a minimum cost determined according to the iterative determination, the process includes storing the file, the linear code key, and the redundant data group in N ≥ N storage nodes. * In the storage node. The storage nodes are sorted from large to small according to storage capacity, so that the largest Z e The storage node stores the linear code key, the minimum NZ e -2Z b The storage nodes store the file partitions, and the redundant groups are stored in the remaining 2Z b Storage node.

[0114] Implementations may include one or more of the following features. For example, at least two of the N storage nodes may allocate memory of different sizes to store files. In this regard, the N storage nodes may be ordered from the largest storage capacity at the first storage node to the smallest capacity at the Nth storage node.

[0115] In one example, the cost function may include the equation N * The value can be based on Select by rounding to an integer. The data of the redundancy amount can be stored in the remaining 2Z b storage. In one example, the redundant data is generated by multiplying a generating matrix of an error correction code by a vector including a linear code key and a file partition masked with the linear code key.

[0116] In one example, the process further includes retrieving data packets from storage nodes and applying an (n,k) error correction code to the data packets.

[0117] Another general aspect of the present disclosure includes a method for securely and distributively storing information about files in a plurality of distributed edge storage nodes. The method includes selecting N storage nodes connected via a network to store a file of size |F| and redundant data of size |F red |. At least two of the N storage nodes are assigned different sizes of memory to store the file. Subsequently, the N storage nodes are sorted from the maximum storage capacity |s1| at the first storage node to the minimum capacity |s N | at the Nth storage node. The N storage nodes are selected at least in part based on an average error calculation for a number of t random nodes selected from the N storage nodes of the maximum number of Z b failed nodes. The method further includes selecting a value of Z e < N such that an attacker who can access Z e storage nodes cannot decode the file. The method further includes partitioning the file into file partitions, generating keys stored in the first Z e of the N storage nodes, and creating storage blocks including independent linear combinations of the keys and the file partitions. Subsequently, the method includes storing storage packets including independent linear combinations of the keys and the file partitions in the Z e +1 to the Nth storage node.

[0118] Embodiments may include one or more of the following features. For example, the storage block may include a first part that is a function of the file partition and a second part that is a function of the key.

[0119] The method may further include calculating an average error calculation for a number of t random nodes selected from the N of the Z b failed nodes for different values of t. The method may further include establishing an acceptable error value. Subsequently, a value of N is selected such that the average error calculation for the t random nodes of the N selected nodes is less than the acceptable error.

[0120] In one example, the method includes retrieving data of a file F by an authorized user by selecting a subset of N storage nodes that is less than t, and retrieving storage packets from the subset of N storage nodes, the storage packets including keys and independent linear combinations of file partitions masked by the keys of each corresponding node of the subset of t from the N storage nodes. The method further includes establishing a plurality of linear equations such that coefficients of the linear key codes used to generate the file portions masked by the linear key codes are equal to the retrieved values. An average residual of each corresponding storage node in the subset of N storage nodes can be determined. Subsequently, the method can include removing a selected number of storage nodes from the subset of storage nodes having the maximum average residual of the subset of storage nodes. The method can include iteratively performing the establishing, determining, and removing operations for a predetermined number of iterations, and extracting the values of the file portions from the plurality of linear equations of the final iteration. The predetermined number of iterations can be based on a guaranteed margin parameter k b and a maximum number of Z b failed nodes.

[0121] In one example, the retrieving operation further includes adding another random storage node to the subset of N storage nodes and extracting the values of the file portions from the plurality of linear equations. The method can further include calculating a first mean absolute error of a previous iteration of the subset of N storage nodes and a second mean absolute error of the current iteration of the subset of N storage nodes that includes the additional one random storage node. The method further includes determining whether the difference between the first mean absolute error and the second mean absolute error is below an iteration threshold. While the difference between the first mean absolute error and the second mean absolute error is above the iteration threshold, the method can include continuing the iteration by adding an additional storage node to the subset of N storage nodes. When the difference between the first mean absolute error and the second mean absolute error is below the iteration threshold, the method can include reconstructing the file based on the file information extracted from the solution of the plurality of linear equations of the last iteration of the solution of the plurality of linear equations.

[0122] Another general aspect includes a distributed secure edge storage system. The system includes at least N storage nodes that are coupled via a network to store a file of size |F| and redundant data of size |F red |. At least two of the N storage nodes are allocated different sizes of memory to store the file. The N storage nodes are sorted from a maximum storage capacity |s1| of a first storage node to a minimum capacity |s N | of the Nth storage node. The N storage nodes are at least partially based on an average error calculation of a number of t random nodes selected from the N storage nodes for a maximum number of Z b failed nodes. The system further includes a storage processing device coupled to the network and operable to select a value Z e < N, wherein Ze The attacker of the storage node cannot decode the file. The storage processing device is also operable to divide the file into file partitions, generate the first Z files stored in the N storage nodes, and store them in the N storage nodes. e and creates a storage block comprising an independent linear combination of the key and the file partition. The storage processing device further operates to e Storage groups including independent linear combinations of keys and file partitions are stored in the +1 to Nth storage nodes.

[0123] Implementations may include one or more of the following features. For example, a storage block may include a first portion that is a function of the file partition and a second portion that is a function of the key. In addition, the storage processing device may be further operable to calculate the value of Z for different values ​​of t. b The average error of many t random nodes selected from N of the damaged nodes is calculated, and an acceptable error value is established. Therefore, the value of N can be selected so that the average error of t random nodes selected from N nodes is less than the acceptable error.

[0124] In one example, the storage processing device further operates to retrieve data of file F by selecting a subset of N storage nodes less than t by an authorized user, retrieving storage groups from a subset of N storage nodes including a key and an independent linear combination of file partitions masked by the key of each corresponding node from the subset t of N storage nodes, and establishing multiple linear equations so that the coefficients of the linear key code used to generate the file portion masked by the linear key code are equal to the retrieved value. The data retrieval of file F may also include determining the average residual of each corresponding storage node in the subset of N storage nodes and removing a selected number of storage nodes from the subset of storage nodes with the largest average residual. The retrieval may include iteratively performing the establishment, determination, and removal operations for a predetermined number of iterations, and extracting the value of the file portion from the multiple linear equations of the final iteration. The predetermined number of iterations may be based on a guarantee margin parameter k b and the maximum number of Z b Damaged node.

[0125] In one example, the storage processing device may further operate to retrieve a storage grouping by adding another random storage node to a subset of N storage nodes and extracting the value of the file portion from a plurality of linear equations. Subsequently, the storage processing device may operate to calculate a first mean absolute error of a previous iteration of the subset of N storage nodes and a second mean absolute error of the current iteration of the subset of N storage nodes including another random storage node, and determine whether the difference between the first mean absolute error and the second mean absolute error is below an iteration threshold. When the difference between the first mean absolute error and the second mean absolute error is above the iteration threshold, the storage processing device continues the iteration by adding another storage node to the subset of N storage nodes. When the difference between the first mean absolute error and the second mean absolute error is below the iteration threshold, the storage processing device operates to reconstruct the file based on the file information extracted from the solution of the plurality of linear equations of the last iteration of the solution of the plurality of linear equations.

[0126] Another general aspect of the present disclosure includes one or more tangible processor-readable storage media embodied with instructions for performing, on one or more processors and circuitry of a device, a process for securely and distributively storing information about a file in a plurality of distributed edge storage nodes. The process includes selecting N storage nodes connected via a network to store a file of size |F| and redundant data of size |F red |. At least two of the N storage nodes are assigned different sizes of memory for storing the file, and the N storage nodes are reordered from the maximum storage capacity |s1| of the first storage node to the minimum capacity |s N | of the Nth storage node, and the N storage nodes are at least partially based on the average error calculation for many t random nodes selected from N for up to a maximum number Z b of corrupted nodes. The process also includes selecting a value of Z e < N such that an attacker who can access Z e storage nodes cannot decode the file. The process also includes partitioning the file into file partitions, generating keys stored in the first Z e of the N storage nodes, and creating storage blocks including independent linear combinations of the keys and the file partitions. The process also includes storing storage groupings including independent linear combinations of the keys and the file partitions in the Z e + 1 to the Nth storage node.

[0127] Embodiments may include one or more of the following features. For example, the storage block may include a first portion that is a function of the file partition and a second portion that is a function of the key.

[0128] In one example, the process may further include calculating for different values of t the mean from Zb The average error of many t random nodes selected from the N nodes of the damaged node is calculated, and an acceptable error value is determined. The value of N can be selected so that the average error of the t random nodes for the N selected nodes is less than the acceptable error.

[0129] In one example, the process also includes retrieving data for file F by an authorized user. The retrieval may include selecting a subset of N storage nodes less than t, and retrieving a storage grouping from the subset of N storage nodes, the storage grouping including a key and an independent linear combination of a file partition masked by the key of each corresponding node from the subset t of N storage nodes. The retrieval may include establishing a plurality of linear equations, making coefficients of a linear key code used to generate a portion of the file masked by the linear key code equal to the retrieved value, and determining an average residual for each corresponding storage node in the subset of N storage nodes. The process may include removing a selected number of storage nodes from the subset of storage nodes having the largest average residual, and iteratively performing the establishing, determining, and removing operations for a predetermined number of iterations. Subsequently, the process may include extracting the value of the file portion from the plurality of linear equations of the final iteration. The predetermined number of iterations may be based on a guarantee margin parameter k b and the maximum number of Z b Damaged node.

[0130] Then, the retrieval operation may also include adding another random storage node to the subset of N storage nodes and extracting the value of the file portion from the plurality of linear equations. The retrieval may include calculating a first mean absolute error of a previous iteration of the subset of N storage nodes and a second mean absolute error of a current iteration of the subset of N storage nodes including the another random storage node, and determining whether the difference between the first mean absolute error and the second mean absolute error is below an iteration threshold. When the difference between the first mean absolute error and the second mean absolute error is above the iteration threshold, the process includes continuing the iteration by adding another storage node to the subset of N storage nodes. When the difference between the first mean absolute error and the second mean absolute error is below the iteration threshold, the process includes rebuilding the file based on file information extracted from the solution of the plurality of linear equations of the last iteration of the solution of the plurality of linear equations.

[0131] The embodiments described herein are implemented as logical steps in one or more computer systems. The logical operations may be implemented as (1) a series of processor-implemented steps executed in one or more computer systems and as (2) interconnected machine or circuit modules within one or more computer systems. The implementation is a matter of choice and depends on the performance requirements of the computer system utilized. Thus, the logical operations that make up the embodiments described herein are variously referred to as operations, steps, objects, or modules. Further, it should be understood that, unless explicitly required otherwise or the language of the claims inherently requires a particular order, the logical operations may be performed in any order.

[0132] Further examples

[0133] Example 1. A method for securely and distributively storing information about files in a plurality of distributed edge storage nodes, comprising:

[0134] selecting N storage nodes coupled via a network to store a file of size |F| and redundant data of size |F red |, wherein at least two of the N storage nodes are allocated different sizes of memory for storing the file, the N storage nodes being sorted from the maximum storage capacity |s1| of the first storage node to the minimum capacity |s N | of the Nth storage node, and the N storage nodes being selected at least in part based on a calculation of the average error of a number of t random nodes selected from N random nodes of the maximum number of Z b failed nodes;

[0135] selecting a value of Z e < such that an attacker with access to Z e storage nodes cannot decode the file;

[0136] dividing the file into file partitions;

[0137] generating keys to be stored in the first Z e storage nodes of the N storage nodes;

[0138] creating storage blocks comprising independent linear combinations of the keys and the file partitions; and

[0139] storing the storage packets comprising independent linear combinations of the keys and the file partitions in the Z e +1 to Nth storage nodes.

[0140] Example 2. The method of Example 1, wherein the storage block comprises a first part that is a function of the file partition and a second part that is a function of the key.

[0141] Example 3. The method as described in Example 2 further includes

[0142] For different values ​​of t, calculate the b The average error calculation of many t random nodes selected from the N random nodes of the damaged node;

[0143] Determine acceptable error values; and

[0144] Wherein N is selected so that the average error of t random nodes of the nodes selected by the N is less than the acceptable error.

[0145] Example 4. The method of Example 2 further includes:

[0146] Authorized users retrieve the data of the file F in the following ways:

[0147] Selecting a subset of the N storage nodes that is smaller than t;

[0148] retrieving a storage group from said subset of N storage nodes comprising an independent linear combination of a key and a file partition masked with a key from each corresponding node of said subset t of N storage nodes;

[0149] establishing a plurality of linear equations that equate coefficients of the linear key code used to generate the portion of the file masked with the linear key code to the retrieved values;

[0150] determining an average residual for each corresponding storage node in the subset of the N storage nodes;

[0151] removing a selected number of storage nodes from the subset of the storage nodes having the largest average residual of the subset of the storage nodes;

[0152] iteratively performing the establishing, determining, and removing operations for a predetermined number of iterations; and

[0153] The value for the portion of the file is extracted from the plurality of linear equations at a final iteration.

[0154] Example 5. The method of Example 4, wherein the predetermined number of iterations is based on a guarantee margin parameter k b and the maximum number of Z b Damaged node.

[0155] Example 6. The method of Example 4, wherein the retrieving operation further comprises:

[0156] adding another random storage node to the subset of the N storage nodes;

[0157] Extract the value of the file part from the plurality of linear equations;

[0158] Calculate a first mean absolute error of a previous iteration of the subset of the N storage nodes and a second mean absolute error of a current iteration of the subset of the N storage nodes including the other random storage node; and

[0159] Determine whether the difference between the first mean absolute error and the second mean absolute error is below an iteration threshold.

[0160] Example 7. The method according to Example 6, wherein although the difference between the first mean absolute error and the second mean absolute error is above the iteration threshold, the iteration is continued by adding an additional storage node to the subset of the N storage nodes.

[0161] Example 8. The method according to Example 7, wherein when the difference between the first mean absolute error and the second mean absolute error is below the iteration threshold, the file is reconstructed based on file information extracted from the solution of the plurality of linear equations of the last iteration of the solution of the plurality of linear equations.

[0162] Example 9. A distributed secure edge storage system, comprising:

[0163] At least N storage nodes coupled via a network to store a file of size |F| and redundant data of size |F red |, wherein at least two of the N storage nodes are assigned different sizes of memory for storing the file, the N storage nodes are sorted from the maximum storage capacity |s1| of the first storage node to the minimum capacity |s N | of the Nth storage node, and the N storage nodes are at least partially based on the average error calculation of many t random nodes selected from N of the maximum number of Z b failed nodes;

[0164] A storage processing device, the storage processing device being coupled to the network and operable to perform the following operations:

[0165] Select a value Z e <N, wherein an attacker who can access Z e storage nodes cannot decode the file;

[0166] Divide the file into file partitions;

[0167] Generate a key stored in the first Z e storage nodes of the N storage nodes;

[0168] creating a storage block comprising an independent linear combination of the key and the file partition; and

[0169] In Z e The storage groups including independent linear combinations of the keys and the file partitions are stored in the +1 to Nth storage nodes.

[0170] Example 10. The system of Example 9, wherein the storage block includes a first portion that is a function of the file partition and a second portion that is a function of the key.

[0171] Example 11. The system of Example 9, wherein the storage processing device is further operative to perform the following operations:

[0172] For different values ​​of t, calculate the b The average error calculation of many t random nodes selected from N of the damaged nodes;

[0173] Determine acceptable error values; and

[0174] Wherein N is selected so that the average error of t random nodes of the nodes selected by the N is less than the acceptable error.

[0175] Example 12. The system of Example 9, wherein the storage processing device is further operable to perform the following operations:

[0176] Authorized users retrieve the data of the file F in the following ways:

[0177] Selecting a subset of the N storage nodes that is smaller than t;

[0178] retrieving a storage group from said subset of N storage nodes comprising an independent linear combination of a key and a file partition masked with a key from each corresponding node of said subset t of N storage nodes;

[0179] establishing a plurality of linear equations that equate coefficients of the linear key code used to generate the portion of the file masked with the linear key code to the retrieved values;

[0180] determining an average residual for each corresponding storage node in the subset of the N storage nodes;

[0181] removing a selected number of storage nodes from the subset of the storage nodes having the largest average residual of the subset of the storage nodes;

[0182] iteratively performing the establishing, determining, and removing operations for a predetermined number of iterations; and

[0183] The value for the portion of the file is extracted from the plurality of linear equations at a final iteration.

[0184] Example 13. The system of Example 12, wherein the predetermined number of iterations is based on a guarantee margin parameter k b and the maximum number of Z b Damaged node.

[0185] Example 14. The system of Example 13, wherein the storage processing device is further operative to retrieve the storage grouping by:

[0186] adding another random storage node to the subset of the N storage nodes;

[0187] extracting said value of said portion of said file from said plurality of linear equations;

[0188] calculating a first mean absolute error of a previous iteration of the subset of the N storage nodes and a second mean absolute error of a current iteration of the subset of the N storage nodes including the another random storage node; and

[0189] It is determined whether a difference between the first mean absolute error and the second mean absolute error is below an iteration threshold.

[0190] Example 15. The system of Example 14, wherein when a difference between the first mean absolute error and the second mean absolute error is above an iteration threshold, iterating continues by adding an additional storage node to the subset of the N storage nodes.

[0191] Example 16. The system of Example 15, wherein when the difference between the first mean absolute error and the second mean absolute error is below an iteration threshold, the file is reconstructed based on file information extracted from the solution of the multiple linear equations from a last iteration of the solution of the multiple linear equations.

[0192] Example 17. One or more tangible processor-readable storage media embodied with instructions for executing, on one or more processors and circuits of a device, a process for securely distributing storage of information about a file in a plurality of distributed edge storage nodes, the process comprising:

[0193] Select N storage nodes coupled via the network to store a file of size |F| and a file of size |F red | redundant data, wherein at least two of the N storage nodes allocate memory of different sizes for storing the file, and the N storage nodes have a maximum storage capacity of |s1| of the first storage node to a minimum capacity of |s1| of the Nth storage node.N Sort, and the N storage nodes are at least partially calculated based on the average error of a number of t random nodes selected from N of the most damaged nodes; b The calculation of the average error of a number of t random nodes selected from N of the most damaged nodes;

[0194] Select Z e The value of N, where an attacker who can access the Z e Storage nodes cannot decode the file;

[0195] Divide the file into file partitions;

[0196] Generate keys stored in the first Z e Storage nodes;

[0197] Create storage blocks including independent linear combinations of the keys and the file partitions; and

[0198] At Z e +1 to the Nth storage node, store the storage groups including independent linear combinations of the keys and the file partitions.

[0199] Example 18. One or more tangible processor-readable storage media as described in Example 17, wherein the storage block includes a first part that is a function of the file partition and a second part that is a function of the key.

[0200] Example 19. One or more tangible processor-readable storage media as described in Example 17, wherein the process further includes:

[0201] For different values of t, calculate the average error calculation of a number of t random nodes selected from N random nodes of the damaged nodes; b The calculation of the average error of a number of t random nodes selected from N random nodes of the damaged nodes;

[0202] Determine an acceptable error value; and

[0203] Wherein N is selected such that the average error calculation of the t random nodes of the N selected nodes is less than the acceptable error.

[0204] Example 20. One or more tangible processor-readable storage media as described in Example 17, wherein the process further includes:

[0205] Authorize a user to retrieve data of the file F by:

[0206] Select a subset of the N storage nodes that is less than t;

[0207] Retrieve storage groups from the subset of the N storage nodes including independent linear combinations of keys and file partitions masked with keys from each corresponding node of the subset t of the N storage nodes;

[0208] establishing a plurality of linear equations that equate coefficients of the linear key code used to generate the portion of the file masked with the linear key code to the retrieved values;

[0209] determining an average residual for each corresponding storage node in the subset of the N storage nodes;

[0210] removing a selected number of storage nodes from the subset of the storage nodes having the largest average residual of the subset of the storage nodes;

[0211] iteratively performing the establishing, determining, and removing operations for a predetermined number of iterations; and

[0212] The value for the portion of the file is extracted from the plurality of linear equations at a final iteration.

[0213] Example 21. One or more tangible processor-readable storage media of Example 20, wherein the predetermined number of iterations is based on a guaranteed margin parameter k b and the maximum number Z b Damaged node.

[0214] Example 22. The one or more tangible processor-readable storage media of Example 20, wherein the retrieving operation further comprises:

[0215] adding another random storage node to the subset of the N storage nodes;

[0216] extracting said value of said portion of said file from said plurality of linear equations;

[0217] calculating a first mean absolute error of a previous iteration of the subset of the N storage nodes and a second mean absolute error of a current iteration of the subset of the N storage nodes including the another random storage node; and

[0218] It is determined whether a difference between the first mean absolute error and the second mean absolute error is below an iteration threshold.

[0219] Example 23. One or more tangible processor-readable storage media as described in Example 22, wherein when the difference between the first mean absolute error and the second mean absolute error is above an iteration threshold, iterating continues by adding another storage node to the subset of the N storage nodes.

[0220] Example 24. One or more tangible processor-readable storage media as described in Example 23, wherein when the difference between the first mean absolute error and the second mean absolute error is below an iteration threshold, the file is reconstructed based on the file information extracted from the solution of the multiple linear equations in a last iteration from the solution of the multiple linear equations.

Claims

1. A method for secure distributed storage of information about files in a plurality of distributed edge storage nodes, comprising: Select N storage nodes coupled via a network to store files of size |F| and files of size |F red | redundant data, wherein at least two of the N storage nodes allocate memory of different sizes for storing the file, and the N storage nodes have a maximum storage capacity of |s1| of the first storage node to a minimum capacity of |s1| of the Nth storage node N | sorted, and the N storage nodes are based at least in part on the largest number of Z b The average error calculation of many t random nodes selected from the N random nodes of the damaged node; Select Z e <The value of N, where Z can be accessed e An attacker of the storage node cannot decode the file; dividing the file into file partitions; Generate the first Z stored in the N storage nodes e The key in the storage node; creating a storage block comprising an independent linear combination of the key and the file partition; as well as In Z e The storage groups including independent linear combinations of the keys and the file partitions are stored in the +1 to Nth storage nodes.

2. The method of claim 1, wherein the storage block comprises a first portion that is a function of the file partition and a second portion that is a function of the key.

3. The method of claim 2, further comprising For different values ​​of t, calculate the b The average error calculation of many t random nodes selected from the N random nodes of the damaged node; Determine acceptable error values; and Wherein N is selected so that the average error of t random nodes of the nodes selected by the N is less than the acceptable error.

4. The method of claim 2, further comprising: Authorized users retrieve the data of the file F in the following ways: Selecting a subset of the N storage nodes that is smaller than t; retrieving a storage group from said subset of N storage nodes comprising an independent linear combination of a key and a file partition masked with a key from each corresponding node of said subset t of N storage nodes; establishing a plurality of linear equations that equate coefficients of the linear key code used to generate the portion of the file masked with the linear key code to the retrieved values; determining an average residual for each corresponding storage node in the subset of the N storage nodes; removing a selected number of storage nodes from the subset of the storage nodes having the largest average residual of the subset of the storage nodes; iteratively performing the establishing, determining, and removing operations for a predetermined number of iterations; as well as The value for the portion of the file is extracted from the plurality of linear equations at a final iteration.

5. The method of claim 4, wherein the predetermined number of iterations is based on a guarantee margin parameter k b and the maximum number of Z b Damaged node.

6. The method of claim 4, wherein the retrieving operation further comprises: adding another random storage node to the subset of the N storage nodes; extracting said value of said portion of said file from said plurality of linear equations; calculating a first mean absolute error of a previous iteration of the subset of the N storage nodes and a second mean absolute error of a current iteration of the subset of the N storage nodes including the another random storage node; as well as It is determined whether a difference between the first mean absolute error and the second mean absolute error is below an iteration threshold.

7. The method of claim 6, wherein while the difference between the first mean absolute error and the second mean absolute error is above an iteration threshold, iteration is continued by adding an additional storage node to the subset of the N storage nodes.

8. The method of claim 7, wherein when the difference between the first mean absolute error and the second mean absolute error is below an iteration threshold, reconstructing the file based on file information extracted from the solution of the plurality of linear equations from a last iteration of the solution of the plurality of linear equations.

9. A distributed secure edge storage system, comprising: At least N storage nodes are coupled via a network to store files of size |F| and files of size |F red | redundant data, wherein at least two of the N storage nodes allocate memory of different sizes for storing the file, and the N storage nodes have a maximum storage capacity of |s1| of the first storage node to a minimum capacity of |s1| of the Nth storage node. N | sorted, and the N storage nodes are based at least in part on the largest number of Z b The average error calculation of many t random nodes selected from N of the damaged nodes; A storage processing device coupled to the network and operable to: Selected value Z e <N, where Z can be accessed e An attacker of the storage node cannot decode the file; dividing the file into file partitions; Generate the first Z stored in the N storage nodes e The key in the storage node; creating a storage block comprising an independent linear combination of the key and the file partition; as well as In Z e The storage groups including independent linear combinations of the keys and the file partitions are stored in the +1 to Nth storage nodes.

10. One or more tangible processor-readable storage media embodied with instructions for executing, on one or more processors and circuits of a device, a process for securely distributing storage of information about a file in a plurality of distributed edge storage nodes, the process comprising: Select N storage nodes coupled via the network to store a file of size |F| and a file of size |F red | redundant data, wherein at least two of the N storage nodes allocate memory of different sizes for storing the file, and the N storage nodes have a maximum storage capacity of |s1| of the first storage node to a minimum capacity of |s1| of the Nth storage node. N | sorted, and the N storage nodes are based at least in part on the largest number of Z b The average error calculation of many t random nodes selected from N of the damaged nodes; Select Z e <The value of N, where Z can be accessed e An attacker of the storage node cannot decode the file; dividing the file into file partitions; Generate the first Z stored in the N storage nodes e The key in the storage node; creating a storage block comprising an independent linear combination of the key and the file partition; as well as In Z e The storage groups including independent linear combinations of the keys and the file partitions are stored in the +1 to Nth storage nodes.

Images