Low-delay aggregated encoded data repair method and system for satellite storage networks

Abstract

The application provides a low-delay aggregated encoded data repair method and system for a satellite storage network, and the low-delay aggregated encoded data repair method comprises the following steps: step one, selecting k source nodes with the shortest paths from n-1 available data storage nodes; step two, constructing a coding path tree (CPT); and step three, repairing data of a low earth orbit satellite storage constellation (LSC) by using the CPT obtained in step two. The application has the beneficial effect that the low-delay aggregated encoded data repair method can reduce energy consumption and repair time delay in the repair process of the LSC storage nodes.

Description

Technical Field

[0001] This invention relates to the field of big data processing, and in particular to a low-latency aggregated encoded data repair method and system for satellite storage networks. Background Technology

[0002] With the rapid development of airborne computers and inter-satellite links, low-Earth orbit (LEO) satellite storage constellations (LSCs) have become an efficient and promising infrastructure for data storage in the era of big data. Compared to terrestrial data centers, they offer advantages in secure data storage and massive inter-satellite connectivity, making them an important complement to satellite internet services such as Starlink, OneWeb, and Kuiper for global data sharing in 6G non-terrestrial networks. However, the harsh space environment can lead to the failure of storage devices in orbit and the loss of stored files. In contrast, terrestrial big data storage systems, such as data centers, typically use erasure coding (EC) to repair damaged nodes and ensure the system's normal operation to ensure reliability. To save bandwidth and energy resources during the repair process, researchers have proposed various coding repair schemes for terrestrial data centers.

[0003] However, while data center storage nodes (servers) are relatively close together, LSC storage nodes (satellites) are much farther apart. The single-hop propagation latency can reach 2ms to 6ms, which can easily cause node repair latency to exceed business timeliness requirements. Furthermore, radio waves attenuate with increasing propagation distance, and due to the limited battery life of satellite networks and the high cost of radio power, LSC requires a low-energy-consumption, low-latency node repair method. These requirements mean that data center node repair methods are not entirely suitable for LSC.

[0004] Currently, research on node repair schemes for big data storage systems, especially Aggregate Coding Repair (ACR), mainly focuses on terrestrial data centers. In this scenario, data transmission latency dominates the repair latency and can well meet the requirements of low bandwidth overhead. However, this method is not well applied to LSC. The main reason is that the inter-satellite link distance is very long, and the single-hop propagation latency of communication can reach 2ms to 6ms. This will make the data propagation latency account for a large part of the repair latency, which can easily cause the node repair latency to exceed the budget. Furthermore, due to the attenuation of radio waves with the increase of propagation distance, the limited battery life of satellite networks, and the high cost of radio power, LSC needs a node repair method with low energy overhead and low repair latency. Summary of the Invention

[0005] To address the problems in the prior art, this invention provides a low-latency aggregated encoded data repair method for satellite storage networks.

[0006] This invention provides a low-latency aggregated coded data repair method for satellite storage networks, comprising the following steps:

[0007] Step 1: Select k source nodes with the shortest path from n-1 available data storage nodes;

[0008] Step 2: Construct the encoding path tree;

[0009] Step 3: Use the encoded path tree obtained in Step 2 to repair the data of the Low Earth Orbit Satellite Storage Constellation (LSC).

[0010] As a further improvement of the present invention, step one includes:

[0011] Step A1, obtain the source node v src To the replacement node v rn Find the k shortest paths and store them in set P;

[0012] Step A2: Define the k shortest paths obtained in step A1 as the paths with the minimum propagation delay, and sort them in descending order of distance.

[0013] As a further improvement of the present invention, step two includes:

[0014] Step 1: Select the path with the largest propagation delay in set P, add it to the encoded path tree, and add its associated nodes to the node set V. CPT In the middle, its delay is defined as the repair delay T. ACR ;

[0015] Step 2: Process the remaining paths in set P in descending order of propagation delay.

[0016] As a further improvement of the present invention, step 2 specifically includes:

[0017] Step S1, determine the source node set v src Is it empty? If it is empty, it means the source node set v is empty. src Once the path processing is complete, it enters the node set V. left Path processing steps; otherwise, proceed to the next step.

[0018] Step S2, from the source node set v src Obtain the nearest node set V from the remaining source nodes. CPT The source node;

[0019] Step S3: Determine whether the propagation delay of the source node obtained in step S2 is greater than the repair delay T. ACR If so, add the source node to the node set V. leftand remove the source node from the source node set v src Remove the path from the tree and return to step S1; otherwise, add the path to the encoded path tree and add the nodes passed through to the node set V. CPT Then remove the node from the source node set v src Remove from the list and then return to step S1.

[0020] As a further improvement of the present invention, in step S1, the node set V left The path processing steps specifically include:

[0021] Step S10, determine the node set V left Is it empty? If it is empty, it means the node set V is empty. left Once the path processing is complete, the required encoded path tree is obtained. Then, step three is executed: the obtained encoded path tree is used to repair the low-Earth orbit satellite storage constellation data. Otherwise, the next step is executed.

[0022] Step S11, from the node set V left Get the distance V from the remaining source nodes CPT The shortest path;

[0023] Step S12: Determine whether the propagation delay of the nearest path obtained in step S11 is greater than the repair delay T. ACR If so, add the original path from set P to the encoded path tree, and add the nodes traversed to the node set V. left Then return to step S10; otherwise, add the path to the encoded path tree and add the nodes it passes through to the node set V. CPT Then return to step S10.

[0024] As a further improvement of the present invention, in step A1, Dijkstra's algorithm is used to obtain the source node v. src To the replacement node v rn Find the k shortest paths and store them in set P.

[0025] The present invention also discloses a low-latency aggregated coded data repair system for satellite storage networks, comprising: a memory, a processor, and a computer program stored on the memory, wherein the computer program is configured to implement the steps of the low-latency aggregated coded data repair method of the present invention when invoked by the processor.

[0026] The beneficial effects of this invention are: the low-latency aggregated coding data repair method of this invention can reduce the energy consumption and repair latency of the storage node repair process in low-orbit storage constellations. Attached Figure Description

[0027] Figure 1This is a flowchart of the low-latency aggregated encoded data repair method of the present invention;

[0028] Figure 2 is a performance diagram of several repair schemes under different RS(n,k) encodings with a fixed number of stripes of 32 according to the present invention; Figure 2(a) is a repair delay diagram; Figure 2(b) is an energy cost diagram.

[0029] Figure 3 is a performance diagram of several repair schemes under different RS(n,k) encodings with a fixed file size of 1024MB; Figure 3(a) is a repair latency diagram; Figure 3(b) is an energy cost diagram. Detailed Implementation

[0030] This invention discloses a low-latency aggregation coding data repair method for satellite storage networks. First, it proposes a Coding Path Tree (CPT) model to describe the data flow of ACR (Accelerated Response Code). Next, the node repair problem of a Low Earth Orbit Storage Constellation (LEO Storage Constellation, LSC) is transformed into a Coding Path Tree (CPT) construction problem. Furthermore, a latency-sensitive, energy-efficient ACR scheme (DE-ACR) is proposed, based on a CPT construction algorithm that combines the ideas of shortest path and minimum Steiner tree. The superiority of DE-ACR is demonstrated by comparison with several other coding schemes under different RS(n,k) coding conditions.

[0031] To ensure the reliability of storage systems, a common method is to use replication by adding data redundancy. In contrast, erasure codes (ECs) offer better storage efficiency by performing linear operations in a finite Galois domain. As a widely used EC, Reed-Solomon (RS) codes divide the source file into k data blocks and generate m = nk parity blocks through linear operations in a finite Galois domain. These n blocks form a block group containing k data blocks and m parity blocks. Each block is further divided into multiple stripes and distributed to different storage nodes for fault tolerance. To repair a damaged storage node, it is only necessary to select k nodes from these n-1 available nodes to obtain k blocks, and then encode them. The k data blocks are represented as D = [d1,...,d...]. k ] T They are generated by an n×k generation matrix G = [g1,...,g n ] T Encode into n blocks C = [c1,...,c n ] TG consists of a k×k identity matrix I and a (nk)×k Vandermonde matrix or Cauchy matrix.

[0032] To describe the data flow of Aggregation Coded Repair (ACR), this invention introduces a Coded Path Tree (CPT) model, which is a tree diagram with m stages and m levels. The root node represents a replacement node, the leaf nodes represent source nodes, and the remaining nodes are aggregation nodes. When a source node also acts as an aggregation node, this invention uses two identical nodes to represent both the source and aggregation nodes.

[0033] Assume that in stage m, u m Each output data point is obtained by encoding the output data of the (m+1)th stage. The source data of the m-th stage is... It means that v m This represents the total number of source data in stage m. The intermediate encoding matrix, S... m It is a size u m ×(u m+1 +v m A matrix of ), where S represents m The elements in the i-th row and j-th column. The aggregation matrix at stage m is represented as:

[0034]

[0035] also, The following requirements must be met:

[0036]

[0037] After constructing the CPT for data repair, the ACR repair delay T can be derived. ACR To simplify the problem, assume the channel capacity C of each link. B The same. The repair delay of ACR can be expressed as:

[0038]

[0039] Here, M represents the number of aggregation stages, p * V represents the longest path in CPT, and e represents the path represented by each hop. aggr This represents the set of all aggregate nodes along the path. Data transfer time. Depends on the data block size |c| and the channel capacity C B . It is p * Propagation delay of each hop The sum of . It is each aggregation node The sum of the encoding calculation delays.

[0040] In ACR, the encoding / decoding process is chunk-based. That is, if chunk-based aggregation is implemented in ACR, the transmission time... It will accumulate over M stages. |c| is divided into s groups, called aggregation units. It can be converted into

[0041] When the aggregation unit is small, i.e. At that time, the transmission of ACR in the M stages can be approximately continuous. Meanwhile, because... The value is very small, and the encoding calculation delay is negligible. Therefore, equation (3) can be rewritten as follows:

[0042]

[0043] In addition, the energy cost of data repair is defined as cost. E The calculation formula is as follows:

[0044]

[0045] Where C B Let P represent channel capacity, B represent channel bandwidth, S represent signal power, and N represent noise power. t L represents the transmission power. f G represents path loss. r and G t Let $c$ represent the gain of the receiving and transmitting antennas, $l$ represent the distance of one hop, $f$ represent the carrier frequency, and $c$ represent the size of the data block. According to equation (5), $cost$... E It can be calculated using the following formula:

[0046]

[0047] This invention abstracts ACR into an encoding path tree (using variable Ω). CPT The algorithm for constructing the representation (in order to effectively reduce the repair time and energy cost of data restoration in Low Earth Orbit Satellite Storage Constellation (LSC) is based on the idea of ​​shortest path and minimum Steiner tree, such as... Figure 1 As shown, this invention proposes a low-latency aggregated coded data repair method for satellite storage networks, comprising the following steps:

[0048] Step 1: Select k source nodes with the shortest path from n-1 available data storage nodes;

[0049] Step 2, construct the encoding path tree (Ω) CPT );

[0050] Step 3: Use the encoded path tree (Ω) obtained in Step 2. CPT Repair the data in the Low Earth Orbit Satellite Storage Constellation (LSC).

[0051] This invention defines the network topology as Ω. Storage nodes that have failed to process data blocks are considered erase nodes v. era The node selected to reconstruct the lost data blocks is considered the replacement node v. rn A data storage node set containing n nodes is considered as v. sto The set of nodes from which k data storage nodes are selected from the available n-1 storage nodes is considered the source node set v. src .

[0052] The algorithm process is as follows:

[0053] Algorithm 1 Delay-sensitive Energy-efficient Aggregate Coding Repair (DE-ACR)

[0054] Input: Network topology Ω, k, substitute node v rn Storage node set v sto Erase nodes: v era

[0055]

[0056]

[0057] As can be seen from the algorithm process, this invention first proposes a method to select k source nodes with the shortest path from n-1 available data storage nodes, and then obtains the encoded path tree Ω through Algorithm 2. CPT Finally, data repair begins.

[0058] The following is a detailed explanation of each step.

[0059] Step 1: Select k source nodes with the shortest paths from n-1 available data storage nodes.

[0060] Step one specifically includes:

[0061] Step A1, use Dijkstra's algorithm to obtain the source node v src To the replacement node v rn Find the k shortest paths and store them in set P;

[0062] Step A2: Define the k shortest paths obtained in step A1 as the paths with the minimum propagation delay, and sort them in descending order of distance.

[0063] Step 2, construct the encoding path tree Ω CPT .

[0064] Step two specifically includes:

[0065] Step 1: Select the path with the largest propagation delay in set P, add it to the encoded path tree, and add its associated nodes to the node set V. CPT In, its delay is defined as T. ACR ;

[0066] Step 2: Process the remaining paths in set P in descending order of propagation delay. The goal is to reduce the energy cost of each path while ensuring that its propagation delay does not exceed the repair delay T. ACR Each path is evaluated to determine whether it can be added to the node set V. CPT To achieve lower energy costs while ensuring that the propagation delay does not exceed the repair delay T. ACR If such a path exists, update the path accordingly; otherwise, add the source node to V. left In the middle. When the loop ends, V is processed again. left If the path still does not meet the update conditions, then the path will not be updated.

[0067] Step 2 is described in detail below:

[0068] Step S1, determine the source node set v src Is it empty? If it is empty, it means the source node set v is empty. src Once the path processing is complete, it enters the node set V. left Path processing steps; otherwise, proceed to the next step.

[0069] Step S2, from the source node set v src Obtain the nearest node set V from the remaining source nodes. CPT The source node;

[0070] Step S3: Determine whether the propagation delay of the source node obtained in step S2 is greater than the repair delay T. ACR If so, add the source node to the node set V. left Then remove the source node from the source node set v src Remove, then return to step S1; otherwise, add the path to Ω. CPT And add the nodes visited to the node set V. CPT Then remove the node from the source node set v src Remove from the list and then return to step S1.

[0071] In step S1, the V left The specific steps for path processing are as follows:

[0072] Step S10, determine the node set V left Is it empty? If it is empty, it means the node set V is empty. left Once the path processing is complete, the required encoded path tree is obtained. Then, step three is executed: the obtained encoded path tree is used to repair the low-Earth orbit satellite storage constellation data. Otherwise, the next step is executed.

[0073] Step S11, from the node set V left Get the distance V from the remaining source nodes CPT The shortest path;

[0074] Step S12: Determine whether the propagation delay of the nearest path obtained in step S11 is greater than the repair delay T. ACR If so, add the original path from set P to the encoded path tree, and add the nodes traversed to the node set V. left Then return to step S10; otherwise, add the path to the encoded path tree and add the nodes it passes through to the node set V. CPT Then return to step S10.

[0075] Algorithm 2: The CPT construction algorithm for DE-ACR in this invention is as follows:

[0076] Input: Network topology Ω, source node set v src Replacement node v rn Path set P

[0077] Output: Ω CPT

[0078]

[0079]

[0080] Table 1 below shows the channel and antenna parameter configurations used in the experiment. This invention compares the simulation performance of DE-ACR with shortest path aggregation coding repair (SP-ACR), shortest path conventional repair (SP-CCR), and single repair pattern tree (SRPT).

[0081] Table 1 Simulation parameter settings

[0082]

[0083] This invention conducted two experiments using Starlink orbital data obtained from STK in an 8x8 LSC. In each experiment, 50 files experienced data block loss and required reconstruction; the results from these 50 experiments were averaged. For the first experiment, the number of stripes was fixed at 32. In the second experiment, the file size was fixed at 1024MB.

[0084] Figure 2(a) shows that, with a fixed block size and number of stripes, the repair latency of the CC-R scheme increases with increasing k, because the amount of data to be transmitted is k times the block size. However, the repair latency of the ACR scheme hardly changes with increasing k, because the traffic on each link is the same under ACR. The average repair latency of DSL-ACR is only 3.5% higher than the theoretically optimal solution SP-ACR, but 7.3% lower than SRPT. Furthermore, Figure 2(b) shows that DSL-ACR has a 6% lower energy cost than SRPT and a 55.7% lower energy cost than SP-ACR. Overall, with a fixed block size and number of stripes, DSL-ACR outperforms SRPT in both latency and energy cost. DSL-ACR's repair latency is only 3.5% higher than SP-ACR, while achieving a significant 55.7% reduction in energy cost.

[0085] Figure 3 shows that, with fixed file and block sizes, the repair latency actually decreases as k increases. This is because the traffic in each link remains constant, so the repair latency per stripe is essentially constant, but the number of stripes decreases, thus reducing the repair latency. In terms of energy cost, DSL-ACR remains the lowest of all options, averaging 6.2% lower than SRPT and 55.6% lower than SP-ACR.

[0086] Figures 2 and 3 show that DE-ACR has a lower energy cost than SP-ACR and SRPT. Theoretically, the repair latency should be the same as SP-ACR. However, in practice, the addition of aggregation nodes increases latency slightly due to data reception, encoding processes, and data transmission within the aggregation nodes. Therefore, the repair latency of DSL-ACR is slightly higher than SP-ACR, but lower than SRPT.

[0087] Because data center storage nodes (servers) are relatively close together, while LSC storage nodes (satellites) are far apart, the single-hop propagation latency can reach 2ms to 6ms. This high propagation latency could lead to node repair delays exceeding budget if traditional repair methods are used. Furthermore, radio waves attenuate with increasing propagation distance, and due to the limited battery life of satellite networks and the high cost of radio power, LSC requires a node repair method with low energy consumption and low repair latency. These requirements mean that data center node repair methods are not entirely suitable for LSC. Therefore, this invention addresses the node repair problem of future LSCs by proposing a novel node repair method to reduce energy consumption and repair latency during the node repair process.

[0088] The low-latency aggregation-coded data repair method for satellite nodes in LSC proposed in this invention includes the following innovations:

[0089] 1. A Coding Path Tree (CPT) model is proposed, which is a tree graph consisting of paths from the source node to the replacement node, used to describe the data flow of ACR. Based on CPT, this invention analyzes the expressions for the repair latency and energy cost of ACR in LSC.

[0090] 2. A "Delay-sensitive Energy-efficient ACR" (DE-ACR) scheme is proposed. It is based on a CPT construction algorithm and aims to reduce energy costs as much as possible while ensuring optimal repair delay by combining the ideas of shortest path and minimum Steiner tree.

[0091] The present invention also discloses a low-latency aggregated coded data repair system for satellite storage networks, comprising: a memory, a processor, and a computer program stored on the memory, wherein the computer program is configured to implement the steps of the low-latency aggregated coded data repair method of the present invention when invoked by the processor.

[0092] The beneficial effects of this invention are: the low-latency aggregated coding data repair method of this invention can reduce the energy consumption and repair latency during the LSC storage node repair process.

[0093] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A low-latency aggregated coded data repair method for satellite storage networks, characterized in that, Includes the following steps: Step 1, from n - Select from 1 available data storage node k The source node with the shortest path; Step 2: Construct the encoding path tree; Step 3: Use the encoded path tree obtained in Step 2 to repair the data in the Low Earth Orbit Satellite Storage Constellation (LSC). Step one includes: Step A1, obtain the source node To the alternative node v rn of k Find the shortest path and store it in a set. P middle; Step A2, take the data obtained in step A1 k The shortest path is defined as the path with the minimum propagation delay, and they are sorted in descending order of distance. Step two includes: Step 1, Select Set P The path with the longest propagation delay is placed into the encoded path tree, and its associated nodes are added to the node set. V CPT In the middle, its delay is defined as repair delay. T ACR ; Step 2: Process the sets in descending order of propagation delay. P The remaining path in; Step 2 specifically includes: Step S1: Determine the source node set v src Is it empty? If it is empty, it represents the set of source nodes. v src Once the path processing is complete, it enters the node set. V left Path processing steps; otherwise, proceed to the next step. Step S2, from the source node set v src Find the nearest set of nodes from the remaining source nodes. V CPT The source node; Step S3: Determine whether the propagation delay of the source node obtained in step S2 is greater than the repair delay. T ACR If so, add the source node to the node set. V left and remove the source node from the source node set. v src Remove the path from the tree and return to step S1; otherwise, add the path to the encoded path tree and add the nodes traversed to the node set. V CPT Then remove the node from the source node set. v src Remove from the list and return to step S1; In step S1, the V left The path processing steps specifically include: Step S10: Determine the node set V left Is it empty? If it is empty, it represents a set of nodes. V left Once the path processing is complete, the required encoded path tree is obtained. Then, step three is executed: the obtained encoded path tree is used to repair the low-Earth orbit satellite storage constellation data. Otherwise, the next step is executed. Step S11, from the node set V left Get the set of distances from the remaining source nodes V CPT The shortest path; Step S12: Determine whether the propagation delay of the nearest path obtained in step S11 is greater than the repair delay. T ACR If so, set P The original path is added to the encoded path tree, and the nodes traversed are added to the node set. V left Then return to step S10; otherwise, add the path to the encoded path tree and add the nodes it passes through to the node set. V CPT Then return to step S10.

2. The low-latency aggregated encoded data repair method according to claim 1, characterized in that, In step A1, Dijkstra's algorithm is used to obtain the source node. v src To the alternative node v rn of k Find the shortest path and store it in a set. P middle.

3. A low-latency aggregated coded data repair system for satellite storage networks, characterized in that, include: A memory, a processor, and a computer program stored on the memory, the computer program being configured to implement the steps of the low-latency aggregate-coded data repair method of any one of claims 1-2 when invoked by the processor.

Images