A mobile data center data storage method based on a chain structure

Abstract

The application discloses a kind of mobile data center data storage methods based on chain structure.The steps of the method include: ① for each server of mobile data center, construct server chain, for storing the data and copy generated in the physical area where the server is located and parallelly executing the data processing task of the server;② store the original data and copy generated by the server on the corresponding server chain, and store the data fragments to the adjacent servers of server chain after being sharded;③ if a certain data fragment on the server is invalid, different ways are used to quickly recover historical data or task data;④ if a certain server is invalid, recover all data stored on it to other servers, and migrate the data processing task on it to other servers to continue execution, and try to reduce the impact of task migration on task execution time.The application can significantly improve the data storage security and data processing performance of mobile data center.

Description

A mobile data storage method based on a chain structure Technical Field

[0001] This invention relates to a chained storage method for secure and efficient data storage in mobile data centers, belonging to the field of mobile device data processing. Background Technology

[0002] Mobile data centers represent a current trend in data processing for large mobile devices. They connect various sensors, connectors, servers, and other data acquisition, transmission, storage, and computing devices deployed on mobile devices, managing and utilizing them in a data center format. This fully leverages the performance of these devices, improves data storage and processing performance and security, and provides data support for the intelligent control and secure operation of mobile devices. Traditional data centers use a centralized data storage method with a single server, where data and copies are stored and processed on the same server. This presents several risks: firstly, server failure renders all stored data unusable and terminates all processing; secondly, the sequential execution of all data processing tasks on the same server is inefficient. As mobile devices become more functional and intelligent, the types and amounts of data collected increase, as do the security and reliability of data storage, the scale and performance of data processing, and the intelligence and responsiveness of control decisions. Diverse data acquisition, driven by hardware devices such as sensors, is crucial for intelligent control. Data storage methods, determined by the user-defined mobile data center data storage strategy, are key factors influencing data storage security and processing performance. Therefore, it is necessary to research data storage methods for secure storage and efficient processing in mobile data centers to ensure the intelligent control and secure operation of mobile devices.

[0003] Traditional data center configurations primarily store raw data using local storage, with backups performed locally or off-site via dedicated buses at other physical locations on the device. Data is then restored from these local or off-site backups when the original data fails. This simple data storage approach focuses on data storage without considering the impact of data storage on data processing performance. Secure and efficient data storage methods are essential for traditional data centers. These methods are designed based on factors such as data usage frequency, server physical distance, and data storage costs. For example, infrequently used data can be stored as cold data on large-capacity, low-performance servers, while frequently used data can be stored on high-performance servers to reduce storage costs. Alternatively, multiple backups can be stored on local servers, adjacent servers, and the furthest server to balance data recovery costs and data security. Another approach is to comprehensively evaluate data storage costs based on data transmission latency, energy consumption, and resource utilization, and optimize data storage locations accordingly. However, these traditional data storage methods are unsuitable for mobile data centers. The main reasons are two fundamental characteristics of traditional data centers: firstly, the large number of servers leads to frequent node failures; and secondly, data transmission between servers requires multiple layers of switches, resulting in transmission costs far exceeding data processing costs. However, mobile data centers have fewer servers, resulting in a lower probability of data and node failures. Furthermore, all servers are connected via a dedicated high-speed bus, leading to extremely low data transmission costs. Therefore, data storage methods in mobile data centers must comprehensively consider the structural characteristics of the mobile data center, data storage security, and the impact on data processing performance.

[0004] Secure and efficient data storage methods in mobile data centers can utilize servers at different physical locations on mobile devices to store multiple copies of data, addressing data loss or server failures due to unforeseen circumstances and ensuring data security. Alternatively, they can leverage data copies across multiple servers for parallel processing, improving data processing performance and preventing task delays caused by excessive load on a single server, thus enhancing resource utilization in the mobile data center. The goal of these secure and efficient mobile data storage methods is to ensure the secure storage and efficient processing of mobile device data, thereby guaranteeing the safe operation and intelligent control of mobile devices. Summary of the Invention

[0005] This invention provides a mobile data center data storage method based on a chain structure. It aims to store data securely and efficiently according to the structural characteristics, number of replicas, and data processing performance of the mobile data center. It constructs a server chain by comprehensively considering the distance between servers, the number of replicas, and the data processing performance, and stores the original data and replicas on the chain. It also comprehensively considers the data migration cost, task migration cost, and task response time to recover failed data and handle failed servers, thereby ensuring the secure storage and efficient processing of mobile device data.

[0006] The technical solution adopted in this invention is as follows:

[0007] A mobile data storage method based on a chain structure specifically includes the following steps:

[0008] (1) Constructing a server chain: Based on the physical characteristics and data processing performance requirements of the mobile data center, a server chain is constructed for each server in the mobile data center to store the data generated in the physical area where the server is located and to store copies of the data and to execute the data processing tasks of the server in parallel.

[0009] (2) Data and replica storage: The original data and replicas generated by each server are stored on the corresponding server chain. The data is then fragmented and stored sequentially on the adjacent servers in the server chain in a staggered manner.

[0010] (3) Data failure handling: If a data fragment on the server fails during the operation of the mobile device, different methods are used to restore the failed data fragment in a timely manner, depending on whether the failed data fragment is historical data or task data.

[0011] (4) Server failure handling: If a server fails due to a fault during the operation of the mobile device, all data on the server will be restored to other servers to ensure that the number of data copies remains unchanged. The tasks currently being executed on the server will be migrated to other servers to continue execution, and the impact of task migration on task execution time will be minimized.

[0012] The beneficial effects of this invention are as follows:

[0013] (1) Each server is connected to several servers that are furthest away to form a server chain, and data copies are sliced ​​and stored on the server chain to reduce the probability of simultaneous server failure and ensure data storage security.

[0014] (2) By utilizing data replicas on the server chain, parallel processing of data can be achieved, and tasks can be quickly migrated to achieve a new load balance when the server fails, thereby improving data processing performance.

[0015] (3) Different data recovery mechanisms are adopted according to the type of failed data. Historical data is recovered as soon as possible, and task data is processed as quickly as possible. This ensures both the safe storage of data and the performance of data processing.

[0016] (4) The chain structure built based on physical distance greatly reduces the probability of all copies of data failing at the same time, and the chain storage can also cope with the failure of multiple servers in a row, improving the availability and security of data. Attached Figure Description

[0017] Figure 1 shows the network architecture of the mobile data center data storage method based on the chain structure of the present invention.

[0018] Figure 2 illustrates the specific flow of the mobile data center data storage method based on a chain structure according to the present invention.

[0019] Figure 3 illustrates the resource allocation prediction process according to an embodiment of the present invention. Detailed Implementation

[0020] The technical solution of the present invention will be further described below with reference to the embodiments.

[0021] The data storage method proposed in this invention is mainly used in mobile data centers on large mobile devices. It securely and efficiently stores various types of data generated by mobile devices, ensuring secure data storage, efficient processing, rapid data recovery in the event of data or server failures, and rapid response to data processing tasks. Because the physical structure and application requirements of mobile data centers differ significantly from traditional data centers, they are characterized by low data transmission costs between servers, low probability of data and server failures, and low data processing latency. Therefore, data storage must comprehensively consider factors such as the structural characteristics of the mobile data center, the number of data copies, and data processing performance to improve data storage and processing efficiency.

[0022] In a mobile data center, any two servers are connected via a dedicated high-speed bus, with the transmission distance between services equal to the bus length. The proposed chain-based mobile data center data storage method constructs a server chain for each server and uses servers on the chain for replica storage, parallel data processing, and failed data recovery. The network structure of the chain-based storage is shown in Figure 1, and the data storage process is shown in Figure 2. This invention prioritizes data storage security and data processing performance, implementing a chain-based storage structure tailored to the physical connection characteristics of mobile data center servers to improve data storage security and data processing performance on mobile devices. The specific data storage process includes the following four steps:

[0023] 1. Constructing a server chain: Based on the physical characteristics and data processing performance requirements of the mobile data center, construct a server chain for each server in the mobile data center. The specific process is as follows:

[0024] ① Information Collection: Collect all server configurations of the mobile device, P = {P1, P2, ..., P}. n}, where the i-th server P i Information includes location loc i and any other server P j distance d i,j The distances between all servers form a distance matrix D. n×n ={d i,j}, where d i,j ≥0, d i,j =d j,i d i,i =0, 1≤i≠j≤n;

[0025] ② Construct a server chain: for each server P i Build a server chain CP i , will server P i and distance P i The largest k-1 servers are added to the server chain CP. i In the middle, it is used to generate data D for the server's coverage area. i Backup storage and parallel processing improve data storage reliability and data processing performance, where k is the parallelism of data processing, 1≤k≤n, 1≤i≤n.

[0026] 2. Data and Replica Storage: The original data and replicas generated by each server are stored on the corresponding server chain. The specific process is as follows:

[0027] ① Data sharding: Sharding each server P i Dataset D generated by the coverage area i Divide horizontally into k segments {D} i,1 D i,2 ,…,D i,k}, when dealing with dataset D i When performing parallel processing, each shard can be assigned to a server for processing, improving data processing performance, where D i =D i,1 ∪D i,2 ∪…∪D i,k D i,a ∪D i,b =Φ, 1≤i≤n, 1≤a≠b≤k;

[0028] ② Data storage: Original data D i Stored on server Pi Above, data sharding D i,a The original data is recorded as If each data segment is D i,a Save r copies Stored on server P i Corresponding server chain CP i The specific storage method is as follows: The server chain CP... i The k-server in the code is renumbered as {P′1, P′2, ..., P′} k}, divide each data into D pieces i,a The first copy Stored in CP i The a-th server P′ a Up; then each data shard D i,a The j-th copy Stored in CP i On a certain server, if aj≥1, then the replica... Stored in CP i The ajth server Up, otherwise copy Stored in CP i The (a-j+k)th server Above, where P′1=P i , 1≤i≤n, 1≤r≤k, 1≤a,j≤k;

[0029] ③ Data processing: If a certain server P i Data D needs to be processed i Then the data processing operation is sent to the server chain CP. i Parallel execution, CP i The first server P′ on each server a a Utilize idle resources to process the a-th data fragment D i,a , where 1≤i≤n, 1≤a≤k.

[0030] 3. Data Failure Handling: If the server P fails to process data during mobile device operation... i If a data shard fails, the failed data should be recovered promptly. The specific steps are as follows:

[0031] ① Historical data expires: When server P i Failed data fragments When dealing with historical data, expired data should be recovered as soon as possible, especially if the data is fragmented. For server P i The raw data fragments generated on the server P are then processed on the server P. i Replicate one copy of the data shard. Replace invalid data fragments Where l = 0, 1 ≤ i = x ≤ n, 1 ≤ y ≤ k, 1 ≤ t ≤ r; if the data fragment fails... Shard the raw data The copy is obtained from the original data shards. and its copies Select the distance from server P in the storage location. i Recent Dungeons And Copy to server P i Replace the invalid data fragment Where 1≤l≤r, 1≤i,x≤n, 1≤y≤k, 0≤t≤r and t≠l;

[0032] ②Task data invalidation: When server P i Failed data fragments When dealing with data pending processing in the current task, the handling of failed data fragments should be completed as soon as possible. If data fragments... For server P i The raw data fragments generated on the server P are then processed on the server P. i Replicate one copy of the data shard. Replace invalid data fragments Where l = 0, 1 ≤ i = x ≤ n, 1 ≤ y ≤ k, 1 ≤ t ≤ r; if the data fragment fails... Shard the raw data The copy is obtained from the original data shards. and its copies Select the server P′ with the most currently available data processing resources from the storage location. s and distance from server P i Recent Dungeons If server P′ s Available data processing resources are greater than P i If the available data processing resources are used, then P will be used. i Data sharding The processing is transferred to P′ s The process is executed and the results and data are sharded. Transmitted to server P i Otherwise, shard the data. Copy to server P i Replace the invalid data fragment And on server P i Continue processing of invalid data, where 1≤l≤r, 1≤i,x≤n, 1≤y≤k, 0≤t,s≤r and t≠l,t≠l.

[0033] 4. Server Failure Handling: When a mobile device is running, if a server P fails... i After a failure, the server P should be... i All data on the server is restored, and the currently executing task is migrated to another server to continue execution. The specific process is as follows:

[0034] ① Data recovery: For failed server P i Each data shard on Re-back up on other normal services, with the backup server located at a distance from P. i Farthest and excluding The replica server P j The backup data source includes The copy is located at a distance of P from the server. j The nearest server, where 1≤l≤r, 1≤i,j,x≤n, 1≤y≤k;

[0035] ②Task recovery: If server P i In processing data sharding If the task fails, it will be migrated to another service for execution, and load balancing will be performed during the task execution process to ensure the lowest possible processing time: first, based on data sharding... Determine the original data D x The server P is located on x and server chain CP x Determine the faulty server P i In the server chain CP x Position P′ in j That is, P i =P′ j The failed server P i Data sharding The processing has been migrated to Server P′ where any replica resides s Execute on server P′; s The original data processing task retains 1 / (k-1) of the data, and the remaining (k-2) / (k-1) portion is migrated to the data fragment to be processed. The server where the copy is located is P′ s-1 Execution is performed on server P′ at this time. s The data to be processed is With 1 / (k-1) Its load is k / (k-1) times the original; server P′ s-1 Processing the original P′ s Responsible for data sharding The (k-2) / (k-1) part, and P′ s-1 Originally in charge of data sharding The 2 / (k-1) portion is then partitioned into data pieces. The (k-3) / (k-1) portion is migrated to server P′ s-2 The above process is repeated until all data has been processed. At this point, the load on each server is k / (k-1) times that before the server failed, and the data processing load of each server is balanced, where 1≤l≤r, 1≤i,x≤n, and 1≤y,s≤k.

[0036] ③ Multiple task interruptions: If server failure causes multiple data processing tasks to be interrupted, follow step ② to migrate each data processing task in turn.

[0037] ④ Server chain update: Update the failed server P i Update all relevant server chains, if P i Data sharding on Re-backup to server P j Above, in data sharding The corresponding original data D x Generate server P x server chain CP x Lieutenant General Server P i Replace with P x Where 1≤l≤r, 1≤i,j,x≤n, and 1≤y≤k;

[0038] ⑤ Multiple Server Failures: If multiple servers fail simultaneously while the mobile device is running, proceed with steps ① to ④ for each server in sequence; if a server chain fails... i If r consecutive servers fail, the task execution cannot be restored using this method. However, since the servers selected during the construction of the server chain are all physically farthest, the probability of r consecutive servers failing at the same time is extremely low, where 1≤i≤n.

[0039] The data storage process in this embodiment is shown in Figure 3. Specifically, if a mobile data center of a mobile device contains 4 servers {P1, P2, P3, P4}, with 2 replicas, a data processing parallelism of 3, and a transmission distance between servers of {(P1, P2): 10, (P1, P3): 15, (P1, P4): 20, (P2, P3): 15, (P2, P4): 20, (P3, P4): 25}, the data storage process of the mobile data center of this mobile device is as follows:

[0040] ① Construct a server chain for each server: CP1 = {P1, P3, P4}, CP2 = {P2, P3, P4}, CP3 = {P3, P1, P4}, CP4 = {P4, P2, P3};

[0041] ② If servers P1, P2, P3, and P4 generate datasets D1, D2, D3, and D4 respectively, and each dataset is horizontally divided into three data partitions {D... 1,1 D 1,2 D 1,3}、{D 2,1 D 2,2 D 2,3}、{D 3,1 D 3,2 D 3,3} and {D 4,1 D 4,2 D 4,3 Each data shard has one original copy stored on the server that generated it, and two copies stored on the corresponding server chains CP1, CP2, CP3, and CP4. The final data storage configuration for each server is as follows: and

[0042] ③ If server P1 needs to process dataset D1 at a certain time, then the three servers in the server chain CP1 = {P1, P3, P4} will process D1 in parallel. Servers P1, P3, and P4 will process data shards respectively. and

[0043] ④ If server P4 is processing data sharding Due to a malfunction during the process Failure, at this time the failed data fragment The original data D 1,3 And another copy The data shards are stored on servers P1 and P3 respectively. Servers P1 and P4 are the closest. If, at this time, server P4 has the most available data processing resources among servers P1, P3, and P4, then data shard D is copied from server P1. 1,3 Recover data fragments on P4 Data sharding The processing continues on server P4;

[0044] ⑤ If at some point the data stored on server P2 is fragmented The data is invalid; this data fragment is historical data, and its original data D... 4,3 And another copy The data shards are stored on servers P4 and P3 respectively. Server P3 is the closest to P2, so data shards are copied from server P3. Data fragments recovered to P2

[0045] ⑥ If server P1 is processing data fragmentation Due to a failure during the process, all data on P1 should first be restored to other servers: Data shards D 1,1 and Available replicas are stored on server P4. Server P3 contains no replicas and is the furthest from server P1. Therefore, data shards are replicated from server P4. To restore data fragments D on server P3 1,1 and Copy data shards from server P3 To recover data fragments D from P4 1,2 and Copy data shards from server P3 To recover data fragments D from P2 1,3 Copy data shards from server P3 Data fragments recovered to P4 Copy data shards D from server P3 3,3 Data fragments recovered to P2 After recovery, the data storage status of servers P2, P3, and P4 is as follows: and Modify the server chain CP3 of server P3 to CP3 = {P3, P2, P4};

[0046] ⑦ Migrate the tasks being processed on the failed server P1 to other servers for execution: This will involve data sharding. The processing was migrated to server P4, and the server sharded the data. Half of the processing tasks were migrated to server P3 for processing, and server P3 processed the original data fragments. In addition, data sharding will be added. Half of the processing tasks are handled by the servers. After the task migration, servers P3 and P4 are each responsible for processing 1.5 data shards, which can ensure that the data processing tasks of dataset D1 are executed normally. During the task migration, no data migration is required and the servers responsible for data task processing are load balanced to ensure the shortest task processing time.

[0047] The embodiments described herein are merely exemplary implementations. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are provided to fully and completely disclose the invention and to fully convey the scope of the invention to those skilled in the art. The terminology used in the exemplary embodiments illustrated in the accompanying drawings is not intended to limit the invention. In the accompanying drawings, the same units / elements are referred to by the same reference numerals.

[0048] Unless otherwise stated, the terms used herein (including technical terms) have the common understanding meaning to those skilled in the art; in addition, it is understood that terms defined by commonly used dictionaries should be understood to have a meaning consistent with the context of their relevant field, and should not be understood to have an idealized or overly formal meaning.

Claims

1. A mobile data center data storage method based on a chain structure, characterized in that, The method specifically includes the following steps: (1) Constructing a server chain: Based on the physical characteristics and data processing performance requirements of the mobile data center, a server chain is constructed for each server in the mobile data center to store the data generated in the physical area where the server is located and the storage of copies, as well as to execute the data processing tasks of the server in parallel. For each server P i Build a server chain CP i , will server P i and distance P i The largest k-1 servers are added to the server chain CP. i In the middle, it is used to generate data D for the server's coverage area. i Backup storage and parallel processing improve data storage reliability and data processing performance, where k is the parallelism of data processing, 1≤k≤n, 1≤i≤n; (2) Data and replica storage: store the original data and replicas generated by each server on the corresponding server chain, and then store the data in a staggered manner on the adjacent servers of the server chain after data fragmentation; original data D i Stored on server P i Above, data sharding D i,a The original data is recorded as If each data segment D i,a Save r copies Stored on server P i Corresponding server chain CP i The specific storage method is as follows: The server chain CP... i The k server in the middle is renumbered as Divide each data into D pieces i,a The first copy Stored in CP i The a-th server Up; then each data shard D i,a The j-th copy Stored in CP i On a certain server, if aj≥1, then the replica... Stored in CP i The ajth server Up, otherwise copy Stored in CP i The (a-j+k)th server Above, among which , 1≤i≤n, 1≤r≤k, 1≤a,j≤k; (3) Data failure handling: If a data fragment on the server fails during the operation of the mobile device, the failed data fragment is restored in a timely manner according to different methods depending on whether the failed data fragment is historical data or task data; (4) Server failure handling: If a server fails due to a fault during the operation of the mobile device, all data on the server is restored to other servers to ensure that the number of data copies remains unchanged, the task currently being executed on the server is migrated to other servers to continue execution, and the impact of task migration on task execution time is reduced.

2. The mobile data center data storage method based on a chain structure according to claim 1, characterized in that, The specific steps of step (1) are as follows: ① Information collection: collect all server information configured on the mobile device, including the number of servers, their location, and the data transmission distance between any two servers; ② Construct a server chain: construct a server chain for each server, and add each server and several servers with the longest transmission distance to the server to the server chain for storing and parallel processing of data and backups generated in the area covered by the server, thereby improving data storage reliability and data processing performance.

3. The mobile data storage method based on a chain structure according to claim 1, characterized in that, The specific steps of step (2) are as follows: ① Data sharding: The dataset generated by each server's coverage area is horizontally divided into multiple data shards. When the dataset is processed in parallel, the data shards are sequentially distributed to each server in the server chain for processing, thereby improving data processing performance; ② Data storage: The original data on each server is stored locally. Each first copy of the data shard is sequentially stored on each server in the server chain. Multiple copies of the same data shard are sequentially stored on adjacent servers in the server chain; ③ Data processing: If a server needs to process a dataset, it sends the data processing operation to its corresponding server chain and uses all servers in the server chain for parallel processing. Each server processes one data shard.

4. The mobile data storage method based on a chain structure according to claim 1, characterized in that, The specific steps of step (3) are as follows: ① Historical data failure: When the failed data fragment on the server is historical data, if the failed data fragment is the original data generated by the server, then the copy stored on the server is used for recovery; otherwise, the copy with the closest storage location to the server is selected from the original data of the failed data fragment and other copies for recovery, ensuring rapid data recovery; ② Task data failure: When the failed data fragment on the server is the data to be processed by the current task, if the failed data fragment is the original data generated by server P, then the copy stored on the server is used for recovery; otherwise, the server with the most available data processing resources and the copy with the closest storage location to the server are selected from the original data of the failed data fragment and all copies. The data processing task is sent to the server with the most data processing resources for processing and the result is sent back to the server. The data fragment is copied from the copy closest to the server for data recovery, ensuring that the processing of the failed data fragment is completed as soon as possible.

5. The mobile data storage method based on a chain structure according to claim 1, characterized in that, The specific steps of step (4) are as follows: ① Data recovery: For each data fragment on the failed server, back it up again on other normal services. The location of the backup server is the server that is furthest from the failed server and does not contain the lost copy of the data fragment. The backup data source is the server that contains the copy of the failed data fragment and is closest to the backup server. ② Task Recovery: If a server failure occurs while processing a data shard, determine the server containing the original data of the data shard to be processed and its corresponding server chain, as well as the position of the failed server in the server chain. Migrate the data processing task to any other server containing a copy of the data shard. Then, retain a portion of the original data processing task on the target server and migrate the rest to adjacent servers. Repeat this process for each normal server in the server chain, migrating portions of the original processing tasks to adjacent servers to ensure balanced data processing load and minimized task processing time after migration. ③ Multiple Task Interruptions: If a server failure causes multiple data processing tasks to be interrupted, use step ② to migrate each data processing task sequentially. ④ Server Chain Update: Update all server chains related to the failed server. If a data shard on the failed server is backed up again to the target server, replace that server with the target server in the server chain corresponding to the server that generated the original data of that data shard. ⑤ Multiple Server Failures: If multiple servers fail simultaneously while the mobile device is running, use steps ① to ④ sequentially for each server. If the number of consecutively failed servers in a server chain is less than the number of data copies, task execution can be recovered using this method.

6. A mobile data center data storage method based on a chain structure according to any one of claims 1 to 5, characterized in that, The method specifically includes the following steps: (1) Constructing a server chain: Based on the physical characteristics and data processing performance requirements of the mobile data center, a server chain is constructed for each server in the mobile data center. The specific process is as follows: ① Information collection: Collect all servers configured on the mobile device P={P1,P2,…P n }, where the i-th server P i Information includes location loc i and any other server P j distance d i,j The distances between all servers form a distance matrix D. n×n ={d i,j }, where d i,j ≥0, d i,j =d j,i d i,i =0, 1≤i≠j≤n; ② Construct a server chain: for each server P i Build a server chain CP i , will server P i and distance P i The largest k-1 servers are added to the server chain CP. i In the middle, it is used to generate data D for the server's coverage area. i Backup storage and parallel processing improve data storage reliability and data processing performance, where k is the parallelism of data processing, 1≤k≤n, 1≤i≤n; (2) Data and replica storage: Store the original data and replicas generated by each server on the corresponding server chain. The specific process is as follows: ① Data sharding: shard each server P i Dataset D generated by the coverage area i Divide horizontally into k segments {D} i,1 D i,2 ,…,D i,k }, when dealing with dataset D i When performing parallel processing, each shard can be assigned to a server for processing, improving data processing performance, where D i = D i,1 ∪D i,2 ∪…∪D i,k D i,a ∪D i,b =Φ, 1≤i≤n, 1≤a≠b≤k; ② Data storage: Original data D i Stored on server P i Above, data sharding D i,a The original data is recorded as If each data segment D i,a Save r copies Stored on server P i Corresponding server chain CP i The specific storage method is as follows: The server chain CP... i The k server in the middle is renumbered as Divide each data into D pieces i,a The first copy Stored in CP i The a-th server Up; then each data shard D i,a The j-th copy Stored in CP i On a certain server, if aj≥1, then the replica... Stored in CP i The ajth server Up, otherwise copy Stored in CP i The (a-j+k)th server Above, among which 1≤i≤n, 1≤r≤k, 1≤a,j≤k; ③ Data processing: If a certain server P i Data D needs to be processed i Then the data processing operation is sent to the server chain CP. i Parallel execution, CP i The first server on each of the a servers Utilize idle resources to process the a-th data fragment D i,a , where 1≤i≤n, 1≤a≤k; (3) Data failure handling: if the server P is running during the operation of the mobile device i If a data shard fails, the failed data should be recovered promptly. The specific steps are as follows: ① Historical data failure: When server P... i Failed data fragments When dealing with historical data, expired data should be recovered as soon as possible, especially if the data is fragmented. For server P i The raw data fragments generated on the server P are then processed on the server P. i Replicate one copy of the data shard. Replace invalid data fragments Where l=0, 1≤i=x≤n, 1≤y≤k, 1≤t≤r; if a data fragment fails Shard the raw data The copy is obtained from the original data shards. and its copies Select the distance from server P in the storage location. i Recent Dungeons and will Copy to server P i Replace the invalid data fragment ① Where 1≤l≤r, 1≤i, x≤n, 1≤y≤k, 0≤t≤r and t≠l; ② Task data invalidation: When server P i Failed data fragments When dealing with data pending processing in the current task, the handling of failed data fragments should be completed as soon as possible. If data fragments... For server P i The raw data fragments generated on the server P are then processed on the server P. i Replicate one copy of the data shard. Replace invalid data fragments Where l=0, 1≤i=x≤n, 1≤y≤k, 1≤t≤r; if a data fragment fails Shard the raw data The copy is obtained from the original data shards. and its copies Select the server with the most currently available data processing resources from the storage location. and distance from server P i Recent Dungeons If the server Available data processing resources are greater than P i If the available data processing resources are used, then P will be used. i Data sharding The processing has been migrated to The process is executed and the results and data are sharded. Transmitted to server P i Otherwise, shard the data. Copy to server P i Replace the invalid data fragment And on server P i Continue processing of failed data, where 1≤l≤r, 1≤i, x≤n, 1≤y≤k, 0≤t, s≤r and t≠l, t≠l; (4) Server failure processing: when a mobile device is running, a certain server P i After a failure, the server P should be... i All data on the failed server P is restored, and the currently executing tasks are migrated to other servers to continue execution. The specific process is as follows: ① Data recovery: For the failed server P i Each data shard on Perform a backup on other normal services, with the backup server located at a distance from P. i Farthest and excluding The replica server P j The backup data source includes The copy is located at a distance of P from the server. j The nearest server, where 1≤l≤r, 1≤i, j, x≤n, 1≤y≤k; ② Task recovery: If server P i In processing data sharding If the task fails, it will be migrated to another service for execution, and load balancing will be performed during the task execution process to ensure the lowest possible processing time: first, based on data sharding... Determine the original data The server P is located on x and server chain CP x Determine the faulty server P i In the server chain CP x The position in the middle ,Right now ； The failed server P i Data sharding The processing has been migrated to The server where any copy is located Execute on the server; The original data processing task retains 1 / (k-1) of the data, and the remaining (k-2) / (k-1) portion is migrated to the data fragment to be processed. The server where the copy is located Execute on the server at this time The data to be processed is With 1 / (k-1) Its load is k / (k-1) times the original; server Processing the original Responsible for data sharding The (k-2) / (k-1) part, and Originally in charge of data sharding The 2 / (k-1) portion is then partitioned into data pieces. The (k-3) / (k-1) portion was migrated to the server. The above process is repeated until all data has been processed. At this point, the load on each server is k / (k-1) times that before the server failure, and the data processing load is balanced across all servers, where 1≤l≤r, 1≤i, x≤n, 1≤y, s≤k; ③ Multiple task interruptions: If the server failure causes multiple data processing tasks to be interrupted, step ② is used to migrate each data processing task sequentially; ④ Server chain update: For the failed server P... i Update all relevant server chains, if P i Data sharding on Re-backup to server P j Above, in data sharding The corresponding original data D x Generate server P x server chain CP x Lieutenant General Server P i Replace with P x Where 1≤l≤r, 1≤i, j, x≤n, 1≤y≤k; ⑤ Multiple server failures: If multiple servers fail simultaneously while the mobile device is running, steps ①~④ are performed sequentially for each server; if a server chain CP i If r consecutive servers fail, the task execution cannot be restored using this method. However, since the servers selected during the construction of the server chain are all physically farthest, the probability of r consecutive servers failing at the same time is extremely low, where 1≤i≤n.

7. A mobile data center data storage method based on a chain structure according to claim 6, characterized in that, The specific data storage process is as follows: If a mobile data center of a mobile device contains 4 servers {P1,P2,P3,P4}, with 2 replicas, a data processing parallelism of 3, and a transmission distance between servers of {(P1,P2):10, (P1,P3):15,(P1,P4):20, (P2,P3):15, (P2,P4):20, (P3,P4):25}, the data storage process of the mobile data center of this mobile device is as follows: ① Construct a server chain for each server, CP1={P1,P3,P4}, CP2={P2,P3,P4}, CP3={P3,P1,P4}, CP4={P4,P2,P3}; ② If servers P1, P2, P3, and P4 generate datasets D1, D2, D3, and D4 respectively, each dataset is horizontally divided into three data shards {D 1,1 D 1,2 D 1,3 }、{D 2,1 D 2,2 D 2,3 }、{D 3,1 D 3,2 D 3,3 } and {D 4,1 D 4,2 D 4,3 Each data shard has one original copy stored on the server that generated it, and two copies stored on the corresponding server chains CP1, CP2, CP3, and CP4. The final data storage configuration for each server is as follows: 、 、 and ③ If server P1 needs to process dataset D1 at a certain moment, then the three servers in the server chain CP1={P1,P3,P4} will process D1 in parallel. Servers P1, P3, and P4 will process data shards respectively. 、 and ④ If server P4 is processing data fragmentation Due to a malfunction during the process Failure, at this time the failed data fragment raw data And another copy The data shards are stored on servers P1 and P3 respectively, with servers P1 being the closest to P4. If, at this point, server P4 has the most available data processing resources among servers P1, P3, and P4, then the data shards are copied from server P1. Recover data fragments on P4 Data sharding The processing continues on server P4; ⑤ If at some point the data fragments stored on server P2 are... The data is invalid; this data fragment is historical data, and its original data... And another copy The data shards are stored on servers P4 and P3 respectively. Server P3 is the closest to P2, so data shards are copied from server P3. Data fragments recovered to P2 ； ⑥ If server P1 is processing data fragmentation Due to a failure during the process, all data on P1 should first be restored to other servers: Data shards D 1,1 and Available replicas are stored on server P4. Server P3 contains no replicas and is the furthest from server P1. Therefore, data shards are replicated from server P4. To restore data fragments D on server P3 1,1 and Copy data shards from server P3 To recover data fragments D from P4 1,2 and Copy data shards from server P3 To recover data fragments D from P2 1,3 Copy data shards from server P3 Data fragments recovered to P4 Copy data shards D from server P3 3,3 Data fragments recovered to P2 ； After recovery, the data storage status of servers P2, P3, and P4 is as follows: 、 and ; Modify the server chain CP3 of server P3 to CP3={P3,P2,P4}; ⑦ Migrate the tasks being processed on the failed server P1 to other servers for execution: This involves data sharding. The processing was migrated to server P4, and the server sharded the data. Half of the processing tasks were migrated to server P3 for processing, and server P3 processed the original data fragments. In addition, data sharding will be added. Half of the processing tasks; After the task migration, servers P3 and P4 are each responsible for processing 1.5 data shards, which can ensure that the data processing tasks of dataset D1 are executed normally. During the task migration, no data migration is required and the servers responsible for data task processing are load balanced to ensure the shortest task processing time.

Images