A method and system for file-based data synchronization under a complex network

Abstract

The application discloses a kind of complex network under file-based data synchronization method, comprising the following steps: in database by the method of data synchronization mark increases mark field, for marking the record of different data;Mark field is compressed after polling and / or query of data modification end and / or query, and is merged into a data synchronization file after encryption;Data synchronization file is sent to synchronization data receiving end after sorting processing in synchronization data sending end and with system synchronization server establishes data transmission channel;In data receiving end, the legality of data synchronization file is verified, obtains synchronization data description file and is updated to the synchronization data body file that obtains and is analyzed.The application realizes the cache and transmission of synchronization data, ensures that system can continuously provide safe and reliable data synchronization service under various extreme conditions, effectively improves the stability of system operation and the security of data under complex network.

Description

Technical Field

[0001] This invention relates to the field of file data synchronization methods, and in particular to a file-based data synchronization method and system for complex networks. Background Technology

[0002] Currently, conventional data synchronization technologies mostly adopt direct network connection, which completes data synchronization through network-based service interfaces. This method has high requirements for network stability, network bandwidth, and security. In complex network environments such as when wired networks cannot be directly connected, the network is unstable, the network bandwidth is limited, and internal and external networks are not connected, conventional data network synchronization technologies cannot meet the system's data synchronization needs due to problems such as the inability to directly connect to wired networks, network instability, network offline, and access restrictions.

[0003] Therefore, how to ensure secure, reliable, and efficient distributed data synchronization in complex network environments where wired networks cannot be directly connected, networks are unstable, network bandwidth is limited, and internal and external networks are not interconnected has become an urgent problem for those skilled in the art. Summary of the Invention

[0004] The purpose of this invention is to provide a file-based data synchronization method and system for complex networks, thereby solving the aforementioned problems existing in the prior art.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A file-based data synchronization method for complex networks includes the following steps:

[0007] S1. Add a tag field to the database using a data synchronization tagging method to tag records with different data;

[0008] S2. After the flag fields are polled and / or queried by the data modification end, they are compressed, encrypted, and then merged into a single data synchronization file.

[0009] S3. After the data synchronization file is sorted and processed at the synchronization data sending end, it is sent to the synchronization data receiving end and a data transmission channel is established with the system synchronization server.

[0010] S4. At the data receiving end, verify the validity of the data synchronization file, parse the obtained synchronization data description file, and update the obtained synchronization data body file.

[0011] As a further improvement of the present invention, the data synchronization tagging method specifically includes the following steps:

[0012] S101. Add a marker field A to the database design to mark whether a record is newly added or modified;

[0013] S102. In the database design, logical deletion is implemented for deletion operations, and a marker field B is added to mark that a record has been deleted;

[0014] S103. Add a marker field C to the database design to mark the importance of records;

[0015] S104. Add a tag field D to the database design to mark the real-time requirements of records.

[0016] As a further improvement of the present invention, the method for generating data synchronization files is specifically as follows:

[0017] S201. The data modification end periodically polls and / or triggers a query when the data changes, grouping the data that needs to be synchronized according to the importance and real-time marking fields;

[0018] S202. Sort the synchronized data retrieved from the grouping in step S201 by priority and put it into the queue to be processed.

[0019] S203. Create a synchronization data description file and write the data sender, data receiver, and data file version information into the description file.

[0020] S204. Create a synchronization data file and write the data in the data queue to be processed in step S202 into the synchronization file in sequence.

[0021] S205. The synchronization data description file and synchronization data body file created in steps S203 and S204 are compressed, encrypted, and then merged into a single data synchronization file.

[0022] As a further improvement of the present invention, the specific steps of step S3 are as follows:

[0023] S301. At the synchronous data sending end, the data synchronization files generated in S2 will be uniformly cached and managed, and sorted according to the importance and real-time nature of the synchronization files.

[0024] S302, the system synchronization server acts as a synchronization data mail server, responsible for receiving, caching, and sending data.

[0025] After receiving the synchronized file, the system first verifies the validity of the data file. If the synchronized data file is valid, the data is cached.

[0026] S303. Decrypt and decompress the synchronization data file to be processed received in step S302 according to the file encryption strategy, read the synchronization data description file, obtain information such as the sender and receiver of the data, classify and archive the synchronization data according to the system sending strategy, and wait to send it to the corresponding synchronization data receiving end.

[0027] S304. At the synchronous data receiving end, a data transmission channel is established with the system synchronization server, and the corresponding data is transmitted to the data receiving end.

[0028] As a further improvement of the present invention

[0029] S401, After receiving the data file sent by the system synchronization server, the data receiving end first performs a validity check on the data file. If the synchronization data file is valid, the data is cached and placed in the processing queue.

[0030] S402. Decrypt and decompress the synchronization data file to be processed received in step S401 according to the file encryption strategy to obtain the synchronization data description file and the synchronization data body file.

[0031] S403. Parse the information stored in the synchronization data description file obtained in step S402 to obtain the description information of the synchronization data body;

[0032] S404. Parse the description information of the synchronization data body file obtained in step S403, find and parse the corresponding even-numbered synchronization data body file;

[0033] S405. Parse the synchronization data file obtained in step S403 and update the synchronization information to the receiving end database.

[0034] A file-based data synchronization system for complex networks, including

[0035] The synchronization tagging module is used to synchronize and tag records of different data.

[0036] The synchronization file generation module is used to compress and encrypt the marked fields and then merge them into a single data synchronization file.

[0037] The synchronization information management module is used to perform unified cache management and sorting of the generated data synchronization files;

[0038] The synchronization data file parsing module is used to obtain the description information of the synchronization data body and update the synchronization information to the receiving end database.

[0039] The beneficial effects of this invention are:

[0040] This invention uses files as the transmission medium for synchronized data, eliminating the strong dependence of data synchronization on networks. Users can independently choose from various data transmission channels, such as wired networks, satellite networks, mobile communication operator networks, email networks, and mobile storage devices, to transmit synchronized data based on the actual situation of the system's network. Data encryption and compression ensure the security of synchronized data. It also provides distributed synchronization information management and data file parsing functions, enabling the caching and transmission of synchronized data. This ensures that the system can continuously provide secure and reliable data synchronization services under various extreme conditions, effectively improving the stability of system operation and data security in complex network environments. Attached Figure Description

[0041] Figure 1 This is a flowchart illustrating the data synchronization method of the present invention;

[0042] Figure 2 This is a schematic diagram of file-based data synchronization according to the present invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0044] A file-based data synchronization method for complex networks includes the following steps:

[0045] S1. Add a tag field to the database using a data synchronization tagging method to tag records with different data;

[0046] S2. After the flag fields are polled and / or queried by the data modification end, they are compressed, encrypted, and then merged into a single data synchronization file.

[0047] S3. After the data synchronization file is sorted and processed at the synchronization data sending end, it is sent to the synchronization data receiving end and a data transmission channel is established with the system synchronization server.

[0048] S4. At the data receiving end, verify the validity of the data synchronization file, parse the obtained synchronization data description file, and update the obtained synchronization data body file.

[0049] As a further improvement of the present invention, the data synchronization tagging method specifically includes the following steps:

[0050] S101. Add a marker field A to the database design to mark whether a record is newly added or modified;

[0051] S102. In the database design, logical deletion is implemented for deletion operations, and a marker field B is added to mark that a record has been deleted;

[0052] S103. Add a marker field C to the database design to mark the importance of records;

[0053] S104. Add a tag field D to the database design to mark the real-time requirements of records.

[0054] In this embodiment (A: critical, B: important, C: general). (1: real-time, 0: non-real-time).

[0055] As a further improvement of the present invention, the method for generating data synchronization files is specifically as follows:

[0056] S201. The data modification end periodically polls and / or triggers a query when the data changes, grouping the data that needs to be synchronized according to the importance and real-time marking fields;

[0057] S202. Sort the synchronized data retrieved from the grouping in step S201 by priority and put it into the queue to be processed.

[0058] S203. Create a synchronization data description file and write the data sender, data receiver, and data file version information into the description file.

[0059] S204. Create a synchronization data file and write the data in the data queue to be processed in step S202 into the synchronization file in sequence.

[0060] S205. The synchronization data description file and synchronization data body file created in steps S203 and S204 are compressed, encrypted, and then merged into a single data synchronization file.

[0061] As a further improvement of the present invention, the specific steps of step S3 are as follows:

[0062] S301. At the synchronous data sending end, the data synchronization files generated in S2 will be uniformly cached and managed, and sorted according to the importance and real-time nature of the synchronization files.

[0063] S302, the system synchronization server acts as a synchronization data mail server, responsible for receiving, caching, and sending data.

[0064] After receiving the synchronized file, the system first verifies the validity of the data file. If the synchronized data file is valid, the data is cached.

[0065] S303. Decrypt and decompress the synchronization data file to be processed received in step S302 according to the file encryption strategy, read the synchronization data description file, obtain information such as the sender and receiver of the data, classify and archive the synchronization data according to the system sending strategy, and wait to send it to the corresponding synchronization data receiving end.

[0066] S304. At the synchronous data receiving end, a data transmission channel is established with the system synchronization server to transmit the corresponding data to the data receiving end. This data transmission channel may include, but is not limited to, wired networks, satellite networks, mobile communication operator networks, email networks, and mobile storage devices or other carriers capable of holding data files. A suitable data transmission channel can be selected based on network conditions, security requirements, and other factors.

[0067] As a further improvement of the present invention

[0068] S401, After receiving the data file sent by the system synchronization server, the data receiving end first performs a validity check on the data file. If the synchronization data file is valid, the data is cached and placed in the processing queue.

[0069] S402. Decrypt and decompress the synchronization data file to be processed received in step S401 according to the file encryption strategy to obtain the synchronization data description file and the synchronization data body file.

[0070] S403. Parse the information stored in the synchronization data description file obtained in step S402 to obtain the description information of the synchronization data body;

[0071] S404. Parse the description information of the synchronization data body file obtained in step S403, find and parse the corresponding even-numbered synchronization data body file;

[0072] S405. Parse the synchronization data file obtained in step S403 and update the synchronization information to the receiving end database.

[0073] A file-based data synchronization system for complex networks, including

[0074] The synchronization tagging module is used to synchronize and tag records of different data.

[0075] The synchronization file generation module is used to compress and encrypt the marked fields and then merge them into a single data synchronization file.

[0076] The synchronization information management module is used to perform unified cache management and sorting of the generated data synchronization files;

[0077] The synchronization data file parsing module is used to obtain the description information of the synchronization data body and update the synchronization information to the receiving end database.

[0078] By adopting the above-disclosed technical solution of this invention, the following beneficial effects are obtained:

[0079] This invention uses files as the transmission medium for synchronized data, eliminating the strong dependence of data synchronization on networks. Users can independently choose from various data transmission channels, such as wired networks, satellite networks, mobile communication operator networks, email networks, and mobile storage devices, to transmit synchronized data based on the actual situation of the system's network. Data encryption and compression ensure the security of synchronized data. It also provides distributed synchronization information management and data file parsing functions, enabling the caching and transmission of synchronized data. This ensures that the system can continuously provide secure and reliable data synchronization services under various extreme conditions, effectively improving the stability of system operation and data security in complex network environments.

[0080] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for file-based data synchronization under complex network, characterized in that, include: The following steps: S1. Add a tag field to the database using a data synchronization tagging method to tag records with different data; S2. The marked field is compressed, encrypted, and merged into a single data synchronization file after being polled and / or queried by the data modification end; S3. The data synchronization file is sorted and processed at the synchronization data sending end and then sent to the synchronization data receiving end to establish a data transmission channel with the system synchronization server. S4. At the data receiving end, the validity of the data synchronization file is verified, the synchronization data description file is parsed, and the obtained synchronization data body file is updated. The data synchronization tagging method specifically includes the following steps: S101. Add a marker field A to the database design to mark whether a record is newly added or modified; S102. In the database design, logical deletion is performed for deletion operations, and a marker field B is added to mark that a record has been deleted; S103. Add a marker field C to the database design to mark the importance of records; S104. Add a tag field D to the database design to mark the real-time requirements of records; The method for generating the data synchronization file is as follows: S201. The data modification end periodically polls and / or triggers a query when the data changes, grouping the data that needs to be synchronized according to the importance and real-time marking fields; S202. Sort the synchronized data retrieved from step S201 by priority and place it into the pending queue. List; S203. Create the synchronization data description file, specifying the data sender, data receiver, and data file. The component version information is written to the description file; S204. Create the synchronization data file, and sequentially process the data in the queue to be processed in step S202. Data is written to the synchronization file; S205. The synchronization data description file and the synchronization data body created in steps S203 and S204 are... The files are compressed, encrypted, and then merged into a single data synchronization file. The specific steps of step S3 are as follows: S301. At the synchronous data sending end, the data synchronization file generated in S2 will be uniformly cached. Storage management, and sorting of the synchronized files according to their importance and real-time requirements; S302, the system synchronization server acts as a synchronization data mail server, responsible for receiving, caching, and sending data. After receiving the data synchronization file, the system first verifies the validity of the data synchronization file. If the data synchronization file is valid, the data is cached. S303. Take the data synchronization file to be processed received in step S302 and process it according to the file encryption policy. After decrypting and decompressing the file, the synchronous data description file is read to obtain the sender and receiver information of the data. The synchronous data is classified and archived according to the system sending strategy and waited to be sent to the corresponding synchronous data receiver. S304. At the synchronization data receiving end, establish a data transmission channel with the system synchronization server and transmit the corresponding... The data is transmitted to the data receiving end.

2. The file-based data synchronization method for complex networks according to claim 1, characterized in that, S401. After receiving the data synchronization file sent by the system synchronization server, the data receiving end first performs a validity check on the data synchronization file. If the data synchronization file is valid, the data is cached and placed in the processing queue. S402. Decrypt and decompress the data synchronization file to be processed received in step S401 according to the file encryption strategy to obtain the synchronization data description file and the synchronization data body file. S403. Parse the information stored in the synchronization data description file obtained in step S402 to obtain the description information of the synchronization data body file; S404. Parse the description information of the synchronization data body file obtained in step S403, find and parse the corresponding synchronization data body file; S405. Parse the synchronization data file obtained in step S403 and update the synchronization data file to the receiving end database.

3. A file-based data synchronization system under complex network, characterized in that, include: The synchronization tagging module is used to add tagging fields in the database through data synchronization tagging to tag records of different data; the tagging fields include tagging field A for tagging whether a record is newly added or modified, tagging field B for tagging whether a record is deleted, tagging field C for tagging the importance level of a record, and tagging field D for tagging the real-time requirements of a record; The synchronization file generation module is used to enable the data modification end to periodically poll and / or trigger queries when data changes. It groups the data to be synchronized according to the importance and real-time flag fields, sorts them by priority, and places them into a processing queue. It creates a synchronization data description file, writing the data sender, data receiver, and data file version information into the synchronization data description file. It creates a synchronization data body file, writing the data from the processing queue into the synchronization data body file sequentially. Finally, it compresses and encrypts the synchronization data description file and the synchronization data body file, merging them into a single data synchronization file. The synchronization information management module is used to uniformly cache and manage the data synchronization files at the synchronization data sending end, and sort the data synchronization files according to their importance and real-time requirements before sending them to the synchronization data receiving end. The synchronization information management module includes a system synchronization server, which acts as a synchronization data mail server and is responsible for receiving, caching and sending data. After the system receives the data synchronization file, it first verifies the validity of the data synchronization file. If it is valid, the data is cached. The received synchronization data file is decrypted and decompressed according to the file encryption policy, and then the synchronization data description file is read to obtain the sender information, receiver information, and data file version information. The synchronization data is then classified and archived according to the system sending policy, awaiting transmission to the corresponding synchronization data receiving end. At the synchronization data receiving end, a data transmission channel is established with the system synchronization server to transmit the corresponding data to the receiving end. The data transmission channel includes at least one of wired network, satellite network, mobile communication operator network, email network, and mobile storage device. The synchronization data file parsing module is used to perform legality verification on the data synchronization file at the data receiving end, obtain the synchronization data description file for parsing, and obtain the synchronization data body file for updating.

Images