Remote data synchronization method, device, equipment and storage medium

Abstract

The present invention relates to data processing technology and discloses a remote data synchronization method, comprising: extracting source data of a preset type from a first data center to obtain a target file; utilizing a pre-built traversal service to extract the file content and file structure in the target file; compressing the text content to obtain a compressed file, and encoding the text content using a message digest algorithm to obtain an encryption tag; utilizing an access token pre-generated by a second data center to transmit the encryption tag, the compressed file, and the text structure to the second data center. The present invention also proposes a remote data synchronization device, an electronic device, and a storage medium. The present invention can reduce data latency during remote information synchronization.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a remote data synchronization method, device, electronic device and computer-readable storage medium. Background Art

[0002] To increase data disaster resilience, many Internet companies are building data centers in countries around the world to synchronize and store important information. However, these data centers are connected via wireless signals and are located far away. Current methods mainly rely on increasing the bandwidth of information transmission and transmitting information in segments.

[0003] However, the technological changes in bandwidth still cannot keep up with the explosive growth of data volume. Regardless of whether the bandwidth is occupied in real time or sent in segments, after the source table data in one data center is added in batches, it will not be possible to start the transmission to another data center immediately, resulting in data delays. Summary of the Invention

[0004] The present invention provides a remote data synchronization method, device and computer-readable storage medium, the main purpose of which is to solve the problem of high data delay when performing remote information synchronization.

[0005] To achieve the above object, the present invention provides a remote data synchronization method, comprising:

[0006] Extracting source data of a preset type from the first data center to obtain a target file;

[0007] Utilize pre-built traversal services to extract the file content and file structure of the target file;

[0008] Compressing the text content to obtain a compressed file, and encoding the text content using a message digest algorithm to obtain an encryption tag;

[0009] The encrypted tag, the compressed file, and the text structure are transmitted to the second data center using an access token pre-generated by the second data center.

[0010] Optionally, extracting source data of a preset type from the first data center to obtain a target file includes:

[0011] Using a preset monitoring service to monitor data updates in the first data center in real time;

[0012] If the first data center generates a new data file, the new data file is automatically extracted to obtain the target file.

[0013] Optionally, the extracting the file content and file structure of the target file by using a pre-built traversal service includes:

[0014] Using the pre-built traversal service, traverse and access each sub-file in the target file to obtain the file content;

[0015] Analyze the file type of each sub-file to obtain a file type set;

[0016] A pre-built Hadoop database tool is used to build a hive table according to the file type set, and the file structure in the hive table is extracted.

[0017] Optionally, encoding the text content using a message digest algorithm to obtain an encrypted tag includes:

[0018] According to a preset configuration strategy, the target files are grouped and filled to obtain initialization data;

[0019] Obtain N pre-constructed official initial values, and use the official initial values ​​to perform a cyclic calculation on the initialization data for a predicted number of times to obtain N encrypted values;

[0020] The N encrypted values ​​are concatenated according to a preset order to obtain the encrypted label.

[0021] Optionally, compressing the text content to obtain a compressed file includes:

[0022] Using a preset replacement rule table, corresponding characters are replaced on characteristic character strings in the text content to obtain a primary compressed file;

[0023] A root node construction operation is performed on the primary compressed file using a Huffman coding algorithm to obtain the compressed file.

[0024] Optionally, after transmitting the encrypted tag, the compressed file, and the text structure to the second data center using the access token pre-generated by the second data center, the method further includes:

[0025] In the second data center, the compressed file is decompressed into the text structure using the text encryption tag to obtain the restored source data.

[0026] Optionally, after decompressing the compressed file into the text structure using the text encryption tag to obtain source data corresponding to the compressed file, the method further includes:

[0027] Encoding the source data using a pre-built encoder to obtain encoded data;

[0028] The coded data is segmented according to preset rules to obtain data blocks, and the data blocks are distributed and stored.

[0029] In order to solve the above problems, the present invention further provides a remote data synchronization device, the device comprising:

[0030] A file acquisition module, configured to extract source data of a preset type from the first data center to obtain a target file;

[0031] A content and structure extraction module, configured to extract the file content and file structure of the target file using a pre-built traversal service;

[0032] A compression and encryption module, used to compress the text content to obtain a compressed file, and encode the text content using a message digest algorithm to obtain an encryption tag;

[0033] The data transmission module is used to transmit the encryption tag, the compressed file and the text structure to the second data center by using the access token pre-generated by the second data center.

[0034] In order to solve the above problem, the present invention further provides an electronic device, comprising:

[0035] at least one processor; and,

[0036] a memory communicatively connected to the at least one processor; wherein,

[0037] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the remote data synchronization method described above.

[0038] In order to solve the above problem, the present invention also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is executed by a processor in an electronic device to implement the above-mentioned remote data synchronization method.

[0039] The embodiment of the present invention reduces the amount of data transmission and increases data transmission efficiency by compressing the data in the first data center. In addition, by extracting the file structure of the file content for transmission, it can automatically create a table to store the data when the data is transmitted to the second data center. This can not only preserve the coupling degree of each sub-file in the transmitted file, but also save the processing time of the second data center receiving the data and reduce data delay. Finally, the present invention encrypts the text content through a message digest algorithm, which can prevent the text content transmitted between data centers from being deciphered. Therefore, the remote data synchronization method, device, electronic device and computer-readable storage medium proposed by the present invention can solve the problem of high data delay when performing remote information synchronization. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A flowchart of a remote data synchronization method provided by one embodiment of the present invention;

[0041] Figure 2 A detailed flowchart of a step in a remote data synchronization method provided by one embodiment of the present invention;

[0042] Figure 3 A functional module diagram of a remote data synchronization device provided by one embodiment of the present invention;

[0043] Figure 4 A schematic structural diagram of an electronic device for implementing the remote data synchronization method provided by one embodiment of the present invention.

[0044] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0045] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0046] The embodiment of the present application provides a remote data synchronization method. The execution subject of the remote data synchronization method includes but is not limited to at least one of the electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided by the embodiment of the present application. In other words, the remote data synchronization method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0047] Reference Figure 1 FIG. 1 is a flow chart of a remote data synchronization method according to an embodiment of the present invention. In this embodiment, the remote data synchronization method includes:

[0048] S1. Extracting source data of a preset type from a first data center to obtain a target file.

[0049] As we all know, data centers are used for data backup and disaster recovery. If an incident occurs in one data center, the other data center can still store the company's data. In this case, data transmission only needs to ensure data integrity. However, in another case, such as when data applications in data center A require immediate use after certain data is collected and delivered, but this data is actually collected and delivered in data center B, a low-latency data synchronization service is required between data centers A and B to ensure the immediate delivery of data between the two data centers.

[0050] In detail, in an embodiment of the present invention, extracting source data of a preset type from the first data center to obtain a target file includes:

[0051] Using a preset monitoring service to monitor data updates in the first data center in real time;

[0052] If the first data center generates a new data file, the new data file is automatically extracted to obtain the target file.

[0053] In the embodiment of the present invention, the monitoring service is an executable program for monitoring whether there are new files in the file directory of the database of the first data center.

[0054] Specifically, in the embodiment of the present invention, when a new folder is created in the file directory, the monitoring service is executed and files in the newly created folder are extracted as the target files.

[0055] S2. Utilize a pre-built traversal service to extract the file content and file structure in the target file.

[0056] The traversal service is a recursive program that can access each folder in the target file in sequence.

[0057] For more details, please refer to Figure 2 As shown, in the embodiment of the present invention, the use of the pre-built traversal service to extract the file content and file structure in the target file includes:

[0058] S21. Using a pre-built traversal service, traverse and access each sub-file in the target file to obtain the file content;

[0059] S22, analyzing the file type of each sub-file to obtain a file type set;

[0060] S23. Using a pre-built Hadoop database tool, construct a hive table according to the file type set, and extract the file structure in the hive table.

[0061] The Hadoop database tool used in the embodiment of the present invention is Hive, which can map structured data files into a database table and provide SQL query functions.

[0062] The present invention utilizes the hive to judge the dependency relationship of each sub-file according to the file type of each sub-file, and constructs a hive table containing the file structure according to the dependency relationship of each sub-file, and stores the file content in the hive table accordingly.

[0063] S3. Compress the text content to obtain a compressed file, and encode the text content using a message digest algorithm to obtain an encryption tag.

[0064] The embodiment of the present invention compresses the text content in the hive table, including not only the data of each sub-file but also the directory hierarchy between the sub-files.

[0065] In detail, in an embodiment of the present invention, compressing the text content to obtain a compressed file includes:

[0066] Using a preset replacement rule table, corresponding characters are replaced on characteristic character strings in the text content to obtain a primary compressed file;

[0067] A root node construction operation is performed on the primary compressed file using a Huffman coding algorithm to obtain the compressed file.

[0068] In this embodiment of the present invention, the rule table is constructed based on commonly used enterprise data, such as [Chinese: 00, people: 01, China: 02...]. For example, if a text message reads "I am a Chinese person, I am from China," the compressed code becomes: "I am a 00 01, I am from 02." This significantly reduces the length of the compressed code.

[0069] Furthermore, the Huffman coding algorithm can be used to mark repeated characters in the primary compressed file to obtain a compressed file. For example, taking the string "aaaaaabbbbccddd" as an example, the frequency of the characters is replaced by the frequency of their occurrence, and the frequencies of the characters 'a', 'b', 'c', and 'd' are '6, 4, 2, and 3' respectively.

[0070] In the embodiment of the present invention, the node weights are first 6, 4, 2, and 3. The smallest 2 and 3 are selected to form a combined node with a root node of 5. The current node weights are 6, 4, and 5. The smallest 4 and 5 are selected to form a combined node with a root node of 9. The current node weights are 6 and 9. The smallest 6 and 9 are selected to form a combined node with a root node of 15. The current node weight is 15, and there is only one node. The Huffman tree is established.

[0071] The present invention wants to obtain the code of each character from the Huffman tree. As long as the Huffman tree is traversed from the root node to the character node, a '0' is added each time when going left and a '1' is added each time when going right, the final string obtained is the code string of the character.

[0072] In short, a Huffman coding algorithm can be used to convert a primary compressed file in which each character is of fixed length into a variable-length encoding by using the left-hand '0' and right-hand '1' method, further reducing the file size.

[0073] Furthermore, in an embodiment of the present invention, encoding the text content using a message digest algorithm to obtain an encrypted tag includes:

[0074] According to a preset configuration strategy, the target files are grouped and filled to obtain initialization data;

[0075] Obtain N pre-constructed official initial values, and use the official initial values ​​to perform a cyclic calculation on the initialization data for a predicted number of times to obtain N encrypted values;

[0076] The N encrypted values ​​are concatenated according to a preset order to obtain the encrypted label.

[0077] In the real-time example of the present invention, the original text length (bit) in the target file is modulo the value '512'. When the result of the modulo operation is not equal to '448', the original text needs to be padded so that the result of the original text modulo '512' is equal to 448. The padding method is to fill the first bit with 1 and the remaining bits with 0. After filling, the length of the information is 512*N+448. After that, the remaining positions (512-448=64 bits) are used to record the actual length of the original text, and the binary value of the length is added at the end. The length of the information after processing is the initialization data of 512*(N+1).

[0078] The embodiment of the present invention then obtains initial values ​​A, B, C, and D from the official website of the message digest algorithm, where A=0x01234567; B=0x89ABCDEF; C=0xFEDCBA98; and D=0x76543210. Then, the target file is cyclically processed using the four initial values ​​to obtain four 8-bit hexadecimal encryption values, and the four encrypted values ​​are concatenated in the order of A, B, C, and D to obtain a 32-bit hexadecimal encryption tag.

[0079] S4. Using the access token pre-generated by the second data center, transmit the encrypted tag, the compressed file, and the text structure to the second data center.

[0080] In detail, in an embodiment of the present invention, after the encrypted tag, the compressed file, and the text structure are transmitted to the second data center using the access token pre-generated by the second data center, the method further includes:

[0081] In the second data center, the compressed file is decompressed into the text structure using the text encryption tag to obtain the restored source data.

[0082] In the embodiment of the present invention, there is a program for regularly generating new access tokens between the second data center and the first data center, which ensures that the data between the first data center and the second data center is not easily intercepted, and also ensures that the information between the first data center and the second data center cannot be interfered with by information from other devices.

[0083] The embodiment of the present invention extracts key fields from the access token, such as the validity period and identity information. When the identity information matches the first data center and the validity period has not expired, the access token is determined to be valid; otherwise, it is invalid.

[0084] Furthermore, in an embodiment of the present invention, a new hive table is automatically constructed in the second data center based on the text structure transmitted by the second data center and assigned the same file name as the compressed file. The second data center uses a preset decompression tool to parse the file encryption tag and determine whether the file encryption tag corresponds to the compressed file. If they do not correspond, the compressed file is not decompressed. If they do correspond, the compressed file is decompressed into the new hive table to obtain the source data corresponding to the compressed file, wherein both the content and structure of the source file have not changed.

[0085] Furthermore, in an embodiment of the present invention, after decompressing the compressed file into the text structure using the text encryption tag to obtain the source data corresponding to the compressed file, the method further includes:

[0086] Encoding the source data using a pre-built encoder to obtain encoded data;

[0087] The coded data is segmented according to preset rules to obtain data blocks, and the data blocks are distributed and stored.

[0088] In order to ensure that data can be stored securely and not easily lost over time, embodiments of the present invention generally use a distributed storage method to store the source data.

[0089] In an embodiment of the present invention, an encoder is used to convert source data into encoded data, and then an erasure code is used to encrypt the encoded data to obtain an encrypted encoded block. The encrypted encoded block is then segmented according to a specific segmentation method of the erasure code to obtain multiple data blocks. Finally, the multiple data blocks are distributed and stored on various memories of the second data center. The erasure code is a data protection method that can segment data into fragments, expand and encode redundant data blocks, and store them in different locations, such as disks, storage nodes, or other network locations.

[0090] The embodiments of the present application can automate the actions in S1 to S4 based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.

[0091] In summary, the embodiments of the present invention reduce the amount of data transmission and increase data transmission efficiency by compressing the data in the first data center. In addition, by extracting the file structure of the file content for transmission, a table can be automatically created to store the data when the data is transmitted to the second data center. This not only preserves the coupling degree of each sub-file in the transmitted file, but also saves the processing time of the second data center receiving the data and reduces data latency. Finally, the present invention encrypts the text content through a message digest algorithm, which can prevent the text content transmitted between data centers from being deciphered. Therefore, the remote data synchronization method proposed by the present invention can solve the problem of high data latency during remote information synchronization.

[0092] like Figure 3 , which is a functional module diagram of a remote data synchronization device provided by an embodiment of the present invention.

[0093] The remote data synchronization device 100 described in the present invention can be installed in an electronic device. Depending on the functionality implemented, the remote data synchronization device 100 may include a file acquisition module 101, a content and structure extraction module 102, a compression and encryption module 103, and a data transmission module 104. A module, also referred to as a unit, is a series of computer program segments that can be executed by an electronic device processor and perform a fixed function, and is stored in the electronic device's memory.

[0094] In this embodiment, the functions of each module / unit are as follows:

[0095] The file acquisition module 101 is used to extract source data of a preset type from the first data center to obtain a target file;

[0096] The content and structure extraction module 102 is used to extract the file content and file structure of the target file using a pre-built traversal service;

[0097] The compression and encryption module 103 is used to compress the text content to obtain a compressed file, and encode the text content using a message digest algorithm to obtain an encryption tag;

[0098] The data transmission module 104 is configured to transmit the encrypted tag, the compressed file, and the text structure to the second data center using the access token pre-generated by the second data center.

[0099] In detail, each module in the remote data synchronization device 100 according to the embodiment of the present invention adopts the same Figures 1 to 2 The remote data synchronization method described in the present invention has the same technical means and can produce the same technical effects, so it will not be repeated here.

[0100] like Figure 4 FIG. 1 is a schematic diagram of the structure of an electronic device for implementing a remote data synchronization method provided by an embodiment of the present invention.

[0101] The electronic device 1 may include a processor 10 , a memory 11 , a communication bus 12 , and a communication interface 13 , and may further include a computer program stored in the memory 11 and executable on the processor 10 , such as a remote data synchronization program.

[0102] In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 10 is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and lines, and executing or executing programs or modules stored in the memory 11 (for example, executing remote data synchronization programs, etc.), as well as calling data stored in the memory 11, to execute various functions of the electronic device and process data.

[0103] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 can also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Furthermore, the memory 11 can also include both an internal storage unit of the electronic device and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device, such as the code of a remote data synchronization program, etc., but can also be used to temporarily store data that has been output or is to be output.

[0104] The communication bus 12 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0105] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0106] Figure 4 Only the electronic device with components is shown, and it can be understood by those skilled in the art that Figure 4 The structure shown does not constitute a limitation on the electronic device 1 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0107] For example, although not shown, the electronic device may further include a power source (such as a battery) for powering various components. Preferably, the power source may be logically connected to the at least one processor 10 via a power management device, thereby implementing functions such as charge management, discharge management, and power consumption management through the power management device. The power source may further include any components such as one or more DC or AC power sources, a recharging device, a power failure detection circuit, a power converter or inverter, a power status indicator, etc. The electronic device may further include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0108] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0109] The remote data synchronization program stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve the following:

[0110] Extracting source data of a preset type from the first data center to obtain a target file;

[0111] Utilize pre-built traversal services to extract the file content and file structure of the target file;

[0112] Compressing the text content to obtain a compressed file, and encoding the text content using a message digest algorithm to obtain an encryption tag;

[0113] The encrypted tag, the compressed file, and the text structure are transmitted to the second data center using an access token pre-generated by the second data center.

[0114] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, which will not be repeated here.

[0115] Furthermore, if the modules / units integrated into the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0116] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor of an electronic device, the computer program can implement:

[0117] Extracting source data of a preset type from the first data center to obtain a target file;

[0118] Utilize pre-built traversal services to extract the file content and file structure of the target file;

[0119] Compressing the text content to obtain a compressed file, and encoding the text content using a message digest algorithm to obtain an encryption tag;

[0120] The encrypted tag, the compressed file, and the text structure are transmitted to the second data center using an access token pre-generated by the second data center.

[0121] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical function division, and other division methods may be used in actual implementation.

[0122] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0123] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0124] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0125] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0126] Blockchain, as used in this article, refers to a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each block contains information about a batch of online transactions, used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0127] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.

[0128] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. Terms such as "first" and "second" are used to indicate names and do not imply any particular order.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A remote data synchronization method, characterized in that: The method comprises: Extracting source data of a preset type from the first data center to obtain a target file; Using a pre-built traversal service, the dependencies of the sub-files of the target file are determined based on their file types, a hive table containing the file structure is constructed based on the dependencies, and the contents of the target file are stored in the hive table. Compress the text content of the hive table to obtain a compressed file, and encode the text content using a message digest algorithm to obtain an encryption tag. The text content includes the data of each sub-file and the directory hierarchy between the sub-files; Using the access token pre-generated by the second data center, the encryption tag, the compressed file and the file structure are transmitted to the second data center. In the second data center, the compressed file is decompressed into the file structure using the encryption tag to obtain the restored source data.

2. The remote data synchronization method according to claim 1, wherein: The step of extracting source data of a preset type from the first data center to obtain a target file includes: Using a preset monitoring service to monitor data updates in the first data center in real time; If the first data center generates a new data file, the new data file is automatically extracted to obtain the target file.

3. The remote data synchronization method according to claim 1, wherein: Before constructing a hive table containing a file structure according to the dependency relationship, the method further includes: Utilize the pre-built traversal service to traverse and access each sub-file in the target file to obtain the text content.

4. The remote data synchronization method according to claim 1, wherein: The method of encoding the text content using a message digest algorithm to obtain an encrypted tag includes: According to a preset configuration strategy, the target files are grouped and filled to obtain initialization data; Obtain N pre-constructed official initial values, and use the official initial values ​​to perform a cyclic calculation on the initialization data for a predicted number of times to obtain N encrypted values; The N encrypted values ​​are concatenated according to a preset order to obtain the encrypted label.

5. The remote data synchronization method according to claim 1, wherein: The compressing the text content of the hive table to obtain a compressed file includes: Using a preset replacement rule table, corresponding characters are replaced on characteristic character strings in the text content to obtain a primary compressed file; A root node construction operation is performed on the primary compressed file using a Huffman coding algorithm to obtain the compressed file.

6. The remote data synchronization method according to claim 1, wherein: After decompressing the compressed file into the file structure using the text encryption tag to obtain source data corresponding to the compressed file, the method further includes: Encoding the source data using a pre-built encoder to obtain encoded data; The coded data is segmented according to preset rules to obtain data blocks, and the data blocks are distributed and stored.

7. A remote data synchronization device, characterized in that: The device comprises: A file acquisition module, configured to extract source data of a preset type from the first data center to obtain a target file; A content and structure extraction module is used to use a pre-built traversal service to determine the dependency relationships of the sub-files of the target file based on their file types, build a Hive table containing the file structure based on the dependencies, and store the contents of the target file in the Hive table; A compression and encryption module is used to compress the text content of the hive table to obtain a compressed file, and encode the text content using a message digest algorithm to obtain an encryption tag. The text content includes the data of each sub-file and the directory hierarchy between the sub-files; A data transmission module is used to use the access token pre-generated by the second data center to transmit the encryption tag, the compressed file and the file structure to the second data center. In the second data center, the encryption tag is used to decompress the compressed file into the file structure to obtain the restored source data.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the remote data synchronization method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the remote data synchronization method according to any one of claims 1 to 6 is implemented.

Images