Method, device and equipment for data synchronization and storage medium

Abstract

The invention relates to the technical field of data processing, and discloses a method, device and equipment for data synchronization and a storage medium, and the method comprises the following steps: obtaining to-be-synchronized data and data definition language information of the to-be-synchronized data; creating a data synchronization completion table, and creating a data storage table based on the data definition language information; writing the to-be-synchronized data into the data storage table based on the data definition language information, and updating the data synchronization completion table based on the writing condition of the to-be-synchronized data to obtain an updating result; and judging the synchronization state of the to-be-synchronized data based on the updating result. According to the invention, the data synchronization process is simplified, the number of data synchronization participants is reduced, the probability of data synchronization errors is reduced, and the reliability and efficiency of data synchronization are improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, an apparatus, a device and a storage medium for data synchronization. Background Art

[0002] Related technologies In cross-company data synchronization links, first, one party writes the data to be synchronized into the Object Storage Service (OSS), and then the other party pulls the data to be synchronized to the OSS of the other party. The process is cumbersome, involving multiple links and participants, and data synchronization is prone to errors. Summary of the invention

[0003] In view of this, the present invention provides a method, apparatus, device and storage medium for data synchronization to solve the problem that in the process of data synchronization in related technologies, the process is cumbersome, involves multiple links and participants, and data synchronization is prone to errors.

[0004] In a first aspect, the present invention provides a method for data synchronization, the method comprising: obtaining data to be synchronized and data definition language information of the data to be synchronized; creating a data synchronization completion table, and creating a data storage table based on the data definition language information; writing the data to be synchronized into the data storage table based on the data definition language information, and updating the data synchronization completion table based on the writing status of the data to be synchronized to obtain an update result; judging the synchronization status of the data to be synchronized based on the update result.

[0005] In an optional implementation, based on the data definition language information, the data to be synchronized is written into the data storage table, and based on the writing status of the data to be synchronized, the data synchronization completion table is updated to obtain an update result, including: writing the data to be synchronized into the data storage table based on the format represented by the data definition language information; after the writing of the data to be synchronized is completed, updating the data synchronization flag in the data synchronization completion table to obtain the update result, wherein whether the data to be synchronized is synchronized to the data storage table is represented based on the status of the data synchronization flag.

[0006] In an optional embodiment, judging the synchronization status of the data to be synchronized based on the update result includes: if the status of the data synchronization identifier indicates that the data to be synchronized is synchronized successfully, reading the data to be synchronized from the data storage table; if the data synchronization identifier is not detected, or the status of the data synchronization identifier indicates that the data to be synchronized is not synchronized, detecting the data synchronization identifier again at a preset time interval, and issuing an alarm if the synchronization of the data to be synchronized is not detected successfully.

[0007] In an optional embodiment, based on the data definition language information, the data to be synchronized is written into the data storage table, and it also includes: encrypting the data to be synchronized and writing it into the data storage table in a format represented by the data definition language information; obtaining a key, dividing the data to be synchronized into blocks based on a preset length, encrypting each block using the key and encryption algorithm, obtaining a ciphertext based on the encrypted block, and writing the ciphertext into the data storage table.

[0008] In an optional implementation, based on the data definition language information, writing the data to be synchronized into the data storage table also includes: after encrypting the data to be synchronized and setting the encrypted data field order, writing the data into the data storage table based on the format represented by the data definition language information.

[0009] In an optional embodiment, if the state of the data synchronization identifier indicates that the data to be synchronized is synchronized successfully, reading the data to be synchronized from the data storage table includes: reading the encrypted data to be synchronized from the data storage table, decrypting the encrypted data to be synchronized to obtain the decrypted data to be synchronized, and reversely restoring the decrypted data to be synchronized based on the order of the encrypted data fields to obtain the original data to be synchronized.

[0010] In a second aspect, the present invention provides a device for data synchronization, the device comprising: a data acquisition module, used to acquire the data to be synchronized and the data definition language information of the data to be synchronized; a table creation module, used to create a data synchronization completion table, and create a data storage table based on the data definition language information; a data update module, used to write the data to be synchronized into the data storage table based on the data definition language information, and update the data synchronization completion table based on the writing status of the data to be synchronized to obtain an update result; and a judgment module, used to judge the synchronization status of the data to be synchronized based on the update result.

[0011] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method for data synchronization of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0012] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to cause a computer to execute the method for data synchronization of the above-mentioned first aspect or any corresponding embodiment thereof.

[0013] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the method for data synchronization of the first aspect or any corresponding embodiment thereof.

[0014] Based on the data definition language information, the data format of the data to be synchronized is made compatible with the structure of the data storage table. Based on the writing status of the data to be synchronized, the data synchronization completion table is updated, and the synchronization status of the data to be synchronized is judged according to the update result. The data synchronization process can be monitored to ensure that the data to be synchronized is written into the data storage table, which simplifies the data synchronization process, reduces the number of data synchronization participants, reduces the probability of data synchronization errors, and improves the reliability and efficiency of data synchronization. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A schematic diagram of a process flow of a method for data synchronization provided by an embodiment of the present invention is shown;

[0017] Figure 2 A schematic flow chart showing another method for data synchronization according to an embodiment of the present invention;

[0018] Figure 3 A schematic flow chart showing another method for data synchronization according to an embodiment of the present invention;

[0019] Figure 4 A schematic diagram showing the structure of a device for data synchronization provided by an embodiment of the present invention is shown;

[0020] Figure 5 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0022] The concepts involved in the embodiments of the present invention are introduced below. OSS is a massive, secure, low-cost and highly reliable cloud storage service that can store any type of files.

[0023] According to an embodiment of the present invention, a method embodiment for data synchronization is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0024] In this embodiment, a method for data synchronization is provided, which can be used in a server or a cloud storage service. Figure 1 A schematic diagram of a method for data synchronization provided by an embodiment of the present invention is shown. Figure 1 As shown, the process includes the following steps:

[0025] Step S101: acquiring data to be synchronized and data definition language information of the data to be synchronized.

[0026] In this step, the data to be synchronized can be determined based on business needs and data consistency requirements. Based on the acquisition requirements of the data to be synchronized, the source of the data to be synchronized is determined. The data definition language (DDL) information of the data to be synchronized can be obtained using a database management tool or a command line interface. The DDL information can include structured query language (SQL) database statements for creating tables, indexes, or constraints. The DDL information can be recorded in a document or stored in a version control system to facilitate re-creation or modification of the database structure.

[0027] Step S102: Create a data synchronization completion table, and create a data storage table based on the data definition language information.

[0028] In this step, the data storage table is the basic and core table in the database for storing actual business data. The data storage table includes one or more of the following: primary key, foreign key, index, field type, and table normalization. Among them, the primary key is used to uniquely identify each row of records in the table, which can be a single column or a combination of multiple columns; the foreign key is used to create relationships between multiple tables to ensure data consistency and integrity; the index is used to increase data retrieval speed; the field type, each field has a specific data type, such as an integer or a string, etc.; the table normalization can reduce data redundancy and synchronization data update anomalies through normalization processing.

[0029] The data synchronization completion table is also a data storage table, which is used to record the completion status and related information of the data synchronization task. The data synchronization completion table may include: a synchronization task identifier, a source device / system, a target device / system, a synchronization start time, a synchronization end time, a synchronization status, a synchronization data volume, an error message, and a timestamp. Among them, the synchronization task identifier can be used to uniquely identify a data synchronization task; the source device / system is used to characterize the starting device or system of data synchronization; the target device / system is used to characterize the destination device or system of data synchronization; the synchronization start time is used to characterize the timestamp when the data synchronization task starts; the synchronization end time is used to characterize the timestamp when the data synchronization task ends; the synchronization status is used to indicate the status of the data synchronization task, such as "success", "failure" or "in progress"; the synchronization data volume is used to characterize the amount of data transmitted during the synchronization process; the error message is used to record the cause of the failure or the error message when the data synchronization task fails; the timestamp can be used to record the time when the data synchronization completion table entry is created or updated.

[0030] Step S103, based on the data definition language information, write the data to be synchronized into the data storage table, and based on the writing status of the data to be synchronized, update the data synchronization completion table to obtain the update result.

[0031] In this step, if the data storage table has not been created, you can create the data storage table based on the parsed DDL information; if the data storage table already exists, you can check and adjust the structure of the data storage table to ensure that the data storage table is consistent with the structure of the data to be synchronized. You can use database operations such as insert or update to write the data to be synchronized into the data storage table.

[0032] Write the data to be synchronized into the data storage table, record the amount of data written, the number of data successfully written, and any errors or exceptions that occurred. Update the data synchronization completion table based on the writing status of the data to be synchronized. Query the data synchronization completion table to obtain the latest synchronization task status and results.

[0033] Step S104: determining the synchronization status of the data to be synchronized based on the update result.

[0034] In this step, if the synchronization field status is "Successful", it means that the data to be synchronized has been successfully written into the data storage table and no errors were encountered during the synchronization process. The data receiver can obtain the synchronized data from the data storage table. If the synchronization field status is "Failed", you can view the error information field to obtain the reason for the data synchronization failure.

[0035] The method for data synchronization provided in this embodiment makes the data format of the data to be synchronized compatible with the structure of the data storage table based on the data definition language information, updates the data synchronization completion table based on the writing status of the data to be synchronized, and judges the synchronization status of the data to be synchronized according to the update result. It can monitor the data synchronization process and ensure that the data to be synchronized is written into the data storage table, which simplifies the data synchronization process, reduces the number of data synchronization participants, reduces the probability of data synchronization errors, and improves the reliability and efficiency of data synchronization.

[0036] In some optional implementations, based on the data definition language information, the data to be synchronized is written into the data storage table, and based on the writing status of the data to be synchronized, the data synchronization completion table is updated to obtain an update result, including: writing the data to be synchronized into the data storage table in a format represented by the data definition language information; after the writing of the data to be synchronized is completed, updating the data synchronization identifier in the data synchronization completion table to obtain an update result, wherein the status of the data synchronization identifier represents whether the data to be synchronized is synchronized to the data storage table.

[0037] In this embodiment, a method for data synchronization is provided, which can be used for the above-mentioned server or cloud storage service, etc. Figure 2 Another schematic diagram of a method for data synchronization according to an embodiment of the present invention is shown. Figure 2 As shown, the process includes the following steps:

[0038] Step S201: Obtain the data to be synchronized and the data definition language information of the data to be synchronized. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0039] Step S202: Create a data synchronization completion table, and create a data storage table based on the data definition language information. Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0040] Step S203: based on the data definition language information, write the data to be synchronized into the data storage table, and based on the writing status of the data to be synchronized, update the data synchronization completion table to obtain an update result.

[0041] Specifically, the above step S203 includes:

[0042] Step S2031, writing the data to be synchronized into the data storage table based on the format represented by the data definition language information.

[0043] In this step, the data to be synchronized is converted into a table format, and the data to be synchronized is converted according to the structure of the data storage table to ensure that each data item corresponds to a corresponding column.

[0044] Step S2032: after the writing of the data to be synchronized is completed, the data synchronization mark is updated in the data synchronization completion table to obtain an update result, wherein the state of the data synchronization mark indicates whether the data to be synchronized is synchronized to the data storage table.

[0045] In this step, a unique synchronization task identifier can be set for the data synchronization task, and the data synchronization status can be tracked and recorded in the data synchronization completion table based on the synchronization task identifier. The synchronization task identifier can be used as a keyword to find the corresponding record in the data synchronization completion table and update the data synchronization identification status in the record.

[0046] In this way, it can be ensured that the data synchronization identification status in the data synchronization completion table can accurately reflect whether the data to be synchronized has been successfully synchronized to the data storage table, and can provide useful information for subsequent processing and monitoring of synchronized data.

[0047] In this embodiment, a method for data synchronization is provided, which can be used for the above-mentioned server or cloud storage service, etc. Figure 3 Another flow chart of the method for data synchronization according to an embodiment of the present invention is shown. Figure 3 As shown, the process includes the following steps:

[0048] Step S301: Obtain the data to be synchronized and the data definition language information of the data to be synchronized. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0049] Step S302: Create a data synchronization completion table, and create a data storage table based on the data definition language information. Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0050] Step S303: based on the data definition language information, write the data to be synchronized into the data storage table, and based on the writing status of the data to be synchronized, update the data synchronization completion table to obtain the update result. Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0051] Step S304: Determine the synchronization status of the data to be synchronized based on the update result

[0052] Specifically, the above step S304 includes:

[0053] Step S3041: If the state of the data synchronization flag indicates that the data to be synchronized is synchronized successfully, the data to be synchronized is read from the data storage table.

[0054] In this step, determine the name and location of the data storage table where the data to be synchronized is written. Construct an SQL query statement based on the structure of the data storage table and the characteristics of the data to be synchronized. The SQL statement can specify the fields or conditions to be read. Check whether the read data is consistent with expectations, including the type, quantity, and content of the data. If inconsistent or missing data is found, check the synchronization process or query statement.

[0055] Step S3042: If the data synchronization mark is not detected, or the state of the data synchronization mark indicates that the data to be synchronized is not synchronized, the data synchronization mark is detected again at a preset interval. If the synchronization of the data to be synchronized is not detected successfully, an alarm is issued.

[0056] In this step, the preset duration can be determined based on the actual application scenario. The setting of the preset duration should allow the data synchronization process to have enough time to complete, and also avoid delayed alarms when problems occur in data synchronization. After an interval of the preset duration, the status of the data synchronization flag is checked again. If the status of the data synchronization flag changes to "success", it means that the data to be synchronized has been successfully synchronized to the data storage table, and the detection process can be ended. If the status of the data synchronization flag is checked again after an interval of the preset duration, and the status of the data synchronization flag still indicates that the data to be synchronized is not synchronized, or the data synchronization flag is not detected at all, an alarm is required.

[0057] Alerts can be sent via email, message notification, or log recording to promptly notify relevant personnel or system administrators.

[0058] In this way, when the data synchronization mark is not detected or the state of the data synchronization mark indicates that the data to be synchronized is not synchronized, the problem can be discovered in time and an alarm can be issued, so as to quickly respond to and solve the synchronization problem.

[0059] In some optional embodiments, based on the data definition language information, the data to be synchronized is written into the data storage table, and the method also includes: encrypting the data to be synchronized and writing it into the data storage table in a format represented by the data definition language information; obtaining a key, dividing the data to be synchronized into blocks based on a preset length, encrypting each block using a key and an encryption algorithm, obtaining a ciphertext based on the encrypted blocks, and writing the ciphertext into the data storage table.

[0060] In this embodiment, the data to be synchronized may be encrypted by a symmetric encryption algorithm (Advanced Encryption Standard, referred to as AES). Other encryption algorithms may also be used to encrypt the data to be synchronized. A key of a length supported by AES is obtained, the data to be synchronized is divided into blocks of a fixed size, each block is encrypted using a key and an encryption algorithm, and the encrypted blocks constitute ciphertext.

[0061] After the synchronization data receiver obtains the aforementioned ciphertext based on the data storage table, it uses the same key and encryption algorithm to decrypt the ciphertext and reassembles the decrypted blocks to obtain the synchronization data.

[0062] In this way, encryption processing is performed on the data to be synchronized, thereby ensuring the security of the synchronized data.

[0063] In some optional implementations, writing the data to be synchronized into the data storage table based on the data definition language information also includes: encrypting the data to be synchronized and setting the encrypted data field order, and then writing it into the data storage table based on the format represented by the data definition language information.

[0064] In this embodiment, the encrypted data is arranged in the order of fields to match the format of the data storage table. Using database operation commands or data synchronization tools, the arranged encrypted data is written into the data storage table to ensure that the value of each field correctly corresponds to the corresponding column of the data storage table. The security of the synchronized data can be further improved.

[0065] In some optional embodiments, if the state of the data synchronization identifier indicates that the synchronization of the data to be synchronized is successful, reading the data to be synchronized from the data storage table includes: reading the encrypted data to be synchronized from the data storage table, decrypting the encrypted data to be synchronized to obtain the decrypted data to be synchronized, and reversely restoring the decrypted data to be synchronized based on the order of the encrypted data fields to obtain the original data to be synchronized.

[0066] In this embodiment, the synchronization data receiver reads the encrypted data to be synchronized from the data storage table, decrypts the data to be synchronized, and reversely restores it according to the field order agreed with the data sender to obtain the original synchronization data, completing the entire data synchronization process.

[0067] In this embodiment, another method for data synchronization is provided, and the sender and receiver of the synchronized data determine the DDL of the data to be synchronized, the order of the encrypted data fields, the data synchronization identifier, and the latest output time promised and guaranteed by the synchronized data. The sender of the synchronized data can use the Alibaba Cloud Big Data Development Kit (DataWorks) to write the data to be synchronized into the data storage table, and can perform AES encryption, or use other encryption algorithms, and write the data synchronization identifier and the number of synchronized data in the data synchronization completion table. The receiver of the synchronized data sets a monitoring node to monitor the data synchronization completion table to ensure that the data is written correctly. The monitoring node can be used to cyclically detect whether the data synchronization identifier exists, and the detection interval can be set to five minutes, and the number of cyclic detections can be set as needed. The receiver of the synchronized data can decrypt and convert the synchronized data, and reversely restore it according to the agreed field order. The receiver of the synchronized data obtains the original synchronized data and stores it correctly in the data warehouse.

[0068] You can write a monitoring script, such as a Python script, to check whether the synchronization mark in the data synchronization completion table exists. If it exists, it means that the data synchronization is successful. If it does not exist, you can wait for a while and check again. You can also set a maximum waiting time. If the data synchronization mark is still not detected when the maximum waiting time is reached, you can terminate the operation and alarm.

[0069] In this embodiment, a device for data synchronization is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0070] Figure 4 FIG. 1 is a schematic diagram showing a structure of a device for data synchronization provided by an embodiment of the present invention. This embodiment provides a device for data synchronization, such as Figure 4 As shown, including:

[0071] The data acquisition module 401 is used to acquire the data to be synchronized and the data definition language information of the data to be synchronized.

[0072] The table creation module 402 is used to create a data synchronization completion table and a data storage table based on the data definition language information.

[0073] The data updating module 403 is used to write the data to be synchronized into the data storage table based on the data definition language information, and to update the data synchronization completion table based on the writing status of the data to be synchronized to obtain the update result.

[0074] The judgment module 404 is used to judge the synchronization status of the data to be synchronized based on the update result.

[0075] In some optional implementations, the data updating module 403 includes:

[0076] The first data updating unit is used to write the to-be-synchronized data into the data storage table based on the format represented by the data definition language information.

[0077] The second data updating unit is used to update the data synchronization mark in the data synchronization completion table after the writing of the data to be synchronized is completed to obtain an update result, wherein the state of the data synchronization mark represents whether the data to be synchronized is synchronized to the data storage table.

[0078] In some optional implementations, the determination module 404 includes:

[0079] The first judgment unit is configured to read the data to be synchronized from the data storage table if the state of the data synchronization identifier indicates that the data to be synchronized is synchronized successfully.

[0080] The second judgment unit is used to detect the data synchronization mark again at a preset interval if the data synchronization mark is not detected, or the state of the data synchronization mark indicates that the data to be synchronized is not synchronized, and to issue an alarm if the synchronization of the data to be synchronized is not detected successfully.

[0081] In some optional implementations, the data updating module 403 further includes:

[0082] The third data updating unit is used to write the encrypted data to be synchronized into the data storage table in a format represented by the data definition language information.

[0083] The fourth data updating unit is used to obtain a key, divide the data to be synchronized into blocks based on a preset length, encrypt each block using a key and an encryption algorithm, obtain a ciphertext based on the encrypted block, and write the ciphertext into a data storage table.

[0084] In some optional implementations, the determination module 404 further includes:

[0085] The third judgment unit is used to encrypt the data to be synchronized and set the encrypted data field sequence, and then write the data into the data storage table based on the format represented by the data definition language information.

[0086] In some optional embodiments, the third judgment unit also includes: a third judgment sub-unit, which is used to read the encrypted data to be synchronized from the data storage table, decrypt the encrypted data to be synchronized, obtain the decrypted data to be synchronized, and reversely restore the decrypted data to be synchronized based on the order of the encrypted data fields to obtain the original data to be synchronized.

[0087] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0088] The device for data synchronization in this embodiment is presented in the form of a functional unit, where the unit refers to an application specific integrated circuit (ASIC) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0089] The device for data synchronization provided by the embodiment of the present invention makes the data format of the data to be synchronized compatible with the structure of the data storage table based on the data definition language information, updates the data synchronization completion table based on the writing status of the data to be synchronized, and judges the synchronization status of the data to be synchronized according to the update result. It can monitor the data synchronization process and ensure that the data to be synchronized is written into the data storage table, which simplifies the data synchronization process, reduces the number of data synchronization participants, reduces the probability of data synchronization errors, and improves the reliability and efficiency of data synchronization.

[0090] The embodiment of the present invention also provides a computer device having the above Figure 4 An apparatus for data synchronization is shown.

[0091] See also Figure 5 , Figure 5 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 5 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display graphical information of a graphical user interface on an external input / output device (such as a display device coupled to an interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.

[0092] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0093] The aforementioned memory 20 stores instructions executable by at least one processor 10, so that the aforementioned at least one processor 10 executes the method shown in the above embodiment.

[0094] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0095] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0096] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0097] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0098] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0099] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for data synchronization, characterized in that: The method comprises: Acquire data to be synchronized and data definition language information of the data to be synchronized; Creating a data synchronization completion table, and creating a data storage table based on the data definition language information; Based on the data definition language information, the data to be synchronized is written into the data storage table, and based on the writing status of the data to be synchronized, the data synchronization completion table is updated to obtain an update result; The synchronization status of the data to be synchronized is determined based on the update result.

2. The method according to claim 1, characterized in that Based on the data definition language information, the data to be synchronized is written into the data storage table, and based on the writing status of the data to be synchronized, the data synchronization completion table is updated, and the update result obtained includes: Writing the data to be synchronized into the data storage table based on the format represented by the data definition language information; After the writing of the data to be synchronized is completed, the data synchronization mark is updated in the data synchronization completion table to obtain the update result, wherein the state of the data synchronization mark represents whether the data to be synchronized is synchronized to the data storage table.

3. The method according to claim 2, characterized in that Determining the synchronization status of the to-be-synchronized data based on the update result includes: If the state of the data synchronization identifier indicates that the data to be synchronized is synchronized successfully, reading the data to be synchronized from the data storage table; If the data synchronization identifier is not detected, or the state of the data synchronization identifier indicates that the data to be synchronized is not synchronized, the data synchronization identifier is detected again at a preset time interval, and if the synchronization of the data to be synchronized is still not detected to be successful, an alarm is issued.

4. The method according to claim 3, characterized in that Based on the data definition language information, writing the to-be-synchronized data into the data storage table further includes: Writing the encrypted data to be synchronized into the data storage table in a format represented by the data definition language information; Obtain a key, divide the data to be synchronized into blocks based on a preset length, encrypt each block using the key and encryption algorithm, obtain ciphertext based on the encrypted block, and write the ciphertext into the data storage table.

5. The method according to claim 4, characterized in that Based on the data definition language information, writing the to-be-synchronized data into the data storage table further includes: After encrypting the data to be synchronized and setting the encrypted data field order, the data is written into the data storage table based on the format represented by the data definition language information.

6. The method according to claim 5, characterized in that If the state of the data synchronization identifier indicates that the data to be synchronized is successfully synchronized, reading the data to be synchronized from the data storage table includes: The encrypted data to be synchronized is read from the data storage table, the encrypted data to be synchronized is decrypted to obtain the decrypted data to be synchronized, and the decrypted data to be synchronized is reversely restored based on the encrypted data field sequence to obtain the original data to be synchronized.

7. A device for data synchronization, characterized in that: The device comprises: A data acquisition module, used to acquire the data to be synchronized and the data definition language information of the data to be synchronized; A table creation module is used to create a data synchronization completion table and a data storage table based on the data definition language information; A data updating module, configured to write the data to be synchronized into the data storage table based on the data definition language information, and to update the data synchronization completion table based on the writing status of the data to be synchronized to obtain an update result; A judgment module is used to judge the synchronization status of the data to be synchronized based on the update result.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for data synchronization according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for data synchronization according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the method for data synchronization according to any one of claims 1 to 6.

Images