A data synchronization method, a synchronization device, a computer device, and a storage medium

Abstract

The application relates to a data synchronization method, a synchronization device, computer equipment and a storage medium. The method comprises the following steps: obtaining target data of a third-party system; reading the attributes and data of the target data of the third-party system; configuring a mapping rule file corresponding to the attributes and data of the target data according to the attributes and data of the target data; and calling a data storage general method to perform corresponding operations on the target data according to the mapping rule file of the attributes and data of the target data. The technical scheme of the application reduces the code development amount caused by the access to the third-party system or the change of the data synchronization field of the third-party system and the code change amount caused by business changes, and stably and efficiently realizes the data synchronization between heterogeneous systems.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data application, and more particularly to a data synchronization method, a synchronization device, a computer device and a storage medium. BACKGROUND

[0002] In the existing data synchronization technical solution, data synchronization between different system databases can be achieved, but it is not stable, efficient and low in code development amount. In related technologies, attribute mapping relationship needs to be established for data synchronization between different system databases, and separate code development is required due to the influence of different system database attributes. Moreover, such data synchronization technology is often limited to specific system databases, and needs to use the characteristics of the database itself for data synchronization, such as database triggers, stored procedures and other tools, and use the database log for synchronization processing. Therefore, the data synchronization has a strong correlation with the database, which is not convenient for system database iteration, replacement and data synchronization, and increases the difficulty of supporting other system databases and the code development amount. Therefore, how to stably and efficiently realize data synchronization between different system databases and reduce the code development amount caused by the change of data synchronization fields between different systems has become a problem to be solved. SUMMARY

[0003] To solve the problems existing in the prior art data synchronization technology, the present application provides a data synchronization method, a synchronization device, a computer device and a storage medium.

[0004] The technical solution adopted by the present application to solve its technical problems is to provide a data synchronization method, characterized by comprising:

[0005] S1, obtaining target data of a third-party system;

[0006] S2, reading the attributes and data of the target data of the third-party system;

[0007] S3, configuring a mapping rule file corresponding to the attributes and data of the target data according to the attributes and data of the target data;

[0008] S4, calling a data storage general method according to the mapping rule file of the attributes and data of the target data to perform corresponding operations on the target data.

[0009] Preferably, the attributes of the target data include:

[0010] dictionary content of the target data;

[0011] correlation of the target data;

[0012] data type of the target data.

[0013] Preferably, the step S3 comprises the following steps:

[0014] S3-1: configuring the mapping rule XML file overall structure fragment of the target data, for defining data transmission rule and file transmission attribute;

[0015] S3-2: according to the data of the target data, configuring the XML data template corresponding to the data of the target data, for reading the data of the target data into the corresponding position of the XML data template;

[0016] S3-3: according to the dictionary content of the target data, configuring the mapping rule XML file fragment corresponding to the dictionary content of the target data, for storing the target data with mapping relationship;

[0017] S3-4: according to the association relationship of the target data, configuring the mapping rule XML file fragment corresponding to the association relationship of the target data, for querying multiple target data results associated with the target data according to the association relationship of the target data;

[0018] S3-5: according to the data type of the target data, configuring the mapping rule XML file fragment corresponding to the synchronization range of the target data, for determining the synchronization range of the target data and marking the transmission progress state of the target data;

[0019] The mapping rule XML file fragment of the synchronization range of the target data comprises: the first synchronization range fragment of the target data and the transmission state value fragment of the target data;

[0020] S3-6: combining the XML data template of the data of the target data and all the mapping rule XML file fragments of the attributes of the target data, combining the mapping rule XML file overall structure fragment of the target data, to form the complete mapping rule XML file of the attributes and data of the target data.

[0021] Preferably, the data storage general method comprises:

[0022] M1: data storage general method of synchronizing data from the third-party system to the Documentum-based document system;

[0023] M2: data storage general method of synchronizing data from the Documentum-based document system to the third-party system;

[0024] M3: data storage general method of uploading files from the third-party system to the Documentum-based document system;

[0025] M4: a data storage universal method for uploading files from the Documentum-based document system to the third-party system.

[0026] Preferably, the step S4 comprises the following steps:

[0027] S4-1: according to the XML data template of the data of the target data and the mapping rule XML file fragment of the attributes of the target data, calling the data storage universal method M1, synchronizing the target data from the third-party system to the Documentum-based document system, moving the target data synchronized successfully into a historical database, and retransmitting the target data synchronized unsuccessfully;

[0028] The historical database is a document database for persistently storing the target data synchronized successfully in the Documentum-based document system;

[0029] S4-2: according to the mapping rule XML file overall structure fragment of the target data, calling the data storage universal method M3, uploading files from the third-party system to the Documentum-based document system.

[0030] Preferably, the step S3-1 comprises:

[0031] According to the third-party system data type defined by the dataType field, calling the mapping rule XML file fragment corresponding to the synchronization range of the target data configured by the resultMap field;

[0032] The third-party system data type is defined by the dataType field and the Documentum-based document system data type is defined by the dctmType field, both of which are String type by default;

[0033] According to the file transmission attribute defined by the fileConfig field, combining the mapping rule XML file fragment of the attributes and data of the target data, and performing corresponding operations on the target data;

[0034] The fileConfig field defines the file transmission attribute, and supports file uploading of multiple transmission protocols such as FTP, SFTP, HTTP, and SAMBA;

[0035] According to the mapping rule XML file fragment of the attributes and data of the target data configured by the resultMap field;

[0036] The resultMap field, id is the attribute of the target data and the unique identifier of the mapping rule XML file of the data, result is the attribute of the target data and the mapping rule XML file fragment of the data;

[0037] The result field, column is the attribute name of the third-party system data type, property is the attribute name of the Documentum-based document system data type.

[0038] Preferably, the step S3-2 comprises the following steps:

[0039] According to the data of the target data read, the data of the target data is sequentially filled into the corresponding position of the data template from the first data to the last data, forming a complete XML data template of the target data.

[0040] Preferably, the step S3-3 comprises the following steps:

[0041] According to the dictionary content of the target data, the mapping rule XML file fragment corresponding to the dictionary content of the target data is called according to the valueMaps field configuration;

[0042] The valueMaps field contains several valueMap fields, and the mapping rule of the dictionary content of the target data is configured according to the valueMap field;

[0043] The valueMap field, key is the dictionary content of the target data of the third-party system, value is the dictionary content of the target data of the Documentum-based document system.

[0044] Preferably, the step S3-4 comprises the following steps:

[0045] According to the association relationship of the target data, the mapping rule XML file fragment corresponding to the association relationship of the target data is called according to the relations field configuration;

[0046] The relations field contains several relation fields, and the mapping rule of the association relationship of the target data is configured according to the relation field, and each association relationship of the target data is stored in a relation field;

[0047] The name in the relation field is the type of the association relation of the target data, the label is the name of the association relation of the target data, and the identity is the unique identifier of the association relation of the target data.

[0048] Preferably, the step S3-5 comprises the following steps:

[0049] According to the data type of the target data, a mapping rule XML file fragment corresponding to the synchronization range of the target data is called according to the transRules field configuration;

[0050] The transRules field contains a whereClause field, according to the query condition statement of the data type of the target data in the whereClause field, the mapping rule of the first synchronization range of the target data is configured, the query condition statement is contained in the CDATA block, and the first synchronization range of the target data is determined;

[0051] The transRules field contains a transFlag field, according to the transFlag field, the transfer state value of the target data is set, and the mapping rule corresponding to the transfer state value of the target data is configured, which is used to mark the transmission state of the target data;

[0052] The name in the transRules field is the name of the mapping rule of the synchronization range of the target data;

[0053] The dctmType field defines the data type of the target data to be transferred;

[0054] The batchSize field is used to limit the maximum batch size of the target data to be transferred each time.

[0055] Preferably, the step S4-1 further comprises:

[0056] According to the mapping rule of the transfer state value of the target data, a timing time is set, and the target data synchronized successfully is moved into the historical database according to the timing time;

[0057] A delay time is set to retransmit the target data that fails to synchronize;

[0058] The count of the number of retransmissions of the target data that fails to synchronize is set;

[0059] The target data that fails to synchronize within the delay time is not retransmitted; and / or

[0060] The count of the target data retransmission times of the synchronization failure reaches 10 times, and the target data of the synchronization failure is not covered retransmitted; and / or

[0061] The target data of the synchronization failure is covered retransmitted; and / or

[0062] According to the attribute and data of the third-party system target data, a mapping rule XML file of the attribute and data of the corresponding target data is configured.

[0063] The application further provides a data synchronization device applying the data synchronization method, and the data synchronization device comprises:

[0064] An acquisition unit is configured to acquire target data of a third-party system and read the attribute and data of the target data;

[0065] A configuration unit is configured to configure a mapping rule file of the attribute and data of the corresponding target data according to the attribute and data of the target data;

[0066] A synchronization unit is configured to call the data storage general method according to the mapping rule file of the attribute and data of the target data, and perform corresponding operations on the target data.

[0067] Preferably, the data synchronization device further comprises:

[0068] A storage unit is configured to store the acquired target data of the third-party system and the configured mapping rule file of the attribute and data of the corresponding target data;

[0069] The acquisition unit further comprises:

[0070] A first acquisition subunit is configured to read the data of the target data;

[0071] A second acquisition subunit is configured to read the dictionary content of the target data;

[0072] A third acquisition subunit is configured to read the association relationship of the target data;

[0073] A fourth acquisition subunit is configured to read the data type of the target data.

[0074] Preferably, the configuration unit further comprises:

[0075] A first configuration subunit is configured to configure the mapping rule XML file overall structure segment of the target data;

[0076] A second configuration subunit is configured to configure an XML data template of the data of the corresponding target data according to the data of the target data;

[0077] a third configuration subunit configured to configure a mapping rule XML file segment corresponding to the dictionary content of the target data according to the dictionary content of the target data;

[0078] a fourth configuration subunit configured to configure a mapping rule XML file segment corresponding to the association relationship of the target data according to the association relationship of the target data;

[0079] a fifth configuration subunit configured to configure a mapping rule XML file segment corresponding to the synchronization range of the target data according to the data type of the target data;

[0080] a sixth configuration subunit configured to combine the XML data template of the data of the target data and the mapping rule XML file segment of all the attributes of the target data, and combine the mapping rule XML file overall structure segment of the target data, to form a complete mapping rule XML file of the attributes and content of the target data.

[0081] Preferably, the data synchronization device further comprises:

[0082] a construction unit configured to construct a data storage general method based on a Documentum document system;

[0083] the construction unit comprises:

[0084] a first construction subunit configured to construct a data storage general method M1 for synchronizing data from the third-party system to the Documentum document system;

[0085] a second construction subunit configured to construct a data storage general method M2 for synchronizing data from the Documentum document system to the third-party system;

[0086] a third construction subunit configured to construct a data storage general method M3 for uploading files from the third-party system to the Documentum document system;

[0087] a fourth construction subunit configured to construct a data storage general method M4 for uploading files from the Documentum document system to the third-party system.

[0088] Preferably, the synchronization unit further comprises:

[0089] The first synchronization subunit is configured to call the data storage universal method M1 according to the XML data template of the data of the target data and the mapping rule XML file fragment of the attribute of the target data, synchronize the target data from the third-party system to the Documentum-based document system, move the target data synchronized successfully into a historical database, and retransmit the target data synchronized unsuccessfully.

[0090] The second synchronization subunit is configured to call the data storage universal method M3 according to the mapping rule XML file overall structure fragment of the target data, and upload a file from the third-party system to the Documentum-based document system.

[0091] Preferably, the first synchronization subunit further comprises:

[0092] A time unit configured to set a timing time, and move the target data synchronized successfully into a historical database;

[0093] A time unit configured to set a delay time, and retransmit the target data synchronized unsuccessfully;

[0094] A counting unit configured to set a count of the retransmission times of the target data synchronized unsuccessfully;

[0095] A judging unit configured to judge whether the target data synchronized unsuccessfully is retransmitted within the delay time and / or whether the count of the retransmission times of the target data synchronized unsuccessfully reaches 10 times;

[0096] A retransmission unit configured to retransmit the target data synchronized unsuccessfully without covering when the target data synchronized unsuccessfully is not retransmitted within the delay time and / or the count of the retransmission times of the target data synchronized unsuccessfully reaches 10 times; and / or

[0097] retransmit the target data synchronized unsuccessfully with covering; and / or

[0098] According to the attribute and data of the target data of the third-party system, reconfigure the mapping rule XML file of the attribute and data of the corresponding target data.

[0099] The application further provides a computer device, characterized by comprising a memory, a processor and a bus, the memory stores a computer program executable by the processor, the processor and the memory communicate through the bus, and the processor executes the computer program to execute the steps of the method in any one of claims 1-11.

[0100] The application further provides a computer readable storage medium, characterized in that the computer readable storage medium stores a computer program, and the computer program is loaded into a processor to execute the steps of the method according to any one of claims 1-11.

[0101] The technical scheme of the application has the beneficial effects that target data of a third-party system is acquired, attributes and data of the target data of the third-party system are read, a mapping rule file of the attributes and data of the target data is configured according to the attributes and data of the target data, and a data storage general method is called to perform corresponding operations on the target data according to the mapping rule file of the attributes and data of the target data. By configuring the mapping rule file of the target data, the code development amount caused by accessing the third-party system or the change of the data synchronization field of the third-party system and the code change amount caused by business changes are reduced, and the data synchronization between heterogeneous systems is stably and efficiently realized. BRIEF DESCRIPTION OF DRAWINGS

[0102] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings described below only show some of the embodiments of the application, and therefore should not be considered as limiting the scope of protection. For those skilled in the art, other related drawings can also be obtained from these drawings without creative labor.

[0103] Figure 1 is a flow diagram of a data synchronization method provided by the application;

[0104] Figure 2 is a flow diagram of the embodiment step S3 of the data synchronization method provided by the application;

[0105] Figure 3 is a schematic diagram of a data synchronization device provided by the application;

[0106] Figure 4 is a logic structure diagram of a data synchronization device provided by the application;

[0107] Figure 5 is a data diagram of the third-party system target data of the data synchronization method provided by the application. DETAILED DESCRIPTION

[0108] In order to make the purposes and advantages of the embodiments of the present application more easily understood, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. The following detailed description of the embodiments of the present application in the drawings is not intended to limit the scope of the present application, but only represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0109] In order to make the technical solutions in the embodiments of the present application more clearly and completely described, the following will make a basic explanation of the third-party technologies mainly used in the embodiments of the present application:

[0110] (1) Documentum: an enterprise-level content management basic platform, mainly applied to large engineering design fields, used for storage and utilization of enterprise unstructured data, and also can store and utilize unstructured data and structured data on an object.

[0111] (2) XML: a lightweight data storage structure, used for configuration and storage of synchronization plan task rules of the data synchronization device embodiments provided by the present application, configuration and storage of attribute mapping rules of target data, configuration and storage of mapping rules of target data types, and configuration and storage of target data forwarding service rules.

[0112] (3) JAVA reflection mechanism: refers to that in the running state of a program, an object of any class can be constructed, a class to which any object belongs can be understood, member variables and methods of any class can be understood, and attributes and methods of any object can be called; such dynamic acquisition of program information and dynamic calling of objects are called JAVA reflection mechanism.

[0113] (4) Spring Boot: a JAVA-based backend framework, which inherits the original excellent features of the Spring framework, and further simplifies the initial construction and development process of the Spring application by simplifying the configuration. The framework uses a specific way to configure, so that developers no longer need to define the template configuration. In addition, Spring Boot solves the problems of version conflicts of dependent packages and instability of references by integrating a large number of frameworks.

[0114] (5) Bootstrap: A simple, intuitive, powerful front-end development framework based on HTML, CSS, JavaScript, makes the development of Web applications and websites more quickly, Bootstrap provides elegant HTML and CSS specifications, including powerful built-in components, easy to customize.

[0115] In order to facilitate the understanding of the embodiments of the application, the following will take an example of synchronizing the data in an engineering cost table in a third-party system database named "DTS" to a Documentum-based document system database, to illustrate the implementation of the provided data synchronization method, referring to Figure 5 The engineering cost table is named "DTS-PJ", and the engineering cost table contains the pre-estimated price, estimated price, final transaction price and other data of materials such as fine sand, gravel, cement, reinforcing agent and steel bar. Only part of the price data of part of the materials contained in the engineering cost table is shown in the figure to illustrate the implementation of the embodiment; the data synchronization method can also refer to the implementation mode of the embodiment when applied to other third-party systems to synchronize other data types of data.

[0116] Referring to Figure 1 The flowchart of the first embodiment of the data synchronization method provided by the application is shown in the figure. The data synchronization method of the embodiment includes the following steps:

[0117] S1, obtaining target data of a third-party system;

[0118] S2, reading the attributes and data of the target data of the third-party system;

[0119] S3, configuring a mapping rule file corresponding to the attributes and data of the target data according to the attributes and data of the target data;

[0120] S4, according to the mapping rule file of the attributes and data of the target data, calling a data storage general method to perform corresponding operations on the target data.

[0121] In step S1 of the embodiment, the target data of the third-party system is obtained.

[0122] In this step, the Documentum-based document system receives a signal that the target data of the third-party system needs to be synchronized, initiates the Spring Boot backend framework to initialize the configuration of data synchronization, calls the setString, setBool and other method mechanisms, obtains the target data of the third-party system by using the JAVA reflection mechanism, creates a storage file of the data corresponding to the target data of the third-party system, and sets the attribute of the storage file of the data corresponding to the target data of the third-party system in the Documentum-based document system. The storage file is used to store the data corresponding to the target data of the third-party system.

[0123] In this embodiment, the Documentum-based document system receives a signal that the DTS system synchronizes the data of the project cost table, initiates the configuration of the Spring Boot backend framework for the data synchronization of the Documentum-based document system, calls the setString, setBool and other method mechanisms, obtains the project cost table of the DTS system, and stores the project cost table as a project file in the Documentum-based document system. Therefore, a project file corresponding to the project cost table of the DTS system is newly created in the database of the Documentum-based document system, named “DTS-PJ-COPY”, and the attribute of the project cost table in the Documentum-based document system is set. The project file stores the data in the project cost table of the DTS system.

[0124] In step S2 of this embodiment, the attribute and data of the target data of the third-party system are read.

[0125] In this step, after obtaining the target data of the third-party system, the Documentum-based document system analyzes the attribute of the target data set by the setString, setBool and other method mechanisms in advance by using the JAVA reflection mechanism, obtains the attribute of the target data of the third-party system, and dynamically reads the data contained in the target data of the third-party system.

[0126] In this embodiment, after obtaining the project cost table of the DTS system, the Documentum-based document system analyzes the attribute of the project cost table set by the setString, setBool and other method mechanisms in advance by using the JAVA reflection mechanism, obtains the attribute of the project cost table of the DTS system, such as the dictionary content attribute, and the ingredients in the dictionary content include fine sand (unit: m 3 ), gravel (unit: m 3), cement (unit: ton), reinforcing agent (unit: kg), steel bar (unit: ton), etc. The price includes forecast price (unit: yuan), quoted price (unit: yuan), final transaction price (unit: yuan), etc. The data contained in the project cost table of the DTS system is dynamically obtained, for example, 30, 32, 28, 27, 30, 25, etc. corresponding to the corresponding forecast price, quoted price and final transaction price of the material.

[0127] In step S3 of the embodiment, a mapping rule file corresponding to the attributes and data of the target data is configured according to the attributes and data of the target data.

[0128] In this step, according to the obtained target data, the mapping rule file overall structure fragment of the target data is configured based on the Documentum-based document system, according to the read attributes of the target data, the mapping rule file fragment corresponding to the attributes of the target data is configured, according to the read data of the target data, the data template of the data of the target data is configured by filling the data of the target data into the corresponding position according to the template format, the data template of the data of the target data is combined with all the mapping rule file fragments of the attributes of the target data, and the complete mapping rule file of the attributes and data of the target data is formed by combining the mapping rule file overall structure fragment of the target data.

[0129] In this embodiment, based on the Documentum-based document system, the mapping rule file overall structure fragment corresponding to the project cost table is configured according to the obtained project cost table, the mapping rule file fragment corresponding to the dictionary content of the project cost table is configured according to the read dictionary content of the DTS system project cost table, the mapping rule file fragment corresponding to the association relationship of the project cost table is configured according to the read association relationship of the DTS system project cost table, the mapping rule file fragment corresponding to the synchronization range of the project cost table is configured according to the read data type of the DTS system project cost table, the data template of the data of the project cost table is formed by filling the data of the project cost table into the corresponding position according to the template format, the data template of the data of the project cost table is combined with all the mapping rule file fragments of the attributes of the project cost table, and the complete mapping rule file of the attributes and data of the project cost table is formed by combining the mapping rule file overall structure fragment of the project cost table.

[0130] In step S4 of the embodiment, according to the mapping rule file of the attributes and data of the target data, the data storage general method is called to perform corresponding operation on the target data.

[0131] In this step, as Figure 3As shown, based on the attribute of the target data and the mapping rule fragment of different attributes of the target data in the mapping rule file of the data and the data template of the data of the target data, the data storage universal method M1 defined in the Documentum-based document system is called to synchronize the target data from the third-party system to the Documentum-based document system, and based on the overall structure fragment of the mapping rule file of the target data, the data storage universal method M3 defined in the Documentum-based document system is called to upload the file from the third-party system to the Documentum-based document system.

[0132] In this embodiment, based on the attribute of the project cost table and the mapping rule fragment of different attributes of the project cost table in the mapping rule file of the data and the data template of the data of the project cost table, the data storage universal method M1 defined in the Documentum-based document system is called to synchronize the data of the project cost table from the DTS system to the Documentum-based document system, and based on the overall structure fragment of the mapping rule file of the project cost table, the data storage universal method M3 defined in the Documentum-based document system is called to upload the project cost table from the DTS system to the Documentum-based document system.

[0133] The technical scheme of the embodiment is implemented to obtain target data of a third-party system, read the attribute and data of the target data of the third-party system, configure a mapping rule file corresponding to the attribute and data of the target data according to the attribute and data of the target data, and call a data storage universal method to perform corresponding operation on the target data according to the mapping rule file of the attribute and data of the target data. By configuring the mapping rule file of the target data, the code development amount caused by the access to the third-party system or the change of the data synchronization field of the third-party system and the code change amount caused by business changes are reduced, and the data synchronization between heterogeneous systems is stably and efficiently realized.

[0134] Referring to Figure 1 As shown, a flowchart of a second embodiment of a data synchronization method provided by the application is shown, and the data synchronization method of the embodiment comprises the following steps:

[0135] S1, obtaining target data of a third-party system;

[0136] S2, reading the attribute and data of the target data of the third-party system;

[0137] S3, configuring a mapping rule file corresponding to the attribute and data of the target data according to the attribute and data of the target data;

[0138] S4. Based on the attributes of the target data and the data mapping rule file, call the general data storage method to perform the corresponding operation on the target data.

[0139] The difference between this embodiment and the first embodiment of the data synchronization method is that, in step S2, after obtaining the target data from the third-party system, the Documentum-based document system uses the JAVA reflection mechanism to parse the attributes corresponding to the target data that were previously set using methods such as setString and setBool, thus obtaining the attributes of the target data from the third-party system. The attributes of the target data from the third-party system include dictionary content, association relationships, data types, etc., and the data contained in the target data from the third-party system is dynamically read. It can be understood that the dictionary content of the target data represents the target data from the third-party system and the Documentum-based document system that have a mapping relationship. The mapping rules for the dictionary content of the target data are configured according to the dictionary content of the target data to store the target data with mapping relationships. The association relationships of the target data represent the associations between target data. The mapping rules for the association relationships of the target data are configured according to the association relationships of the target data to query the results of various target data associated with the target data. The data type of the target data represents the type of the target data from the third-party system. The mapping rules for the synchronization range of the target data are configured according to the data type of the target data to determine the synchronization range of the target data and mark the transmission status of the target data.

[0140] In this embodiment, see Figure 5 As shown, after obtaining the engineering cost table from the DTS system, the Documentum-based document system uses Java reflection to parse the attributes of the corresponding engineering cost table that were previously set using methods such as setString and setBool. This yields the dictionary contents, relationships, data types, and other attributes of the engineering cost table from the DTS system. For example, the dictionary contents include materials such as fine sand (unit: m³). 3 ), crushed stone (unit: m) 3 The prices include cement (unit: ton), reinforcing agent (unit: kg), steel bars (unit: ton), etc., and include the preliminary price (unit: yuan), the quoted price (unit: yuan), and the final transaction price (unit: yuan). For example, the preliminary price for fine sand is 30 yuan / m³. 3For example, the final transaction price of cement is 115 yuan / ton, and the offer price of steel bar is 520 yuan / ton; the data type is, for example, a project cost table of the DTS system, which is synchronized, and the corresponding storage in the Documentum-based document system database is a project file; the data contained in the project cost table of the DTS system is dynamically acquired, for example, 30, 32, 28, 27, 30, 25, 120, 130, 115, etc., corresponding to the corresponding pre-offer price, offer price and final transaction price of the corresponding material.

[0141] Referring to Figure 1 As shown in the figure, a flowchart of a third embodiment of the data synchronization method provided by the application is shown, and the data synchronization method of the embodiment comprises the following steps:

[0142] S1, acquiring target data of a third-party system;

[0143] S2, reading the attributes and data of the target data of the third-party system;

[0144] S3, configuring a mapping rule file of the attributes and data of the corresponding target data according to the attributes and data of the target data;

[0145] S4, calling a data storage general method according to the mapping rule file of the attributes and data of the target data, and performing corresponding operations on the target data.

[0146] The difference between this embodiment and the first embodiment of the data synchronization method is that, referring to Figure 2 As shown in the figure, after the Documentum-based document system reads the attributes and data of the target data in step S3, the attributes of the target data mainly include dictionary content, association relationship, data type, etc., and the mapping rule XML file of the attributes and data of the corresponding target data is configured, and the following steps are performed:

[0147] S3-1, configuring a mapping rule XML file of the target data, which is used to define data transmission rules and file transmission attributes;

[0148] S3-2, configuring an XML data template of the data of the corresponding target data according to the read data of the target data, which is used to read the data of the target data into the corresponding position of the XML data template;

[0149] S3-3, configuring a mapping rule XML file segment of the dictionary content of the corresponding target data according to the read dictionary content of the target data, which is used to store the target data with mapping relationship;

[0150] S3-4: According to the read target data association relationship, the mapping rule XML file segment corresponding to the target data association relationship is configured, which is used for querying multiple target data results associated with the target data according to the target data association relationship;

[0151] S3-5: According to the read target data type, the mapping rule XML file segment corresponding to the target data synchronization range is configured, which is used to determine the synchronization range of the target data and mark the transmission state of the target data, and the mapping rule XML file segment of the synchronization range of the target data includes: the first synchronization range segment of the target data and the transmission state value segment of the target data;

[0152] S3-6: Jointly target data XML data template and all target data contained attribute mapping rule XML file segment, combined with the mapping rule XML file overall structure segment of the target data, form the complete attribute and data mapping rule XML file of the target data.

[0153] In this embodiment, after obtaining the attributes and data of the DTS system engineering cost table, the attributes of the engineering cost table mainly include dictionary content, association relationship, data type, etc.; the Documentum-based document system configures the mapping rule XML file overall structure segment of the DTS system engineering cost table; according to the read engineering cost table data, the data in the engineering cost table is filled into the corresponding position according to the template format, and the XML data template corresponding to the engineering cost table data is configured; according to the read dictionary content of the engineering cost table, the mapping rule XML file segment corresponding to the dictionary content of the engineering cost table is configured; according to the read association relationship of the engineering cost table, the mapping rule XML file segment corresponding to the association relationship of the engineering cost table is configured; according to the read data type of the engineering cost table, the mapping rule XML file segment corresponding to the synchronization range of the engineering cost table is configured; Jointly engineering cost table XML data template and all engineering cost table attribute mapping rule XML file segment, combined with the mapping rule XML file overall structure segment of the engineering cost table, form the complete attribute and data mapping rule XML file of the engineering cost table.

[0154] Further, for example, in the third embodiment, step S3-1 includes:

[0155] According to the third-party system data type defined by the dataType field, the mapping rule XML file segment corresponding to the synchronization range of the target data is configured by calling the resultMap field;

[0156] The third-party system data type is defined by the dataType field and the Documentum-based document system data type is defined by the dctmType field, both of which are by default String type;

[0157] According to the file transmission attribute defined by the fileConfig field, the attribute and data mapping rule XML file segment of the target data, and the corresponding operation of the target data is performed;

[0158] The file transmission attribute is defined by the fileConfig field, and file uploading of multiple transmission protocols such as FTP, SFTP, HTTP and SAMBA is supported;

[0159] The attribute and data mapping rule XML file segment of the target data is configured according to the resultMap field;

[0160] In the resultMap field, the id is the unique identifier of the attribute and data mapping rule XML file of the target data, and the result is the attribute and data mapping rule XML file segment of the target data;

[0161] In the result field, the column is the attribute name of the third-party system data type, and the property is the attribute name of the Documentum-based document system data type.

[0162] The specific implementation of step S3-1 in the embodiment is as follows:

[0163]

[0164] In the above XML file, <?xml version="1.0"encoding="UTF-8"standalone="yes"?> represents the header file attribute of the XML, which explains the format of the XML file; wherein version represents the version, encoding represents the character encoding mode, and standalone indicates whether the XML file is independent or dependent on an externally defined dtd file, and the value yes indicates that the XML document is self-contained (self-contained);

[0165] The DTS system data type is defined by the dataType field and the Documentum-based document system data type is defined by the dctmType field, both of which are by default String type; according to the DTS system data type defined by the dataType field, the mapping rule of the synchronization range of the corresponding project cost table is configured by calling the resultMap field;

[0166] The file transmission attribute is defined by the fileConfig field, and the file uploading of multiple transmission protocols such as FTP, SFTP, HTTP and SAMBA is supported; according to the file transmission attribute defined by the fileConfig field, the transmission rule, file transmission attribute and storage mode of the project cost table are determined;

[0167] The attribute and data mapping rule XML file fragment of the project cost table is configured according to the resultMap field; in the resultMap field, the id field is the unique identifier of the attribute and data mapping rule XML file of the project cost table, which is distinguished from the ordinary result field, and is used to set the default uploading path of the file;

[0168] The result field is the attribute and data mapping rule XML file fragment of the project cost table, and the attribute mapping rule XML file fragment of the project cost table mainly includes: the mapping rule of the dictionary content of the project cost table, the mapping rule of the association relationship of the project cost table and the mapping rule of the data type of the project cost table, and the data mapping rule XML file fragment of the project cost table includes the XML data template of the data of the project cost table;

[0169] In the result field, the column is the attribute name of the DTS system data type, and the property is the attribute name of the Documentum-based document system data type.

[0170] Further, for example, in the third embodiment, step S3-2 includes:

[0171] According to the data of the target data read, the data of the target data is sequentially filled into the corresponding position of the data template from the first data to the last data, to form a complete XML data template of the target data.

[0172] In this embodiment, as shown in Figure 5 The data of the project cost table read has 30, 32, 28, 27, 30, 25, 120, 130, 115, 30, 35, 28, 480, 520, 450, etc., and is sequentially filled into the corresponding position of the XML data template according to the format of the corresponding XML data template of the project cost table configured by the Documentum-based document system, to form a complete XML data template of the project cost table.

[0173] The third-party system XML data template example is as follows (taking an XML data template of a third-party system named DTS as an example):

[0174]

[0175]

[0176]

[0177]

[0178] When the third-party system uses other data transmission templates or other file transmission methods, we only need to modify the dataType configuration (for example, com.cnpdc.adds.pojo.SynDataLetterFromXml) and fileConfig configuration in the mapping rule file of the properties and data of the target data, without repeating the implementation of the data archiving logic.

[0179] Further, for example, in the third embodiment, step S3-3 includes:

[0180] According to the dictionary content of the target data, a mapping rule XML file segment corresponding to the dictionary content of the target data is called based on the valueMaps field configuration;

[0181] The valueMaps field contains several valueMap fields, and the mapping rule of the dictionary content of the target data is configured according to the valueMap field;

[0182] In the valueMap field, the key is the dictionary content of the target data of the third-party system, and the value is the dictionary content of the target data of the Documentum-based document system.

[0183] The specific implementation of step S3-3 of this embodiment is as follows:

[0184]

[0185] In the document system based on Documentum, the dictionary content of the target data is defined in the result field by column and property, respectively, to define the name of the dictionary content of the target data of the third-party system and the name of the dictionary content of the target data of the document system based on Documentum, and then dctmDefaultValue and dataDefaultValue are declared to declare the default value of the dictionary content of the target data of the third-party system and the dictionary content of the target data of the document system based on Documentum, respectively, to define the transmission mode of the dictionary content of the target data, and finally, the mapping rule of the dictionary content of the target data is configured in several valueMap fields included in the valueMaps field, in which the key is the dictionary content of the target data of the third-party system and the value is the dictionary content of the target data of the document system based on Documentum, and the key and the value correspond to each other and have a mapping relationship.

[0186] In the embodiment, the key in the DTS system engineering cost table includes material content: fine sand (unit: m 3 ), gravel (unit: m 3 ), cement (unit: ton), enhancer (unit: kg), rebar (unit: ton) and the like, and price content: expected price (unit: yuan), price (unit: yuan), final transaction price (unit: yuan) and the like; the value of the engineering cost table based on Documentum is defined as material content: Fine_Sand (m 3 ), Gravel (m 3 ), Cement (Ton), Enhancer (kg), Rebar (Ton) and the like, and price content: Expected_Price (CNY), Price (CNY), Final_Price (CNY) and the like, so it can be seen that the key in the DTS system engineering cost table and the value of the engineering cost table based on Documentum correspond to each other and have a mapping relationship.

[0187] Further, for example, in the third embodiment, the step S3-4 includes:

[0188] According to the association relationship of the target data, the mapping rule XML file fragment of the association relationship of the corresponding target data is called by the relations field;

[0189] The relations field contains several relation fields, and the mapping rule of the association relation of the corresponding target data is configured according to the relation field. The association relation of each target data is stored in a relation field.

[0190] The specific implementation of step S3-4 in this embodiment is as follows:

[0191]

[0192] In the relation field, name defines the type of the association relation of the target data, label defines the name of the association relation of the target data, and finally identity gives the unique identifier of the association relation of the target data.

[0193] The above specific implementation only shows part of the association relations. Among them, name="ecm_attach_rel" indicates that the type of the association relation between the data of the project cost table of the DTS system and the data of the Documentum-based document system is ecm_attach_rel, label="Relevant" indicates that the name of the association relation is Relevant, <identity>Cement-Final_Price< / identity> In the field, identity gives the unique identifier of the association relation between the data of the project cost table of the DTS system and the data of the Documentum-based document system, and Cement-Cement-Final_Price is the identifier name of the association relation. The data belonging to this association relation can be queried through the unique identifier.

[0194] Further, for example, in the third embodiment, step S3-5 includes:

[0195] According to the data type of the target data, the mapping rule XML file fragment of the synchronization range of the corresponding target data is called according to the transRules field;

[0196] In the transRules field, the whereClause field is contained, and the mapping rule of the first synchronization range of the target data is configured according to the query condition statement of the data type of the target data of the whereClause field. The query condition statement is contained in the CDATA block, which is used to determine the first synchronization range of the target data.

[0197] In the transRules field, the transFlag field is contained, the transfer state value of the target data is set according to the transFlag field, and the mapping rule of the transfer state value of the corresponding target data is configured, which is used to mark the transmission state of the target data.

[0198] The specific implementation of step S3-5 in this embodiment is as follows:

[0199]

[0200]

[0201]

[0202] The mapping rule of the synchronization range of the project cost table and the counter-write rule are configured through the transRules field in the transRuleConfig, each transRules contains a name attribute, that is, the name of the mapping rule of the synchronization range of the project cost table, to distinguish different mapping rules of the synchronization range of the project cost table, contains a dctmType field to determine the data type of the transferred project cost table, contains a whereClause field to determine the query condition statement of the data type of the project cost table, since there are often special characters in the XML field in the query statement, therefore, the content of this field is contained in the CDATA block, contains a batchSize field to limit the maximum batch size of the project cost table each time, to avoid system performance problems caused by too large quantity of one-time transmission, contains a transFlag field to set different state values after the transmission is completed, for moving the synchronized project cost table into the historical database, retransmitting the project cost table of the synchronization failure, and cooperating with the data synchronization real-time monitoring system to view and process the project cost table of the synchronization failure.

[0203] Further, for example, in the third embodiment, step S3-6 includes:

[0204] In the mapping rule XML file overall structure fragment of the configured target data, the XML data template of the data of the configured target data and the mapping rule XML file fragment of each attribute contained by the target data are filled in, to form the mapping rule XML file of the attributes and data of the target data.

[0205] In this embodiment, in the mapping rule XML file overall structure fragment of the configured project cost table, the XML data template of the data of the configured project cost table and the mapping rule XML file fragment of the dictionary content, the association relationship, the data type and other attributes of the project cost table are filled in, to form the mapping rule XML file of the attributes and data of the project cost table.

[0206] Referring to Figure 1 As shown in the figure, a flowchart of a fourth embodiment of a data synchronization method provided by the application is shown, the data synchronization method of this embodiment includes the following steps:

[0207] S1, obtaining target data of a third-party system;

[0208] S2, reading the attribute and data of the target data of the third-party system;

[0209] S3, configuring a mapping rule file of the attribute and data of the corresponding target data according to the attribute and data of the target data;

[0210] S4, calling a data storage general method according to the mapping rule file of the attribute and data of the target data to perform corresponding operation on the target data.

[0211] The embodiment differs from the first embodiment of the data synchronization method in that, in step S4, the following steps are included:

[0212] S4-1: calling the data storage general method M1 to synchronize the target data from the third-party system to the Documentum-based document system according to the mapping rule XML file segment of the attribute of the target data and the XML data template of the data of the target data, moving the successfully synchronized target data into a history database, and retransmitting the unsuccessfully synchronized target data;

[0213] The history database is a document database for persistently storing the successfully synchronized target data of the Documentum-based document system;

[0214] S4-2: calling the data storage general method M3 to upload the file from the third-party system to the Documentum-based document system according to the mapping rule XML file overall structure segment of the target data.

[0215] In this embodiment, the Documentum-based document system parses the mapping rule XML file of the attribute and data of the project cost table, determines the transmission rule, file transmission attribute, and storage mode of the project cost table according to the parsed mapping rule XML file overall structure segment of the project cost table, calls the data storage general method M3 to upload the file from the DTS system to the Documentum-based document system, and calls the data storage general method M1 to synchronize the project cost table from the DTS system to the Documentum-based document system according to the parsed XML data template of the data of the project cost table and the mapping rule XML file segment of the attribute of the project cost table according to the corresponding data template format and the corresponding mapping rule. In addition, the successfully synchronized project cost table is moved into the history database, and the unsuccessfully synchronized project cost table is retransmitted. The history database is a document database for persistently storing the successfully synchronized target data of the Documentum-based document system.

[0216] Referring to Figure 1As shown, a flow diagram of a fifth embodiment of a data synchronization method provided by the application is shown. The data synchronization method of this embodiment includes the following steps:

[0217] S1, obtaining target data of a third-party system;

[0218] S2, reading the attributes and data of the target data of the third-party system;

[0219] S3, configuring a mapping rule file corresponding to the attributes and data of the target data according to the attributes and data of the target data;

[0220] S4, according to the mapping rule file of the attributes and data of the target data, calling a data storage general method to perform corresponding operations on the target data.

[0221] The difference between this embodiment and the fourth embodiment of the data synchronization method is that in step S4-1, the following steps are included:

[0222] According to the mapping rule of the delivery state value of the target data, a timing time is set, and the target data that is successfully synchronized is moved into the historical database according to the timing time;

[0223] A delay time is set to retransmit the target data that fails to synchronize;

[0224] The number of retransmissions of the target data that fails to synchronize is counted;

[0225] The target data that fails to synchronize within the delay time is not retransmitted; and / or

[0226] When the number of retransmissions of the target data that fails to synchronize reaches 10 times, the target data that fails to synchronize is not retransmitted; and / or

[0227] The target data that fails to synchronize is retransmitted; and / or

[0228] According to the attributes and data of the target data of the third-party system, the mapping rule XML file corresponding to the attributes and data of the target data is reconfigured.

[0229] In this embodiment, a matching data synchronization real-time monitoring system is started, and during the operation of the real-time monitoring system, the project cost table to be synchronized is divided into a historical data table and a running data table. The running data table performs a project cost table synchronization operation, a timing time is set, the project cost table that is successfully synchronized is moved into the historical data table according to the timing time, the delivery state of the project cost table to be synchronized is monitored, and the number of retransmissions of the project cost table that fails to synchronize is counted.

[0230] According to the transmission state of the project cost table, the project cost table synchronized successfully is transferred to the historical database at a fixed time; a delay time is set, and within the delay time, it is judged whether the project cost table synchronized unsuccessfully is retransmitted or not and / or whether the count of the retransmission times of the project cost table synchronized unsuccessfully reaches 10 times; if not, the corresponding data storage general method is called, the project cost table synchronized unsuccessfully is retransmitted every half an hour, and the retransmission times of the project cost table synchronized unsuccessfully are counted; when the retransmission times reach 10 times, the retransmission is stopped, the document monitoring management personnel manually judges the reason of the transmission error, and the project cost table synchronized unsuccessfully is retransmitted with or without coverage; or the mapping rule XML file of the corresponding project cost table is reconfigured according to the attribute and data of the DTS system project cost table.

[0231] The real-time monitoring system provides the function of modifying the mapping rule XML file, helps the document monitoring management personnel to quickly solve the problem, and initiates the retransmission of the project cost table; the modification of the mapping rule XML file needs permission, and the project cost table is generally retransmitted with coverage from the source.

[0232] Referring to Figure 3 The principle diagram of the data synchronization device provided by the application is shown in the figure, in some embodiments of the data synchronization device provided by the application, the data general storage method comprises:

[0233] M1: data storage general method of synchronizing data from a third-party system to a Documentum-based document system;

[0234] M2: data storage general method of synchronizing data from the Documentum-based document system to the third-party system;

[0235] M3: data storage general method of uploading files from the third-party system to the Documentum-based document system;

[0236] M4: data storage general method of uploading files from the Documentum-based document system to the third-party system.

[0237] In some embodiments, according to the XML data template of the data of the target data and the mapping rule XML file segment of the attribute of the target data, the data storage general method M1 is called to synchronize the target data from the third-party system to the Documentum-based document system, the target data synchronized successfully is moved to the historical database, and the target data synchronized unsuccessfully is retransmitted;

[0238] According to the data type, data transmission rule, file transmission attribute and storage mode of the target data defined in the mapping rule XML file of the target data, a data storage general method M3 is called to upload the file from the third-party system to the Documentum-based document system.

[0239] In the embodiment, after the Documentum-based document system configures the mapping rule XML file of the attributes and data of the project cost table, according to the overall structure fragment of the mapping rule XML file of the project cost table, the data storage general method M3 defined in the Documentum-based document system is called to upload the project cost table from the DTS system to the Documentum-based document system; according to the mapping rule XML file fragment of the attributes of the dictionary content, association relationship and data type contained in the project cost table and the XML data template of the data of the project cost table, the data storage general method M1 defined in the Documentum-based document system is called to synchronize the data of the project cost table from the DTS system to the Documentum-based document system, the project cost table successfully synchronized is moved to the historical database, and the project cost table failed to be synchronized is retransmitted.

[0240] Referring to Figure 4 As shown in the figure, the logic structure diagram of some embodiments of the data synchronization device provided by the application specifically includes:

[0241] The acquisition unit is configured to acquire the target data of the third-party system and read the attributes and data of the target data.

[0242] The configuration unit is configured to configure a mapping rule file of the attributes and data of the corresponding target data according to the attributes and data of the target data.

[0243] The synchronization unit is configured to call a data storage general method according to the mapping rule file of the attributes and data of the target data, and perform corresponding operations on the target data.

[0244] The storage unit is configured to store the data of the target data of the third-party system acquired and the mapping rule file of the attributes and data of the corresponding target data configured.

[0245] Further, in some embodiments, the acquisition unit further includes:

[0246] The first acquisition subunit is configured to read the data of the target data.

[0247] The second acquisition subunit is configured to read the dictionary content of the target data.

[0248] The third acquisition subunit is configured to read the association relationship of the target data.

[0249] The fourth obtaining sub-unit is configured to read the data type of the target data.

[0250] Further, in some embodiments, the configuration unit further comprises:

[0251] The first configuration sub-unit is configured to configure the mapping rule XML file overall structure segment of the target data;

[0252] The second configuration sub-unit is configured to configure the XML data template of the corresponding target data according to the data of the target data;

[0253] The third configuration sub-unit is configured to configure the mapping rule XML file segment of the corresponding target data dictionary content according to the dictionary content of the target data;

[0254] The fourth configuration sub-unit is configured to configure the mapping rule XML file segment of the corresponding target data association relationship according to the association relationship of the target data;

[0255] The fifth configuration sub-unit is configured to configure the mapping rule XML file segment of the corresponding target data synchronization range according to the data type of the target data;

[0256] The sixth configuration sub-unit is configured to combine the XML data template of the target data and the mapping rule XML file segment of all attributes of the target data, and combine the mapping rule XML file overall structure segment of the target data, to form a complete mapping rule XML file of the attributes and content of the target data.

[0257] Further, in some embodiments, the data synchronization device further comprises:

[0258] The construction unit is configured to construct a data storage general method based on the Documentum document system;

[0259] The construction unit comprises:

[0260] The first construction sub-unit is configured to construct a data storage general method M1 for synchronizing data from a third-party system to the Documentum document system;

[0261] The second construction sub-unit is configured to construct a data storage general method M2 for synchronizing data from the Documentum document system to a third-party system;

[0262] The third construction sub-unit is configured to construct a data storage general method M3 for uploading files from a third-party system to the Documentum document system;

[0263] A fourth configuration subunit is configured to configure a data storage universal method M4 for uploading files from the Documentum-based document system to the third-party system.

[0264] Further, in some embodiments, the synchronization unit further comprises:

[0265] A first synchronization subunit is configured to invoke the data storage universal method M1 according to the XML data template of the data of the target data and the mapping rule XML file fragment of the attributes of the target data, synchronize the target data from the third-party system to the Documentum-based document system, move the successfully synchronized target data into the historical database, and retransmit the failed synchronized target data.

[0266] A second synchronization subunit is configured to invoke the data storage universal method M3 according to the mapping rule XML file overall structure fragment of the target data, upload files from the third-party system to the Documentum-based document system.

[0267] Further, in some embodiments, the first synchronization subunit further comprises:

[0268] A time unit is configured to set a timing time, and move the successfully synchronized target data into the historical database.

[0269] A time unit is configured to set a delay time, and retransmit the failed synchronized target data.

[0270] A counting unit is configured to set a count of the retransmission times of the failed synchronized target data.

[0271] A judgment unit is configured to judge whether the failed synchronized target data is retransmitted within the delay time and / or whether the count of the retransmission times of the failed synchronized target data reaches 10 times.

[0272] A retransmission unit is configured to retransmit the failed synchronized target data without covering when the failed synchronized target data is not retransmitted within the delay time and / or the count of the retransmission times of the failed synchronized target data reaches 10 times; and / or

[0273] retransmit the failed synchronized target data with covering; and / or

[0274] Reconfigure the mapping rule XML file of the attributes and data of the corresponding target data according to the attributes and data of the target data of the third-party system.

[0275] In some embodiments of the data synchronization device provided by the application, the data synchronization device can include an acquisition unit, a configuration unit, a synchronization unit, a storage unit, and a construction unit, etc. The acquisition unit can include a first acquisition subunit, a second acquisition subunit, a third acquisition subunit, and a fourth acquisition subunit. The configuration unit can include a first configuration subunit, a second configuration subunit, a third configuration subunit, a fourth configuration subunit, a fifth configuration subunit, and a sixth configuration subunit. The construction unit can include a first construction subunit, a second construction subunit, a third construction subunit, and a fourth construction subunit. The synchronization unit can include a first synchronization subunit and a second synchronization subunit. The first synchronization subunit can further include a time unit, a counting unit, a judgment unit, and a retransmission unit.

[0276] In step S1, the acquisition unit acquires target data of a third-party system. In some embodiments, based on the signal that the Documentum-based document system receives that the target data of the third-party system needs to be synchronized, the Spring Boot backend framework is started to initialize the configuration of data synchronization, the setString, setBool, and other method mechanisms are called, the JAVA reflection mechanism is used, the acquisition unit acquires the target data of the third-party system, creates a storage file of the data corresponding to the target data of the third-party system, and sets the properties of the storage file of the data corresponding to the target data of the third-party system in the Documentum-based document system. The storage file is used to store the data corresponding to the target data of the third-party system.

[0277] For example, based on the signal that the Documentum-based document system receives that the DTS system synchronizes the data of the project cost table, the Spring Boot backend framework is started to initialize the configuration of the data synchronization of the Documentum-based document system, the setString, setBool, and other method mechanisms are called, the acquisition unit acquires the project cost table of the DTS system, the project cost table is stored as a project file in the Documentum-based document system, a project file corresponding to the project cost table of the DTS system is newly created in the database thereof, named “DTS-PJ-COPY”, and the properties of the project cost table in the Documentum-based document system are set. The project file stores the data in the project cost table of the DTS system.

[0278] In step S2, the acquisition unit reads the attributes and data of the target data of the third-party system. In some embodiments, after acquiring the target data of the third-party system, the Documentum-based document system uses JAVA reflection mechanism to analyze the attributes of the target data set by the setString, setBool and other method mechanisms, the first acquisition subunit reads the data of the target data of the third-party system, the second acquisition subunit reads the dictionary content of the target data of the third-party system, the third acquisition subunit reads the association of the target data of the third-party system, and the fourth acquisition subunit reads the data type of the target data of the third-party system.

[0279] For example, after acquiring the project cost table of the DTS system, the Documentum-based document system uses JAVA reflection mechanism to analyze the attributes of the project cost table set by the setString, setBool and other method mechanisms, the first acquisition subunit acquires the data contained in the project cost table of the DTS system, such as the corresponding forecast price, price and final transaction price of the corresponding materials, such as 30, 32, 28, 27, 30 and 25; the second acquisition subunit reads the dictionary content of the project cost table, the ingredients in the dictionary content contain fine sand (unit: m 3 ), gravel (unit: m 3 ), cement (unit: ton), reinforcing agent (unit: kg), steel bar (unit: ton) and the like, and the price contains forecast price (unit: yuan), price (unit: yuan), final transaction price (unit: yuan) and the like; the third acquisition subunit reads the association of the project cost table, for example, the forecast price of fine sand is 30 yuan / m 3 , the final transaction price of cement is 115 yuan / ton, and the price of steel bar is 520 yuan / ton; and the fourth acquisition subunit reads the data type of the project cost table, for example, it is a project cost table of the DTS system, and the corresponding storage in the database of the Documentum-based document system is a project file, and for another example, it is an invitation letter for celebrating the 50th anniversary of the establishment of the DTS company, and the corresponding storage in the database of the Documentum-based document system is a letter.

[0280] In step S3, the configuration unit configures the mapping rule file of the attribute and data of the target data according to the attribute and data of the target data. In some embodiments, the configuration unit configures the mapping rule file of the target data according to the obtained target data, configures the mapping rule file of the data of the target data according to the read data of the target data, configures the mapping rule file of the attribute of the target data according to the read attribute of the target data, combines the data template of the data of the target data and the mapping rule file of the attribute of the target data, combines the mapping rule file of the target data, and forms the mapping rule file of the attribute and data of the target data.

[0281] For example, the configuration unit configures the mapping rule XML file of the attribute and data of the target data according to the obtained engineering cost table of the DTS system, the attribute of the engineering cost table mainly includes dictionary content, association relationship, data type, etc.; the first configuration subunit configures the mapping rule XML file of the attribute and data of the target data according to the obtained engineering cost table of the DTS system, the second configuration subunit configures the mapping rule XML file of the attribute and data of the target data according to the read data of the engineering cost table of the DTS system, the third configuration subunit configures the mapping rule XML file of the attribute and data of the target data according to the read dictionary content of the engineering cost table of the DTS system, the fourth configuration subunit configures the mapping rule XML file of the attribute and data of the target data according to the read association relationship of the engineering cost table of the DTS system, the fifth configuration subunit configures the mapping rule XML file of the attribute and data of the target data according to the read data type of the engineering cost table of the DTS system, and the sixth configuration subunit configures the mapping rule XML file of the attribute and data of the target data according to the read data type of the engineering cost table of the DTS system. The sixth configuration subunit combines the data template of the data of the engineering cost table and the mapping rule XML file of the attribute of the engineering cost table, combines the mapping rule XML file of the engineering cost table, and forms the mapping rule XML file of the attribute and data of the engineering cost table.

[0282] In step S4, the synchronization unit calls a general data storage method to perform corresponding operations on the target data based on the attributes of the target data and the data mapping rule file. In some embodiments, the construction unit is used to construct a general data storage method based on the Documentum document system; wherein, the first construction subunit is used to construct a general method M1 for synchronizing data from a third-party system to data storage based on the Documentum document system, the second construction subunit is used to construct a general method M2 for synchronizing data from the Documentum document system to data storage from a third-party system, the third construction subunit is used to construct a general method M3 for uploading files from a third-party system to data storage based on the Documentum document system, and the fourth construction subunit is used to construct a general method M4 for uploading files from the Documentum document system to data storage from a third-party system.

[0283] like Figure 3 As shown, in some embodiments, the first synchronization subunit is used to call the general data storage method M1 to synchronize the target data from the third-party system to the Documentum-based document system based on the attributes of the target data, the mapping rule file fragments of different attributes of the target data in the data mapping rule file, and the data template of the target data. The target data that is successfully synchronized is moved into the historical database, and the target data that fails to be synchronized is retransmitted. The second synchronization subunit is used to call the general data storage method M3 to upload the file from the third-party system to the Documentum-based document system based on the overall structure fragment of the mapping rule file of the target data.

[0284] Furthermore, in some embodiments, the first synchronization subunit further includes:

[0285] The time unit is used to set a time interval to move the successfully synchronized target data into the historical database.

[0286] Used to set a delay time for retransmission of target data that fails to synchronize;

[0287] The counting unit is used to set the count of the number of retransmissions of the target data in case of synchronization failure;

[0288] The judgment unit is used to determine whether the target data that failed to synchronize within the delay time has been retransmitted and / or whether the count of the number of retransmissions of the target data that failed to synchronize has reached 10.

[0289] The retransmission unit is used to retransmit the target data that failed to synchronize within the delay time without overwriting it, and / or when the count of retransmissions of the target data that failed to synchronize reaches 10; and / or

[0290] overwriting retransmission of the target data of the synchronization failure; and / or

[0291] reconfiguring the mapping rule XML file of the attributes and data of the target data according to the attributes and data of the third-party system target data.

[0292] For example, in some embodiments, the synchronization unit parses the mapping rule XML file of the attributes and data of the project cost table, the first synchronization sub-unit calls the data storage general method Ml according to the parsed XML data template of the project cost table data and the mapping rule XML file segment of the attributes of the project cost table, and synchronizes the project cost table from the third-party system to the Documentum-based document system according to the corresponding data template format and the corresponding mapping rule, moves the synchronized successful project cost table into the historical database, and performs retransmission on the project cost table of the synchronization failure, wherein the historical database is a document database for persistently storing the synchronized successful target data of the Documentum-based document system; the second synchronization sub-unit determines the transmission rule, file transmission attribute, and storage mode of the project cost table according to the parsed mapping rule XML file overall structure segment of the project cost table, and calls the data storage general method M3 to upload the file from the DTS system to the Documentum-based document system.

[0293] For another example, in some embodiments, the first synchronization sub-unit starts the matching data synchronization real-time monitoring system, and during the working process of the real-time monitoring system, divides the project cost table to be synchronized into the historical data table and the running data table, performs the project cost table synchronization operation on the running data table, uses the time unit to set the timing time, periodically moves the synchronized successful project cost table into the historical data table according to the timing time, monitors the transmission state of the project cost table to be synchronized, and starts the counting unit to count the retransmission times of the project cost table of the synchronization failure.

[0294] According to the transmission state of the project cost table, periodically transfers the synchronized successful project cost table to the historical database; the time unit is used to set the delay time, and during the delay time, the judging unit judges whether the project cost table of the synchronization failure has retransmission or not and / or whether the count of the retransmission times of the project cost table of the synchronization failure reaches 10 times or not; if not, calls the corresponding data storage general method, retransmits the project cost table of the synchronization failure every half an hour, and when the retransmission times reach 10 times, stops the retransmission, the document monitoring management personnel manually judges the reason of the transmission error, starts the retransmission unit to perform the overwriting retransmission or non-overwriting retransmission on the project cost table of the synchronization failure; or reconfigures the mapping rule XML file of the attributes and data of the corresponding project cost table according to the attributes and data of the project cost table of the DTS system.

[0295] The real-time monitoring system provides a function of modifying the mapping rule XML file, helps the document monitoring manager to quickly solve problems, and initiates retransmission of the project cost table, and modification of the mapping rule XML file needs authority, and generally, the project cost table is retransmitted from the source.

[0296] Referring to Figure 3 As shown in the principle diagram of the data synchronization device provided by the application, specifically comprises:

[0297] The A system, the B system, the C system and the D system are third-party systems, the A1 data type mapping file, the B1 data type mapping file, the C1 data type mapping file and the D1 data type mapping file are mapping rule XML files of properties and data of corresponding target data according to properties and data configurations of the target data of different third-party systems, the BaseDao is a running data table of a Documentum-based document system, and the Documentum Content Server is a historical database of the Documentum-based document system, belongs to the field of engineering design, defines different data types such as engineering files, letters, documents, archives, contracts and invoices according to business definitions, the copyFromSourceToDctm is a defined data storage general method M1, the copyFromDctmToTarget is a defined data storage general method M2, the uploadFile is a defined data storage general method M3, and the setFile is a defined data storage general method M4.

[0298] The working principle of the data synchronization device provided by the application for synchronizing target data of the A1 system to the Documentum-based document system will be briefly described below.

[0299] The Documentum-based document system of the embodiment first receives a signal that the target data of the A1 system needs to be synchronized, starts a Spring Boot backend framework for data synchronization initialization configuration, calls a setString, a setBool and other method mechanisms, acquires the target data of the third-party system by using a JAVA reflection mechanism, creates a storage file of the data corresponding to the target data of the third-party system in the BaseDao, and sets properties of the storage file of the data corresponding to the target data of the Documentum-based document system, the storage file being used for storing the data corresponding to the target data of the third-party system.

[0300] After the target data of the third-party system is acquired, the JAVA reflection mechanism is used to parse the attributes of the target data corresponding to the attributes set by the setString, setBool and other method mechanisms based on the Documentum document system, to obtain the attributes of the target data of the A1 system in the BaseDao, mainly including the dictionary content, the association relationship and the data type, and the like. Understandably, the dictionary content of the target data is used to store the target data with the mapping relationship, the association relationship of the target data is used to query the multiple target data results associated with the target data according to the association relationship of the target data, and the data type of the target data is used to configure the mapping rule of the synchronization range and to determine the synchronization range of the target data and the transmission state of the target data according to the mapping rule, and to dynamically read the data contained in the target data in the BaseDao.

[0301] Then, according to the acquired target data, the overall structure fragment of the mapping rule XML file is configured to determine the basic configuration framework of the entire XML file, the data transmission rule, the file transmission attribute and the storage mode; the data of the target data of the A1 system in the BaseDao is filled into the corresponding position in the XML data template according to the template format in sequence to form the XML data template of the corresponding target data; according to the attributes of the target data of the A1 system in the BaseDao, mainly including the dictionary content, the association relationship and the data type, the mapping rule XML file fragment of the dictionary content, the association relationship and the data type of the corresponding target data is configured according to the dictionary content, the association relationship and the data type attributes of the target data in sequence; the XML data template of the data of the target data and the mapping rule XML file fragment of the attributes contained in the target data are combined, and the mapping rule XML file of the attributes and the data of the target data of the A1 system is formed by combining the overall structure fragment of the mapping rule XML file of the target data.

[0302] Finally, according to the attributes of the target data of the A1 system and the mapping rule XML file of the target data in the mapping rule XML file of the target data of the A1 system, the data template of the data of the target data is called to the data storage general method M1 defined in the Documentum-based document system, the target data is synchronized from the A1 system to the Documentum-based document system, the target data synchronized successfully is moved to the Documentum ContentServer, and the target data synchronized unsuccessfully is retransmitted; and according to the overall structure fragment of the mapping rule file of the target data, the data storage general method M3 defined in the Documentum-based document system is called to upload the file from the A1 system to the Documentum-based document system.

[0303] The application further provides a computer device, comprising a memory, a processor and a bus, the memory stores a computer program executable by the processor, when the computer device is running, the processor communicates with the memory through the bus, the processor executes the computer program to execute the steps of any of the above methods or any possible implementation of any of the methods.

[0304] The application further provides a computer readable storage medium, the computer readable storage medium stores a computer program, the processor executes the computer program to execute the steps of any of the above methods or any possible implementation of any of the methods.

[0305] It can be understood by those skilled in the art that the embodiments provided by the application are only illustrative, the writing order of the steps in the method of the embodiments does not mean strict execution order and constitute any limitation on the implementation process, and the order can be adjusted, combined and deleted according to actual needs, the modules or sub-modules, units or sub-units in the device of the embodiments can be combined, divided and deleted according to actual needs. For example, the division of the units is only a logical function division, and another division mode can be used in actual implementation, for example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0306] It can be understood by those skilled in the art that all or part of the steps of the method described in the embodiments provided by the application can be directly implemented by using electronic hardware or a computer program executable by a processor, or a combination of the two. The computer program can be stored in a random access memory (RAM), a read-only memory (ROM), an internal memory, an electrically programmable ROM, an electrically erasable programmable ROM, a CD-ROM, a register, a hard disk, a removable disk, or any other form of storage medium known in the art.

[0307] It should be noted that the above embodiments are only for illustrating the technical concept and characteristics of the application, the purpose is to enable those skilled in the art to understand the content of the application and implement it accordingly, and it cannot be used to limit the protection scope of the application, and those skilled in the art can understand that the implementation of all or part of the above embodiments is based on the equivalent changes and modifications of the claims of the application, and should belong to the scope of the claims of the application.

Claims

1. A method of data synchronization, the method comprising: The method comprises the following steps: S1, obtaining target data of a third-party system; S2, reading attributes and data of the target data of the third-party system; The attributes of the target data comprise: dictionary content of the target data; correlation of the target data; data type of the target data; S3, according to the attributes and data of the target data, configuring a mapping rule file corresponding to the attributes and data of the target data; the step S3 comprises the following steps: S3-1, configuring a mapping rule XML file structure segment of the target data, which is used for defining data transmission rules and file transmission attributes; the step S3-1 comprises: according to a third-party system data type defined by a dataType field, calling a resultMap field to configure a mapping rule XML file segment corresponding to a synchronization range of the target data; defining the third-party system data type by the dataType field and defining a Documentum-based document system data type by a dctmType field, both of which are String types by default; according to file transmission attributes defined by a fileConfig field, combining the mapping rule XML file segment of the attributes and data of the target data, and performing corresponding operations on the target data; the fileConfig field defines file transmission attributes, and supports file uploading of multiple transmission protocols such as FTP, SFTP, HTTP and SAMBA; according to the resultMap field, configuring the mapping rule XML file segment of the attributes and data of the target data; in the resultMap field, id is a unique identifier of the mapping rule XML file of the attributes and data of the target data, and result is the mapping rule XML file segment of the attributes and data of the target data; in the result field, column is an attribute name of the third-party system data type, and property is an attribute name of the Documentum-based document system data type; S3-2, according to data of the target data, configuring an XML data template corresponding to the data of the target data, which is used for reading the data of the target data into a corresponding position of the XML data template; S3-3, according to dictionary content of the target data, configuring a mapping rule XML file segment corresponding to the dictionary content of the target data, which is used for storing the target data with a mapping relationship; the step S3-3 comprises the following steps: according to the dictionary content of the target data, calling a valueMaps field to configure the mapping rule XML file segment corresponding to the dictionary content of the target data; the valueMaps field comprises a plurality of valueMap fields, according to which the mapping rule XML file segment corresponding to the dictionary content of the target data is configured; in the valueMap field, key is the dictionary content of the target data of the third-party system, and value is the dictionary content of the target data of the Documentum-based document system. S3-4: According to the association relationship of the target data, a mapping rule XML file segment corresponding to the association relationship of the target data is configured, which is used to query a plurality of target data results associated with the target data according to the association relationship of the target data; the step S3-4 includes the following steps: according to the association relationship of the target data, a relations field is called to configure a mapping rule XML file segment corresponding to the association relationship of the target data; the relations field contains a plurality of relation fields, and a mapping rule corresponding to the association relationship of the target data is configured according to the relation field; each association relationship of the target data is stored in a relation field; in the relation field, name is the type of the association relationship of the target data, label is the name of the association relationship of the target data, and identity is the unique identifier of the association relationship of the target data; S3-5: According to the data type of the target data, a mapping rule XML file segment corresponding to the synchronization range of the target data is configured, which is used to determine the synchronization range of the target data and mark the transmission state of the target data; the mapping rule XML file segment of the synchronization range of the target data includes: a first synchronization range segment of the target data and a transmission state value segment of the target data; The step S3-5 includes the following steps: according to the data type of the target data, a transRules field is called to configure a mapping rule XML file segment corresponding to the synchronization range of the target data; in the transRules field, a whereClause field is included, a mapping rule corresponding to the first synchronization range of the target data is configured according to the query condition statement of the data type of the target data in the whereClause field, the query condition statement is included in a CDATA block, and the first synchronization range of the target data is determined; in the transRules field, a transFlag field is included, the transmission state value of the target data is set according to the transFlag field, and a mapping rule corresponding to the transmission state value of the target data is configured, which is used to mark the transmission state of the target data; in the transRules field, name is the name of the mapping rule of the synchronization range of the target data; dctmType field defines the data type of the target data to be transmitted; batchSize field is used to limit the maximum batch size of the target data transmitted each time; S3-6: The XML data template of the target data and the mapping rule XML file segment of all attributes of the target data are combined, and the mapping rule XML file overall structure segment of the target data is combined to form a complete mapping rule XML file of the attributes and data of the target data; S4, according to the attribute and data mapping rule file of the target data, calling a data storage method to perform corresponding operation on the target data.

2. The data synchronization method of claim 1, wherein, The data storage method includes: a data storage method of synchronizing data from the third-party system to the Documentum-based document system; or a data storage method of synchronizing data from the Documentum-based document system to the third-party system; or a data storage method of uploading files from the third-party system to the Documentum-based document system; or a data storage method of uploading files from the Documentum-based document system to the third-party system.

3. The data synchronization method of claim 1, wherein, The step S4 includes the following steps: According to the XML data template of the target data and the mapping rule XML file segment of the attribute of the target data, synchronizing the target data from the third-party system to the Documentum-based document system, moving the target data of successful synchronization into a historical database, and retransmitting the target data of failed synchronization; the historical database is a document database for persistently storing the target data of successful synchronization in the Documentum-based document system; Or, according to the mapping rule XML file overall structure segment of the target data, uploading files from the third-party system to the Documentum-based document system.

4. The data synchronization method of claim 1, wherein, The step S3-2 includes the following steps: According to the read target data, the data of the target data is sequentially filled into the corresponding position of the data template from the first data to the last data, forming a complete XML data template of the target data.

5. The data synchronization method of claim 3, wherein, The step S4-1 further includes: According to the mapping rule of the delivery state value of the target data, setting a timing time, and moving the target data of successful synchronization into the historical database according to the timing time; Setting a delay time to retransmit the target data of failed synchronization; Setting a count of the retransmission number of the target data of failed synchronization; The target data of failed synchronization is not retransmitted within the delay time; and / or The count of the retransmission number of the target data of failed synchronization reaches 10 times, and the target data of failed synchronization is not covered by retransmission; and / or The target data of failed synchronization is covered by retransmission; and / or According to the attribute and data of the target data of the third-party system, reconfiguring the mapping rule XML file of the attribute and data of the corresponding target data.

6. A data synchronization apparatus applying the data synchronization method according to any one of claims 1 to 5, characterized by The data synchronization device includes: An acquisition unit configured to acquire target data of the third-party system and read the attribute and data of the target data; A configuration unit configured to configure a mapping rule file of the attribute and data of the corresponding target data according to the attribute and data of the target data; A synchronization unit configured to call a data storage method according to the mapping rule file of the attribute and data of the target data to perform corresponding operation on the target data.

7. The data synchronization apparatus according to claim 6, wherein The data synchronization device further includes: a storage unit configured to store the data of the target data of the third-party system and a mapping rule file corresponding to the attributes and data of the target data; the obtaining unit further comprises: a first obtaining subunit configured to read the data of the target data; a second obtaining subunit configured to read the dictionary content of the target data; a third obtaining subunit configured to read the association relationship of the target data; a fourth obtaining subunit configured to read the data type of the target data.

8. The data synchronization apparatus according to claim 7, wherein the configuration unit further comprises: a first configuration subunit configured to configure a mapping rule XML file overall structure segment of the target data; a second configuration subunit configured to configure an XML data template corresponding to the data of the target data according to the data of the target data; a third configuration subunit configured to configure a mapping rule XML file segment corresponding to the dictionary content of the target data according to the dictionary content of the target data; a fourth configuration subunit configured to configure a mapping rule XML file segment corresponding to the association relationship of the target data according to the association relationship of the target data; a fifth configuration subunit configured to configure a mapping rule XML file segment corresponding to the synchronization range of the target data according to the data type of the target data; a sixth configuration subunit configured to combine the XML data template of the data of the target data and the mapping rule XML file segments of all the attributes of the target data, combine the mapping rule XML file overall structure segment of the target data, and form a complete mapping rule XML file of the attributes and content of the target data.

9. The data synchronization apparatus according to claim 8, wherein the data synchronization device further comprises: a construction unit configured to construct a data storage method based on a Documentum document system; the construction unit comprises: a first construction subunit configured to construct a data storage method for synchronizing data from the third-party system to the data storage method based on the Documentum document system; a second construction subunit configured to construct a data storage method for synchronizing data from the data storage method based on the Documentum document system to the third-party system; a third construction subunit configured to construct a data storage method for uploading files from the third-party system to the data storage method based on the Documentum document system; a fourth construction subunit configured to construct a data storage method for uploading files from the data storage method based on the Documentum document system to the third-party system.

10. The data synchronization apparatus according to claim 9, wherein, the synchronization unit further comprises: a first synchronization subunit configured to synchronize the target data from the third-party system to the Documentum-based document system according to the XML data template of the data of the target data and the mapping rule XML file segment of the attributes of the target data, move the target data that is successfully synchronized to a historical database, and retransmit the target data that fails to be synchronized; a second synchronization subunit configured to upload files from the third-party system to the Documentum-based document system according to the mapping rule XML file overall structure segment of the target data.

11. The data synchronization apparatus according to claim 10, wherein the first synchronization subunit further comprises: a time unit, configured to set a timing time, and move the target data whose synchronization is successful into a history database; a time unit, configured to set a delay time, and retransmit the target data whose synchronization fails; a counting unit, configured to set a count of the number of times of retransmission of the target data whose synchronization fails; a judging unit, configured to judge whether the target data whose synchronization fails is retransmitted within the delay time and / or whether the count of the number of times of retransmission of the target data whose synchronization fails reaches 10 times; a retransmitting unit, configured to, when the target data whose synchronization fails is not retransmitted within the delay time and / or the count of the number of times of retransmission of the target data whose synchronization fails reaches 10 times, retransmit the target data whose synchronization fails without coverage; and / or retransmit the target data whose synchronization fails with coverage; and / or reconfigure a mapping rule XML file of attributes and data of the corresponding target data according to attributes and data of the third-party system target data.

12. A computer device, comprising: The computer readable storage medium stores a computer program executable by the processor, and the processor is in communication with the memory through the bus, and the processor executes the computer program to perform the steps of the method of any one of claims 1-5.

13. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is adapted to be loaded by the processor to perform the steps of the method of any one of claims 1-5.

Images