Data synchronization method and system

Abstract

The present invention discloses a method and system for data synchronization, and relates to the field of computer technology. A specific implementation of the method includes: obtaining an external transaction file generated by an external system and online transaction data generated by an online system; generating a merged file according to the external transaction file and the online transaction data; performing a format transcoding operation on the merged file to generate batch data; and synchronizing the batch data to a batch system. This implementation avoids the technical defects of the prior art in that the import time is long and the batch processing time is long when performing data synchronization, and achieves the technical effect of meeting the current production needs when updating the business.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and system for data synchronization. Background Art

[0002] The prior art adopts a dual information mechanism system in many scenarios.

[0003] For example, bank credit cards use a dual information mechanism system. During the day, the online system is used to record business needs; at night, the batch system synchronizes the changed business data from the online system to the batch system, and after the batch system completes the processing, it synchronizes the data to the online system.

[0004] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0005] 1. When full data synchronization is used, the import time is long and the batch processing time is long;

[0006] 2. The development and debugging of existing host programs is difficult, that is, the existing host technology cannot meet the current production needs. Summary of the invention

[0007] In view of this, an embodiment of the present invention provides a method and system for data synchronization, which can avoid the technical defects of the prior art such as long import time and long batch processing time when performing data synchronization, and meet current production needs when updating business.

[0008] To achieve the above object, according to one aspect of an embodiment of the present invention, a data synchronization method is provided, which is used to synchronize online data to a batch system, comprising:

[0009] Obtain external transaction files generated by external systems and online transaction data generated by online systems;

[0010] generating a merged file according to the external transaction file and the online transaction data;

[0011] Performing a format transcoding operation on the merged file to generate batch data;

[0012] The batch data is synchronized to a batch system.

[0013] Optionally, generating a merged file according to the external transaction file and the online transaction data includes:

[0014] Screening the online transaction data to obtain an internal transaction file; converting the internal transaction file into a first software interface file;

[0015] Performing format verification on the external transaction file; converting the external transaction file that has completed format verification into a second software interface file;

[0016] The first software interface file and the second software interface file are merged to generate a merged file.

[0017] Optionally, performing a format transcoding operation on the merged file to generate batch data includes:

[0018] Determining a first development language used by the online system and the external system;

[0019] Determining a second development language used by the batch system;

[0020] According to the first development language and the second development language, the merged file is formatted and converted to generate batch data.

[0021] According to another aspect of an embodiment of the present invention, a data synchronization method is provided for synchronizing batch data to an online system and an external system, comprising:

[0022] Get batch data from batch systems;

[0023] Splitting the batch data to generate a first processing result for synchronization to an online system and a second processing result for synchronization to an external system;

[0024] updating the online system according to the first processing result;

[0025] The external system is updated according to the second processing result.

[0026] Optionally, splitting the batch data to generate a first processing result for synchronization to an online system and a second processing result for synchronization to an external system includes:

[0027] Determining a first development language corresponding to the online system and the external system;

[0028] Determining a second development language corresponding to the batch data;

[0029] Performing format verification on the batch data;

[0030] Performing a transcoding operation on the batch data according to the first development language and the second development language;

[0031] The batch data that has completed the transcoding operation is split and processed to generate a first processing result for synchronization to the online system and a second processing result for synchronization to the external system.

[0032] Optionally, before performing a transcoding operation on the batch data according to the first development language and the second development language, the method includes:

[0033] A format check is performed on the batch data.

[0034] According to another aspect of an embodiment of the present invention, there is provided a data synchronization system for synchronizing online data to a batch system, comprising: an external system, an online system, a batch front-end system and a batch system;

[0035] Obtain external transaction files generated by external systems and online transaction data generated by online systems;

[0036] In the batch front-end system, a merged file is generated according to the external transaction file and the online transaction data; a format transcoding operation is performed on the merged file to generate batch data;

[0037] The batch data is synchronized to a batch system.

[0038] Optionally, generating a merged file according to the external transaction file and the online transaction data includes:

[0039] Screening the online transaction data to obtain an internal transaction file; converting the internal transaction file into a first software interface file;

[0040] Performing format verification on the external transaction file; converting the external transaction file that has completed format verification into a second software interface file;

[0041] The first software interface file and the second software interface file are merged to generate a merged file.

[0042] According to another aspect of an embodiment of the present invention, a data synchronization system is provided, which is used to synchronize batch data to an online system and an external system, including: an external system, an online system, a batch front-end system and a batch system;

[0043] Get batch data from batch systems;

[0044] In the batch front-end system, the batch data is split and processed to generate a first processing result for synchronization to an online system and a second processing result for synchronization to an external system;

[0045] In the online system, updating the online system according to the first processing result;

[0046] In the external system, the external system is updated according to the second processing result.

[0047] According to another aspect of an embodiment of the present invention, a data synchronization system is provided, characterized in that it includes: an external system, an online system, a batch front-end system and a batch system;

[0048] When online data is synchronized to a batch system, it includes: obtaining an external transaction file generated by an external system and online transaction data generated by an online system; generating a merged file in the batch front-end system according to the external transaction file and the online transaction data; performing a format transcoding operation on the merged file to generate batch data; synchronizing the batch data to the batch system; and / or,

[0049] When batch data is synchronized to an online system and an external system, the process includes: acquiring batch data of the batch system; in the batch front-end system, splitting and processing the batch data to generate a first processing result for synchronization to the online system and a second processing result for synchronization to the external system; in the online system, updating the online system according to the first processing result; in the external system, updating the external system according to the second processing result.

[0050] According to another aspect of an embodiment of the present invention, there is provided an electronic device for data synchronization, comprising:

[0051] one or more processors;

[0052] a storage device for storing one or more programs,

[0053] When the one or more programs are executed by the one or more processors, the one or more processors implement the data synchronization method provided by the present invention.

[0054] According to another aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method for data synchronization provided by the present invention is implemented.

[0055] One embodiment of the above invention has the following advantages or beneficial effects:

[0056] In an optional embodiment of the present application, data conversion methods of batch-to-link and link-to-batch are used to achieve data synchronization between the open platform and the host system, thereby reducing the transformation of the host system to adapt to the open platform, reducing the system operation and maintenance costs, and improving the business docking speed of the open platform, avoiding the technical defects of the prior art such as long import time and long batch processing time during data synchronization.

[0057] In an optional embodiment of the present application, batch jobs are scheduled by a batch front-end system so that the load of machines in the cluster can be dynamically scheduled according to actual needs, thereby effectively achieving system expansion and ensuring system security. By setting up a batch front-end system, the development and debugging of the host system can be unaffected by the open system, and the development of the open system can also be unaffected by the host system; when updating the business, it can also meet the current production needs.

[0058] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with specific implementation examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0060] Figure 1 is a schematic diagram of the main process of a data synchronization method for synchronizing online data to a batch system according to an embodiment of the present invention;

[0061] Figure 2 is a schematic diagram of main modules of a data synchronization device for synchronizing online data to a batch system according to an embodiment of the present invention;

[0062] Figure 3 is a schematic diagram of main modules of a data synchronization system for synchronizing online data to a batch system according to an embodiment of the present invention;

[0063] Figure 4 is a schematic diagram of the main process of a data synchronization method for synchronizing batch data to an online system and an external system according to an embodiment of the present invention;

[0064] Figure 5 is a schematic diagram of main modules of a data synchronization device for synchronizing batch data to an online system and an external system according to an embodiment of the present invention;

[0065] Figure 6 is a schematic diagram of main modules of a data synchronization system for synchronizing batch data to an online system and an external system according to an embodiment of the present invention;

[0066] Figure 7 is a schematic diagram of main modules of a general data synchronization system according to an embodiment of the present invention;

[0067] Figure 8 is a schematic diagram of main modules of a general data synchronization system according to an embodiment of the present invention;

[0068] Fig. 9 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0069] Fig.10 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0070] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0071] With the rapid development of businesses in various industries, business demands are increasing, and the existing mainframe systems can no longer meet the current needs. Especially in the field of Internet finance, IBM mainframe technology cannot meet the needs of batch data synchronization updates. At the same time, the rise of Internet finance has made the situation in the banking industry more severe. In order to quickly respond to the challenges of Internet finance, technological transformation is the top priority of the strategic transformation of the banking industry.

[0072] In order to meet the needs of the business, the IBM mainframe system can be used offline, and the new system of the open platform can be used online, and then gradually transition to the whole system running on the development platform. However, the banking system is not only complex in function but also numerous, and it is inevitable that there will be a transition period in which the open platform system and the mainframe system coexist. Data synchronization between the mainframe system and the open platform system is inevitable, and high stability and security between the mainframe system and the development platform system must be met.

[0073] Data synchronization between the host system and the open platform system mainly involves two aspects: 1. Synchronizing the data of the online system to the batch system, which can be referred to as "link to batch"; 2. Synchronizing the processing results of the batch system to the online system.

[0074] In the field of Internet finance, the application scenario of link-to-batch is generally to synchronize the demand data generated by the online system during the day to the batch system; the application scenario of batch-to-link is generally to update the batch data in the batch system that has been processed at night to the online system.

[0075] In the existing system, the online system of the open platform and the IBM mainframe system usually coexist. Due to the large differences between the online system and the mainframe system, especially in the storage structure and syntax adopted, a batch front-end system is added in the optional embodiment of the present application to facilitate the conversion and synchronization of data in the batch front-end system. Specifically, the batch front-end system provides a unified interactive format to the outside.

[0076] In the system applied to banks, the online system refers to the system of business transactions generated by daytime bankers or ATMs in the banking industry, which records any business changes in real time;

[0077] Those skilled in the art also refer to the batch system as the core system. The main function of the batch system is to uniformly process the transaction records generated by the online system during the day at night, such as account entry, interest calculation, etc. Every day, some data of the batch system needs to be updated to the online system.

[0078] There are multiple hosts and corresponding storage spaces in the batch system. The hosts are mainly IBM hosts, which are computer systems developed in the 1960s. After more than 40 years of continuous updates, their stability and security are good. At the same time, they have powerful data processing capabilities. IBM mainframes are generally used by governments, banks, insurance companies and large manufacturing companies. The programming language related to mainframe technology is COBOL, but the syntax and instruction set of the COBOL language are difficult to learn and debug, there are relatively few learning materials, the learning cost is high, and the COBOL program needs to run on the mainframe, and the mainframe equipment is very expensive, resulting in a shortage of mainframe technical talents in the market. The expensiveness of the mainframe equipment and the closed nature of the mainframe technology lead to huge costs for enterprises to use the mainframe.

[0079] In order to reduce the cost of the entire system, online systems generally use open platforms, specifically running system services such as X86 architecture on the open platform. Its equipment cost is low, and the languages ​​used in system service development are diverse. For example, JAVA language not only has the characteristics of open source, but also has a complete set of solutions in terms of distribution and microservices. As a result, more and more enterprises are turning to open platforms for basic information construction.

[0080] Figure 1 is a schematic diagram of the main process of a data synchronization method for synchronizing online data to a batch system according to an embodiment of the present invention; Figure 1 As shown, it is used to synchronize online data to the batch system, including:

[0081] Step S101, obtaining an external transaction file generated by an external system and online transaction data generated by an online system;

[0082] Step S102: generating a merged file according to the external transaction file and the online transaction data;

[0083] Step S103, performing a format transcoding operation on the merged file to generate batch data;

[0084] Step S104: synchronizing the batch data to a batch system.

[0085] Since the external system is mainly built with open source languages ​​and architectures, the programming language associated with the batch system (including the host) is COBOL, and the syntax and instruction set of the COBOL language are difficult to learn and debug, and there are relatively few learning materials and high learning costs. COBOL programs need to run on the host, and the host equipment is very expensive. The existing external component system and online system are both development platforms, and their languages ​​can be open source JAVA, etc., and then the external transaction files generated by the external system and the online transaction data generated by the online system can be merged and converted before entering the batch system. The batch pre-operation can shield other systems from direct interaction with the host, greatly reducing the transformation of the host system.

[0086] In addition, batch jobs can be executed through batch pre-scheduling, which can be dynamically scheduled according to the load capacity of the machines in the cluster, thereby achieving the technical effect of reducing the load of the batch system.

[0087] Since batch pre-processing can uniformly process external transaction files and online transaction data, the data to be synchronized is consistent, making the data synchronization process more efficient.

[0088] In order to facilitate unified processing of external transaction files and online transaction data, optionally, a merged file is generated according to the external transaction files and the online transaction data, including:

[0089] Screening the online transaction data to obtain an internal transaction file; converting the internal transaction file into a first software interface file (UI file);

[0090] Performing format verification on the external transaction file; converting the external transaction file that has completed format verification into a second software interface file;

[0091] The first software interface file and the second software interface file are merged to generate a merged file.

[0092] Since the content of online transaction data is large and some data does not need to be synchronized to the batch system, the online transaction data can be screened before performing the pre-operation, thereby achieving the technical effect of reducing the workload of merging and batch pre-operation and reducing the workload of data synchronization.

[0093] Optionally, performing a format transcoding operation on the merged file to generate batch data includes:

[0094] Determine a first development language used by the online system and the external system; wherein the first development language may include but is not limited to Java, C language, C++ language, etc.

[0095] Determine a second development language used by the batch system; generally, the second development language refers to COBOL language.

[0096] According to the first development language and the second development language, the merged file is formatted and converted to batch data. In an optional embodiment, the merged file is formatted and converted to batch data in COBOL language.

[0097] Figure 2 is a schematic diagram of main modules of a data synchronization system for synchronizing online data to a batch system according to an embodiment of the present invention, such as Figure 2 As shown, a data synchronization system 200 is provided for synchronizing online data to a batch system, including: an external system 201, an online system 202, a batch front-end system 203 and a batch system 204;

[0098] Obtain external transaction files generated by external systems and online transaction data generated by online systems;

[0099] In the batch front-end system, a merged file is generated according to the external transaction file and the online transaction data; a format transcoding operation is performed on the merged file to generate batch data;

[0100] The batch data is synchronized to a batch system.

[0101] Optionally, generating a merged file according to the external transaction file and the online transaction data includes:

[0102] Screening the online transaction data to obtain an internal transaction file; converting the internal transaction file into a first software interface file;

[0103] Performing format verification on the external transaction file; converting the external transaction file that has completed format verification into a second software interface file;

[0104] The first software interface file and the second software interface file are merged to generate a merged file.

[0105] Optionally, performing a format transcoding operation on the merged file to generate batch data includes:

[0106] Determining a first development language used by the online system and the external system;

[0107] Determining a second development language used by the batch system;

[0108] According to the first development language and the second development language, the merged file is formatted and converted to generate batch data.

[0109] The specific process of synchronizing online data to a batch system is described in detail below with reference to a specific embodiment 1.

[0110] Figure 3 1 is a schematic diagram of the main modules of a data synchronization system for synchronizing online data to a batch system according to an embodiment of the present invention. Assume that the application scenario of this specific embodiment 1 is the synchronization of bank credit card data.

[0111] Specifically, since the external transaction files generated by the external system and the online transaction data generated by the online system mainly occur during the day and the transaction volume is relatively small at night, in this embodiment, it can be set to perform the joint batch data synchronization operation after 19:00 every day, thereby completing the data synchronization without affecting the operation of the online system.

[0112] like Figure 3 As shown, a data synchronization system 300 for synchronizing online data to a batch system includes: an external system 301, an online system 302, a batch front-end system 303 and a batch system 304;

[0113] The batch front-end system 303 obtains the external transaction file generated by the external system 301 and the online transaction data generated by the online system 302;

[0114] In the batch front-end system 303, the online transaction data is screened to generate internal files;

[0115] In the batch front-end system 303, the external transaction file is format-checked, and the format-checked external transaction file is merged with the internal file to generate a merged file; and the merged file is format-transcoded to generate batch data;

[0116] The batch data is synchronized to the batch system 304, and the batch data is processed to generate a processing result.

[0117] Figure 4 1 is a schematic diagram of the main process of a method for synchronizing batch data to an online system and an external system according to an embodiment of the present invention, including:

[0118] Step S401, obtaining batch data of a batch system;

[0119] Step S402: split the batch data to generate a first processing result for synchronization to an online system and a second processing result for synchronization to an external system;

[0120] Step S403: updating the online system according to the first processing result;

[0121] Step S404: Update the external system according to the second processing result.

[0122] Optionally, splitting the batch data to generate a first processing result for synchronization to an online system and a second processing result for synchronization to an external system includes:

[0123] Determining a first development language corresponding to the online system and the external system;

[0124] Determining a second development language corresponding to the batch data;

[0125] Performing format verification on the batch data;

[0126] Performing a transcoding operation on the batch data according to the first development language and the second development language;

[0127] The batch data that has completed the transcoding operation is split and processed to generate a first processing result for synchronization to the online system and a second processing result for synchronization to the external system.

[0128] Optionally, before performing a transcoding operation on the batch data according to the first development language and the second development language, the method includes:

[0129] A format check is performed on the batch data.

[0130] Figure 5 is a schematic diagram of main modules of a data synchronization device for synchronizing batch data to an online system and an external system according to an embodiment of the present invention; Figure 5 As shown, a data synchronization system 500 is provided, which is used to synchronize batch data to an online system and an external system, including: an external system 501, an online system 502, a batch front-end system 503 and a batch system 504;

[0131] Get batch data from batch systems;

[0132] In the batch front-end system, the batch data is split and processed to generate a first processing result for synchronization to an online system and a second processing result for synchronization to an external system;

[0133] In the online system, updating the online system according to the first processing result;

[0134] In the external system, the external system is updated according to the second processing result.

[0135] The specific process of synchronizing batch data to the online system and the external system is described in detail below with reference to the second specific embodiment.

[0136] Figure 6 1 is a schematic diagram of the main modules of a data synchronization system for synchronizing batch data to an online system and an external system according to an embodiment of the present invention. Assume that the application scenario of this specific embodiment 2 is the synchronization of bank credit card data.

[0137] Specifically, after the batch data is processed at night, it is necessary to synchronize the processed batch data to the online system and the external system, so that the online system and the external system can directly use the processed batch data during the day.

[0138] like Figure 6 As shown, a data synchronization system 600 for synchronizing batch data to an online system and an external system includes: an external system 601, an online system 602, a batch front-end system 603 and a batch system 604;

[0139] The batch front-end system 603 obtains batch data from the batch system 604;

[0140] In the batch front-end system 603, the batch data 604 is format checked; the batch data after the format check is completed is transcoded; after the transcoding operation is completed, a splitting process is performed to generate a first processing result for synchronization to an online system and a second processing result for synchronization to an external system;

[0141] Using the first processing result, updating the online system 602;

[0142] The external system 601 is updated using the second processing result.

[0143] Figure 7 Schematic diagram of main modules of a general data synchronization system according to an embodiment of the present invention. Figure 7 As shown, a data synchronization system 700 is provided, characterized in that it includes: an external system 701, an online system 702, a batch front-end system 703 and a batch system 704;

[0144] When online data is synchronized to a batch system, it includes: obtaining an external transaction file generated by an external system and online transaction data generated by an online system; generating a merged file in the batch front-end system according to the external transaction file and the online transaction data; performing a format transcoding operation on the merged file to generate batch data; synchronizing the batch data to the batch system; and / or,

[0145] When batch data is synchronized to an online system and an external system, the process includes: acquiring batch data of the batch system; in the batch front-end system, splitting and processing the batch data to generate a first processing result for synchronization to the online system and a second processing result for synchronization to the external system; in the online system, updating the online system according to the first processing result; in the external system, updating the external system according to the second processing result.

[0146] The specific process of synchronizing online data to a batch system is described in detail below with reference to a third specific embodiment.

[0147] Figure 8 Schematic diagram of main modules of a general data synchronization system according to an embodiment of the present invention. Assume that the application scenario of this specific embodiment 3 is the synchronization of bank credit card data.

[0148] Specifically, since the external transaction files generated by the external system and the online transaction data generated by the online system mainly occur during the day, and the transaction volume at night is relatively small, in this embodiment, it can be set to perform the joint-to-batch data synchronization operation after 19:00 every day. After the joint-to-batch data synchronization is completed, the batch data is processed in the batch system, and then the processed batch data is synchronized to the external system and the online system.

[0149] like Figure 8 As shown, a schematic diagram of the main modules of the general data synchronization system; Figure 8 As shown:

[0150] When performing the batch transfer operation, the online transaction data of the online system and the external transaction files of the external system are transferred to the batch front-end system, wherein the transfer process has been described in detail in the above scheme and will not be repeated here. In the batch front-end system, the online transaction data and the external transaction files are batch pre-processed, and the processed batch data is synchronized to the batch system.

[0151] When performing the batch transfer operation, the batch data processed in the batch system is updated to the batch front-end system; the batch data is processed in the batch front-end system (the specific processing process is the same as the processing process in the specific embodiment 2, which will not be repeated here) to generate a first processing result and a second processing result;

[0152] According to the first processing result, the online system is updated; according to the second processing result, the external system is updated.

[0153] In an optional embodiment of the present application, a batch front-end system is set up and data conversion methods of batch-to-link and link-to-batch are adopted to achieve data synchronization between the open platform and the host system, thereby reducing the transformation of the host system to adapt to the open platform, reducing the system operation and maintenance costs, and improving the business docking speed of the open platform, avoiding the technical defects of the prior art such as long import time and long batch processing time when performing data synchronization.

[0154] In an optional embodiment of the present application, batch jobs are executed by scheduling the batch front-end system so that the machine load in the cluster is dynamically scheduled according to actual needs, thereby effectively achieving system expansion and ensuring system security.

[0155] By setting up a batch front-end system, the development and debugging of the host system will not be affected by the open system, and the development of the open system will not be affected by the host system. When updating the business, it can meet the current production needs.

[0156] Fig. 9 An exemplary system architecture 900 is shown to which the data synchronization method or data synchronization device according to the embodiment of the present invention can be applied.

[0157] like Fig. 9 As shown, system architecture 900 may include terminal devices 901, 902, 903, network 904 and server 905. Network 904 is used to provide a medium for communication links between terminal devices 901, 902, 903 and server 905. Network 904 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0158] Users can use terminal devices 901, 902, 903 to interact with server 905 through network 904 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 901, 902, 903, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

[0159] The terminal devices 901 , 902 , and 903 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers, etc.

[0160] The server 905 may be a server that provides various services, such as a backend management server (only an example) that provides support for shopping websites browsed by users using the terminal devices 901, 902, and 903. The backend management server may analyze and process the received data such as product information query requests, and feed back the processing results (such as target push information, product information - only an example) to the terminal device.

[0161] It should be noted that the data synchronization method provided in the embodiment of the present invention is generally executed by the server 905 , and accordingly, the data synchronization device is generally arranged in the server 905 .

[0162] It should be understood that Fig. 9 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0163] Reference below Fig.10 , which shows a schematic diagram of the structure of a computer system 1000 of a terminal device suitable for implementing an embodiment of the present invention. Fig.10 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0164] like Fig.10 As shown, the computer system 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the system 1000 are also stored. The CPU 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0165] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage section 1008 as needed.

[0166] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the central processing unit (CPU) 1001, the above-mentioned functions defined in the system of the present invention are executed.

[0167] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0168] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0169] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The modules described may also be set in a processor. For example, they may be described as: a processor includes a sending module, an acquisition module, a determination module, and a first processing module. The names of these modules do not, in some cases, constitute limitations on the modules themselves. For example, the sending module may also be described as a "module for sending a picture acquisition request to the connected server."

[0170] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: a method for synchronizing online data to a batch system, including:

[0171] Obtain external transaction files generated by external systems and online transaction data generated by online systems;

[0172] generating a merged file according to the external transaction file and the online transaction data;

[0173] Performing a format transcoding operation on the merged file to generate batch data;

[0174] The batch data is synchronized to a batch system.

[0175] According to the technical solution of the embodiment of the present invention, the following beneficial effects can be achieved:

[0176] In an optional embodiment of the present application, data conversion methods of batch-to-link and link-to-batch are used to achieve data synchronization between the open platform and the host system, thereby reducing the transformation of the host system to adapt to the open platform, reducing the system operation and maintenance costs, and improving the business docking speed of the open platform, avoiding the technical defects of the prior art of long import time and long batch processing time when performing data synchronization.

[0177] In an optional embodiment of the present application, batch jobs are scheduled by a batch front-end system so that the machine load in the cluster is dynamically scheduled according to actual needs, thereby effectively achieving system expansion and ensuring system security. By setting up a batch front-end system, the development and debugging of the host system can be unaffected by the open system, and the development of the open system is also unaffected by the host system, so that when updating the business, the current production needs can be met.

[0178] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for data synchronization, characterized in that: Used to synchronize online data to batch systems, including: Obtain external transaction files generated by external systems and online transaction data generated by online systems; In the batch front-end system, a merged file is generated according to the external transaction file and the online transaction data; a format transcoding operation is performed on the merged file to generate batch data, and batch jobs are scheduled and executed by the batch front-end system so that the machine load in the cluster is dynamically scheduled according to actual needs; synchronizing the batch data to a batch system; Generating a merged file according to the external transaction file and the online transaction data includes: screening the online transaction data to obtain an internal transaction file; converting the internal transaction file into a first software interface file; performing format verification on the external transaction file; converting the external transaction file that has completed the format verification into a second software interface file; merging the first software interface file and the second software interface file to generate a merged file; Performing a format transcoding operation on the merged file to generate batch data includes: determining a first development language used by the online system and the external system; determining a second development language used by the batch system; performing a format transcoding operation on the merged file according to the first development language and the second development language to generate batch data; wherein performing a format transcoding operation on the merged file is to convert the merged file developed in the first development language into batch data in COBOL language.

2. A method for data synchronization, characterized in that: Used to synchronize batch data to online and external systems, including: Acquire batch data of a batch system; wherein the batch system receives batch data of an external system and an online system, the batch front-end system performs format transcoding operation on a merged file to generate batch data of the external system and the online system, the merged file is generated by merging a first software interface file and a second software interface file, the first software interface file is generated by screening and transforming online transaction data of the online system, the second software interface file is generated by format checking and transforming an external transaction file of the external system, and the batch front-end system is used to schedule and execute batch jobs so that the machine load in the cluster is dynamically scheduled according to actual needs; the batch data of the batch system is split and processed to generate a first processing result for synchronization to the online system and a second processing result for synchronization to the external system; updating the online system according to the first processing result; updating the external system according to the second processing result; The batch front-end system performs format transcoding operation on the merged file to generate batch data of the external system and the online system, including: determining a first development language adopted by the online system and the external system; determining a second development language adopted by the batch system; performing format transcoding operation on the merged file according to the first development language and the second development language to generate batch data of the external system and the online system; wherein the format transcoding operation on the merged file is to convert the merged file developed in the first development language into batch data in COBOL language.

3. The method according to claim 2, characterized in that The batch data of the batch system is split and processed to generate a first processing result for synchronization to the online system and a second processing result for synchronization to the external system, including: Performing a transcoding operation on the batch data of the batch system according to the first development language and the second development language; The batch data that has completed the transcoding operation is split and processed to generate a first processing result for synchronization to the online system and a second processing result for synchronization to the external system.

4. The method according to claim 3, characterized in that Before performing a transcoding operation on the batch data of the batch system according to the first development language and the second development language, the method includes: Performing format check on batch data of the batch system.

5. A data synchronization system, characterized in that: Used to synchronize online data to batch systems, including: external systems, online systems, batch front-end systems, and batch systems; Obtain external transaction files generated by external systems and online transaction data generated by online systems; In the batch front-end system, a merged file is generated according to the external transaction file and the online transaction data; a format transcoding operation is performed on the merged file to generate batch data, and batch jobs are scheduled and executed by the batch front-end system so that the machine load in the cluster is dynamically scheduled according to actual needs; synchronizing the batch data to a batch system; The generating of the merged file according to the external transaction file and the online transaction data includes: screening the online transaction data to obtain an internal transaction file; converting the internal transaction file into a first software interface file; performing format verification on the external transaction file; converting the external transaction file that has completed the format verification into a second software interface file; merging the first software interface file and the second software interface file to generate a merged file; Performing a format transcoding operation on the merged file to generate batch data includes: determining a first development language used by the online system and the external system; determining a second development language used by the batch system; performing a format transcoding operation on the merged file according to the first development language and the second development language to generate batch data; wherein performing a format transcoding operation on the merged file is to convert the merged file developed in the first development language into batch data in COBOL language.

6. A data synchronization system, characterized in that: Used to synchronize batch data to online systems and external systems, including: external systems, online systems, batch front-end systems, and batch systems; Acquire batch data of a batch system; wherein the batch system receives batch data of an external system and an online system, the batch front-end system performs format transcoding operation on a merged file to generate batch data of the external system and the online system, the merged file is generated by merging a first software interface file and a second software interface file, the first software interface file is generated by screening and converting online transaction data of the online system, the second software interface file is generated by format checking and converting an external transaction file of the external system, and the batch front-end system is used to schedule and execute batch jobs so that the machine load in the cluster is dynamically scheduled according to actual needs; In the batch front-end system, batch data of the batch system is split and processed to generate a first processing result for synchronization to the online system and a second processing result for synchronization to the external system; In the online system, updating the online system according to the first processing result; In the external system, updating the external system according to the second processing result; Among them, the batch front-end system performs format transcoding operation on the merged file to generate batch data of the external system and the online system, including: determining the first development language adopted by the online system and the external system; determining the second development language adopted by the batch system; performing format transcoding operation on the merged file according to the first development language and the second development language to generate batch data of the external system and the online system; performing format transcoding operation on the merged file is to convert the merged file developed in the first development language into batch data in COBOL language.

7. A data synchronization system, characterized in that: include: External systems, online systems, batch front-end systems, and batch systems; When online data is synchronized to a batch system, it includes: obtaining an external transaction file generated by an external system and online transaction data generated by an online system; in the batch front-end system, generating a merged file according to the external transaction file and the online transaction data; performing a format transcoding operation on the merged file to generate batch data of the external system and the online system; synchronizing the batch data of the external system and the online system to the batch system, and scheduling and executing batch jobs through the batch front-end system so that the machine load in the cluster is dynamically scheduled according to actual needs; When batch data is synchronized to an online system and an external system, the method includes: acquiring batch data of the batch system; in the batch front-end system, splitting and processing the batch data of the batch system to generate a first processing result for synchronization to the online system and a second processing result for synchronization to the external system; in the online system, updating the online system according to the first processing result; in the external system, updating the external system according to the second processing result; The generating of the merged file according to the external transaction file and the online transaction data includes: screening the online transaction data to obtain an internal transaction file; converting the internal transaction file into a first software interface file; performing format verification on the external transaction file; converting the external transaction file that has completed the format verification into a second software interface file; merging the first software interface file and the second software interface file to generate a merged file; Performing a format transcoding operation on the merged file to generate batch data for the external system and the online system includes: determining a first development language used by the online system and the external system; determining a second development language used by the batch system; performing a format transcoding operation on the merged file according to the first development language and the second development language to generate batch data for the external system and the online system; wherein performing a format transcoding operation on the merged file is to convert the merged file developed in the first development language into batch data in the COBOL language.

Images