A data synchronization method, device, electronic device and storage medium
By using temporary tables and business query views during data synchronization, the problems of business suspension and historical data cannot be deleted during data synchronization in the prior art are solved, and normal business operations and data cleaning of the target business system during data synchronization are achieved.
Patent Information
- Application Number
- CN202210741567.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-27
- Publication Date
- 2025-06-27
- Estimated Expiration
- 2042-06-27
AI Technical Summary
When the existing data synchronization method is synchronized, the target business table cannot be used normally, resulting in business suspension. Incremental data synchronization cannot delete historical invalid data, which cannot meet the system's requirements for data synchronization.
By obtaining the full amount of data to be synchronized and writing it to the target temporary table, adjusting the temporary table based on business needs, generating a business table, and creating a business query view, so that the target business table can still be used normally during the data synchronization process.
It realizes that during the data synchronization process, the target business system can perform business operations normally, avoiding business suspension during data synchronization, and solving the problem of historical invalid data in incremental data synchronization that cannot be deleted.
Smart Images

Figure CN115080569B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of data synchronization, and in particular, to a data synchronization method, device, electronic device, and storage medium. Background Art
[0002] With the rapid development of the times, the data we need to process is gradually growing. Currently, the more efficient processing systems for a large amount of data all adopt a distributed system architecture. For a distributed system architecture, in many scenarios, business data needs to synchronize and store the same business report data among various business subsystems. To ensure the accuracy of business data, it is necessary to ensure periodic update synchronization among subsystems.
[0003] Most of the existing data synchronization methods are of two types. The first type is to empty the business target table in the system to be synchronized and write all the business data to be synchronized in full volume. During the process of data synchronization in this synchronization method, the business target table cannot be used normally, which causes the business in the system to be synchronized to pause. The second type is incremental data synchronization. Incremental data synchronization requires determining an update to update the data, and the invalid historical data in stock cannot be deleted. The existing synchronization methods can no longer meet the requirements of the system for data synchronization. Summary of the Invention
[0004] In view of this, the purpose of the present application is to provide a data synchronization method, device, electronic device, and storage medium, so that the target business system can still perform normal business operations while data synchronization is being performed.
[0005] In a first aspect, an embodiment of the present application provides a data synchronization method, and the method includes:
[0006] According to the data synchronization requirement, obtain all the first data to be synchronized from the first target data table in the source data system that can meet the data synchronization requirement;
[0007] If the first target temporary table corresponding to the first target data table is detected, write the first data to be synchronized into the first target temporary table;
[0008] Based on the requirement element information corresponding to the business requirement, adjust the first target temporary table to generate a first target business table;
[0009] According to the business requirement, create a business query view, and use the first target business table as the current business pointing table of the business query view to complete data synchronization, so that the target business table can still perform normal business queries through the business query view during the process of data synchronization; the business query view includes a query statement and a business pointing table that has a mapping relationship with the query statement.
[0010] In some technical solutions of the present application, the above method further includes:
[0011] According to the data synchronization requirement, obtain all the second data to be synchronized from the second target data table included in the source data system that can meet the data synchronization requirement;
[0012] When detecting the second target temporary table corresponding to the second target data table, write the second data to be synchronized into the second target temporary table;
[0013] Based on the requirement element information corresponding to the business requirement, adjust the second target temporary table to generate a second target business table;
[0014] Use the second target business table to replace the first target business table as the current business pointing table of the business query view.
[0015] In some technical solutions of the present application, when the above first target business table becomes a historical business pointing table, the method further includes:
[0016] Process the historical business pointing table according to the retention requirement for the historical business pointing table.
[0017] In some technical solutions of the present application, when the business requirement changes, the method further includes:
[0018] According to the changed business requirement, select a target business pointing table from the current business pointing table and the historical business pointing table; and use the target business pointing table as the new current business pointing table to meet the changed business requirement.
[0019] In some technical solutions of the present application, the above method further includes:
[0020] If the first target temporary table corresponding to the first target data table is not detected, generate the first target temporary table corresponding to the first target data table, and write the first data to be synchronized into the first target temporary table.
[0021] In some technical solutions of the present application, the above business requirement is a query requirement, and the requirement element information is time period information;
[0022] The adjusting the first target temporary table based on the requirement element information corresponding to the business requirement to generate a first target business table includes:
[0023] Based on the time period information of the query requirement, modify the table attribute information of the first target temporary table to generate a first target business table.
[0024] In some technical solutions of the present application, the above table attribute information is the table name; the first target business table is generated in the following manner:
[0025] Based on the time period information of the query requirement, modify the table name of the first target temporary table to generate the first target business table; the table name of the first target business table includes the time period information.
[0026] In a second aspect, an embodiment of the present application provides a data synchronization device, which includes:
[0027] An acquisition module, configured to acquire all the first data to be synchronized from the first target data table that can meet the data synchronization requirement included in the source data system according to the data synchronization requirement;
[0028] A writing module, configured to write the first data to be synchronized into the first target temporary table if it detects the first target temporary table corresponding to the first target data table;
[0029] A generation module, configured to adjust the first target temporary table based on the requirement element information corresponding to the service requirement to generate the first target business table;
[0030] A view module, configured to create a service query view according to the service requirement, and use the first target business table as the current business pointing table of the service query view to complete data synchronization, so that during the data synchronization process of the target business table, normal service queries can also be performed through the service query view; the service query view includes a query statement and a business pointing table that has a mapping relationship with the query statement.
[0031] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the above data synchronization method.
[0032] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the above data synchronization method.
[0033] The technical solutions provided by the embodiments of the present application may include the following beneficial effects:
[0034] The method of the present application includes obtaining all of the first data to be synchronized from the first target data table included in the source data system that can meet the data synchronization requirements according to the data synchronization requirements; if the first target temporary table corresponding to the first target data table is detected, writing the first data to be synchronized into the first target temporary table; adjusting the first target temporary table based on the requirement element information corresponding to the business requirements to generate a first target business table; creating a business query view according to the business requirements, and using the first target business table as the current business pointing table of the business query view to complete data synchronization, so that during the data synchronization process of the target business table, normal business queries can also be performed through the business query view; the business query view includes a query statement and a business pointing table that has a mapping relationship with the query statement. The present application realizes that when the target business system performs data synchronization, it has no impact on the business of the target business system itself.
[0035] In order to make the above objects, features, and advantages of the present application more obvious and understandable, the following specifically gives preferred embodiments and, in conjunction with the accompanying drawings, the detailed description is as follows. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.
[0037] Figure 1 Shows a schematic flowchart of a data synchronization method provided by an embodiment of the present application;
[0038] Figure 2 Shows a schematic flowchart of another data synchronization method provided by an embodiment of the present application;
[0039] Figure 3 Shows a schematic diagram of a data synchronization device provided by an embodiment of the present application;
[0040] Figure 4 Is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0041] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the accompanying drawings in the embodiments of this application. It should be understood that the accompanying drawings in this application are only for the purposes of illustration and description, and are not used to limit the protection scope of this application. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate the operations implemented according to some embodiments of this application. It should be understood that the operations in the flowchart may not be implemented in sequence, and steps without logical context relationships may be reversed in order or implemented simultaneously. In addition, those skilled in the art may add one or more other operations to the flowchart or remove one or more operations from the flowchart under the guidance of the content of this application.
[0042] In addition, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. The components of the embodiments of this application usually described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of this application claimed, but only represents the selected embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative efforts belong to the protection scope of this application.
[0043] It should be noted that the term "including" will be used in the embodiments of this application to indicate the existence of the features stated thereafter, but does not exclude the addition of other features.
[0044] With the rapid development of the times, the data we need to process is gradually growing. Currently, the more efficient processing systems for facing a large amount of data all adopt a distributed system architecture. For a distributed system architecture, in many scenarios, business data needs to synchronously store the same business report data among various business subsystems. To ensure the accuracy of business data, it is necessary to ensure periodic update synchronization among each subsystem.
[0045] Then how can the data among each business system be synchronized efficiently and accurately? Usually, enterprises use two methods for data synchronization. One is full-volume data synchronization. Before synchronization, the target table is emptied first, and then a full-volume data copy is written into the target table. The disadvantage is that during the data synchronization process, when the business target table is queried and used, there will be a problem of short-term data unavailability, and the business peak period needs to be avoided during synchronization.
[0046] Another way is incremental data synchronization. Incremental synchronization requires determining the update point, that is, it is necessary to capture the data for synchronization after a certain moment, such as the update time or the checkpoint, rather than unregulated full-volume data synchronization. An important point here is that the data source end needs to record or determine which data has been updated by querying the update log. A difficulty of this solution is to determine the update point to capture the incrementally updated data. Another problem with this solution is that the historical invalid data in the stock cannot be deleted. If there is data that needs to be deleted, this solution cannot be used.
[0047] Based on this, the embodiments of the present application provide a data synchronization method, device, electronic device, and storage medium, which will be described below through embodiments.
[0048] Figure 1 The flowchart of a data synchronization method provided by the embodiments of the present application is shown. Among them, this method is applied to the target business system and includes steps S101-S104; specifically:
[0049] S101. According to the data synchronization requirement, obtain the full-volume first data to be synchronized from the first target data table included in the source data system that can meet the data synchronization requirement.
[0050] S102. If the first target temporary table corresponding to the first target data table is detected, write the first data to be synchronized into the first target temporary table.
[0051] S103. Based on the requirement element information corresponding to the business requirement, adjust the first target temporary table to generate the first target business table.
[0052] S104. According to the business requirement, create a business query view, and use the first target business table as the current business pointing table of the business query view to complete data synchronization, so that during the data synchronization process of the target business table, normal business queries can also be performed through the business query view; the business query view includes a query statement and a business pointing table that has a mapping relationship with the query statement.
[0053] When the present application realizes data synchronization in the target business system, it has no impact on the business of the target business system itself.
[0054] Some embodiments of the present application will be described in detail below. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.
[0055] S101. According to the data synchronization requirement, obtain the full-volume first data to be synchronized from the first target data table included in the source data system that can meet the data synchronization requirement.
[0056] The data synchronization method in this application is applied to a target business system, which can be any system in a distributed system or any system in independent systems. When the target business system in this application performs relevant business operations, due to its own business needs, it needs to obtain some data from other systems. In this application, the other system from which the target business system obtains data is called the source data system. Or rather, the source data system contains the data required by the target business system, and the part of the data obtained by the target business system needs to be synchronized with the corresponding part of the data in the source data system.
[0057] The data synchronization requirement characterizes the requirements of the target business system's own business operations. The target business system determines the target data it needs according to its own business operation needs. After determining the target data required by the target business system, it is necessary to find the source data system containing the target data from multiple external systems. The source data system contains multiple data tables, and each data table contains multiple data related to the source data system. After determining the source data system in this application, it is necessary to screen all the data tables contained in the source data system, and find the first target data table containing the target data from the data tables contained in the source data system. The first target data table also contains multiple data, where the multiple data includes the target data required by the target business system and other data not considered by the target business system. What this application needs to obtain from the first target data table is all the target data required by the target business system. After the target business system obtains the target data, it needs to synchronize the target data to its own system. Since the obtained target data needs to be synchronized into the target business system's own system, the target data obtained by the target business system itself is also called the first data to be synchronized. That is to say, the target data (the first data to be synchronized) that can meet the data synchronization requirement is the same data, just with different names at different time periods. This data is called target data in the source data system and the first data to be synchronized in the target business system.
[0058] S102. If the first target temporary table corresponding to the first target data table is detected, write the first data to be synchronized into the first target temporary table.
[0059] After the target business system in this application obtains the first data to be synchronized, it is necessary to write this first data to be synchronized into its own system. When the target business system writes the first data to be synchronized, it needs to first detect from its own system whether there is a first target data table that can write the first data to be synchronized. The first target data table that can write the first data to be synchronized indicates that the capacity of the first target data table needs to be greater than or equal to the capacity required by the first data to be synchronized. The target business system only writes the first data to be synchronized into the first target data table when it detects the first target temporary table.
[0060] In this application, the first target temporary table needs to correspond to the first target data table. Here, the correspondence means that the capacity of the first target temporary table needs to be greater than or equal to the capacity of all the first target data tables. In this application, the first target data table includes one, two, or more. When the first target data table is one, the capacity of the first target temporary table corresponding to this first target data table is greater than or equal to the capacity of this first target data table. When the first target data table is two or more, the capacity of the first target temporary table corresponding to the first target data table needs to be greater than or equal to the total capacity of these two or more first target data tables.
[0061] In this application, if the target business system itself does not detect the first target temporary table corresponding to the first target data table, the target business system needs to first create a first target temporary table corresponding to the first target data table. Then write the first data to be synchronized into the newly created first target temporary table. When creating the first target temporary table, it is necessary to first determine the total capacity of all the first target data tables, and then create a first target temporary table with a capacity greater than or equal to the total capacity of all the first target data tables.
[0062] S103. Based on the requirement element information corresponding to the business requirement, adjust the first target temporary table to generate a first target business table.
[0063] After the first data to be synchronized is written into the first target temporary table of the target business system itself in this application, this application also needs to adjust the first target temporary table to convert the first target temporary table into a first target business table that can meet the business functions of the target business system itself. That is to say, the first target temporary table in this application does not have business functions and cannot perform related business operations. In order to enable the first data to be synchronized to be used normally in the target business system, this application adjusts the first target temporary table.
[0064] When adjusting the first target temporary table, it is necessary to adjust it based on the business requirements of the target business system itself. The business requirements correspond to some requirement element information, and the requirement element information here guarantees some keywords of the business requirements. For example, if the business requirement of the target business system is to perform data query operations, the requirement element information corresponding to the data query operation includes: time period information, key factor information, etc. When the business requirement of the target business system is a query requirement, the present application adjusts the first target temporary table to the first target business table so that users can query the first target temporary table when using the target business system for queries. Specifically, the adjustment of the first target temporary table includes modifying the table attribute information of the first target temporary table so that when the target business system performs data query operations, the first target temporary table can be searched through the modified table attribute information.
[0065] As an optional embodiment in the embodiments of the present application, the present application can modify the table name of the first target temporary table to a table name containing time period information. For example, the first target temporary table is: Temporary Table A: 1, 2, 3; after adjusting Temporary Table A according to the time information, it becomes: Table 2022.6.13: 1, 2, 3. When the user queries data through the target business system and queries the data on June 13, 2022, the target business system will display the content 1, 2, 3 of Table 2022.6.13.
[0066] When generating the first target business table in the present application, it should be noted that the generated first target business table cannot conflict with the existing business tables in the target business system, that is, the generated first target business table cannot be the same as the existing business tables in the target business system. Therefore, before generating the first target business table, the present application needs to obtain the existing business tables in the target business system, and compare the table attribute information of the existing business tables with the table attribute information of the first target business table to be generated. If there are identical parts between the table attribute information of the existing business tables and the table attribute information of the first target business table to be generated, it is necessary to process the existing business tables. The processing of the existing business data tables here includes deletion or modification. By analyzing the business requirements, if it is determined that the existing business table has no value, the existing business table will be deleted. If it is determined that the existing business table still has value, the existing business table will be modified so that there is no conflict between the existing data table and the first target business table to be generated. If there are no identical parts between the table attribute information of the existing business tables and the table attribute information of the first target business table to be generated, the first target business table can be directly generated.
[0067] S104. Create a business query view according to the service requirement, and use the first target business table as the current business reference table of the business query view to complete data synchronization, so that during the data synchronization process of the target business table, normal business queries can also be performed through the business query view; the business query view includes a query statement and a business reference table that has a mapping relationship with the query statement.
[0068] In this application, after the target business system obtains the first target business table, in order to enable the first target business table to be normally used in business during data synchronization, this application creates a business query view in the target business system. From the user's perspective, a business query view is to view the data in the database from a specific perspective. From the perspective of the internal database system, a business query view is a virtual table defined by a query composed of SELECT statements. From the perspective of the internal database system, a business query view is composed of data from one or more tables. From the perspective of the external database system, a business query view is like a table. A business query view is a virtual table, and its content is defined by a query. Similar to a real table, the role of a view is similar to filtering. The filtering defined by the view can come from one or more tables in the current or other databases, or other views. In other words, a business query view is the SQL statement of a query stored in the database and a business reference table that has a mapping relationship with the query statement, that is, inputting a query statement can output a business reference table that has a mapping relationship with the query statement.
[0069] This application creates a business query view according to the service requirements of the target business system, and different service requirements correspond to different business query views. When the service requirement changes, the business query view created according to the service requirement also needs to be modified accordingly. The target business system in this application provides a view modification function, so that the method of this application can be applied to more scenarios.
[0070] In this application, the target business system performs data queries through the created business query view. Specifically, the user inputs a preset query statement into the target business system through an operation, and the target business system displays the business reference table that has a mapping relationship with the query statement input by the user to the user. There are multiple business query views in the target business system, the query statements of each business query view are fixed, and the query statements between different business query views are different. For example, the target business system includes view A and view B. View A corresponds to query statement A and business reference table A, and view B corresponds to query statement B and business reference table B. When the user inputs query statement A or B in the target business system, the target business system outputs business reference table A or B.
[0071] In an optional embodiment of this applicationFigure 2 As shown in Figure 2 , the data synchronization method in this application further includes:
[0072] S201. According to the data synchronization requirement, obtain all the second data to be synchronized from the second target data table in the source data system that can meet the data synchronization requirement;
[0073] S202. When detecting the second target temporary table corresponding to the second target data table, write the second data to be synchronized into the second target temporary table;
[0074] S203. Based on the requirement element information corresponding to the service requirement, adjust the second target temporary table to generate a second target service table;
[0075] S204. Use the second target service table to replace the first target service table as the current service pointing table of the service query view.
[0076] When the target service system in this application obtains the data synchronization requirement, a second target service table is generated. The specific generation method is the same as that of the first target service table, and will not be elaborated here. At this time, the current service pointing table of the service query view is still the first target service table. After obtaining the second target service table in this application, the second target service table is used to replace the first target service table. That is to say, after obtaining the second target service table in this application, the second target service table serves as the current service pointing table of the service query view. When inputting a query statement, the output is the second target service table. The first target service table is converted into a historical service pointing table. This application can select whether to retain the historical service pointing table according to the retention requirement of the target service system for the historical service pointing table. For multiple service pointing tables, selective deletion or retention can be performed. For example, the current service pointing table is: Table 6.13, and the historical service pointing tables include: Table 6.10, Table 6.11, Table 6.12, Table 6.13. The retention requirement is a cycle of three days, then Table 6.11, Table 6.12, and Table 6.13 can be retained, and Table 6.10 can be deleted.
[0077] As an optional embodiment in the embodiments of this application, the service requirement in this application may change. When the service requirement changes, select a target service pointing table from the current service pointing table and the historical service pointing table, and use the target service pointing table as the new current service pointing table to meet the changed service requirement. For example, the service requirement of 6.13 changes from querying the data of the current day to querying the data of the previous day. At this time, Table 6.12 needs to be used as the target service pointing table for feedback.
[0078] Figure 3The figure shows a schematic structural diagram of a data synchronization device provided by an embodiment of the present application. The device includes:
[0079] An acquisition module, configured to obtain all of the first data to be synchronized from a first target data table that can meet the data synchronization requirements in the source data system according to the data synchronization requirements.
[0080] A writing module, configured to write the first data to be synchronized into the first target temporary table if the first target temporary table corresponding to the first target data table is detected.
[0081] A generation module, configured to adjust the first target temporary table based on the requirement element information corresponding to the service requirements to generate a first target service table.
[0082] A view module, configured to create a service query view according to the service requirements, and use the first target service table as the current service pointing table of the service query view to complete data synchronization, so that during the data synchronization process of the target service table, normal service queries can also be performed through the service query view; the service query view includes a query statement and a service pointing table that has a mapping relationship with the query statement.
[0083] The acquisition module is further configured to obtain all of the second data to be synchronized from a second target data table that can meet the data synchronization requirements in the source data system according to the data synchronization requirements.
[0084] The writing module is further configured to write the second data to be synchronized into the second target temporary table when the second target temporary table corresponding to the second target data table is detected.
[0085] The generation module is further configured to adjust the second target temporary table based on the requirement element information corresponding to the service requirements to generate a second target service table.
[0086] The view module is further configured to use the second target service table to replace the first target service table as the current service pointing table of the service query view.
[0087] The view module is further configured to transform the first target service table into a historical service pointing table and process the historical service pointing table according to the retention requirement for the historical service pointing table.
[0088] When the service requirements change, select a target service pointing table from the current service pointing table and the historical service pointing table according to the changed service requirements; and use the target service pointing table as the new current service pointing table to meet the changed service requirements.
[0089] The writing module is further configured to generate the first target temporary table corresponding to the first target data table if the first target temporary table corresponding to the first target data table is not detected, and write the first data to be synchronized into the first target temporary table.
[0090] The service requirement is a query requirement, and the requirement element information is time period information;
[0091] Adjusting the first target temporary table based on the requirement element information corresponding to the service requirement to generate a first target service table includes:
[0092] Modifying the table attribute information of the first target temporary table based on the time period information of the query requirement to generate a first target service table.
[0093] The table attribute information is the table name; the first target service table is generated by the following method:
[0094] Modifying the table name of the first target temporary table based on the time period information of the query requirement to generate a first target service table; the table name of the first target service table includes the time period information.
[0095] As Figure 4 shown, an embodiment of the present application provides an electronic device for executing the data synchronization method in the present application. The device includes a memory, a processor, a bus, and a computer program stored on the memory and executable on the processor. Wherein, when the above-mentioned processor executes the above-mentioned computer program, the steps of the above-mentioned data synchronization method are implemented.
[0096] Specifically, the above-mentioned memory and processor can be general memory and processor, which are not specifically limited here. When the processor runs the computer program stored in the memory, it can execute the above-mentioned data synchronization method.
[0097] Corresponding to the data synchronization method in the present application, an embodiment of the present application further provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the above-mentioned data synchronization method are executed.
[0098] Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, etc. When the computer program on the storage medium is run, the above-mentioned data synchronization method can be executed.
[0099] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. The system embodiments described above are merely illustrative. For example, the division of the units is only a logical functional division. In actual implementation, there may be other division methods. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections between each other can be through some communication interfaces. The indirect couplings or communication connections of systems or units can be electrical, mechanical or other forms.
[0100] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0101] In addition, each functional unit in the embodiments provided in this application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.
[0102] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.
[0103] It should be noted that similar reference numerals and letters indicate similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In addition, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.
[0104] Finally, it should be noted that the above-described embodiments are only specific implementation manners of the present application, used to illustrate the technical solutions of the present application, rather than limiting it. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that any person skilled in the art within the technical scope disclosed by the present application can still modify the technical solutions recorded in the foregoing embodiments or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. All should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.
Claims
1. A data synchronization method, characterized in that, Applied to a target business system, the method includes: During the business execution process, according to the data synchronization requirement, obtain all the first data to be synchronized from the first target data table included in the source data system that can meet the data synchronization requirement. If the first target temporary table corresponding to the first target data table is detected, write the first data to be synchronized into the first target temporary table. Based on the requirement element information corresponding to the business requirement, adjust the first target temporary table to generate a first target business table. According to the business requirement, create a business query view, and use the first target business table as the current business pointing table of the business query view to complete data synchronization, so that during the data synchronization process of the target business table, normal business queries can also be performed through the business query view; the business query view includes a query statement and a business pointing table that has a mapping relationship with the query statement. When the business requirement changes, the method further includes: According to the data synchronization requirement, obtain all the second data to be synchronized from the second target data table included in the source data system that can meet the data synchronization requirement. When the second target temporary table corresponding to the second target data table is detected, write the second data to be synchronized into the second target temporary table. Based on the requirement element information corresponding to the business requirement, adjust the second target temporary table to generate a second target business table. Use the second target business table to replace the first target business table as the current business pointing table of the business query view.
2. The method according to claim 1, wherein When the first target business table becomes a historical business pointing table, the method further includes: Process the historical business pointing table according to the retention requirement for the historical business pointing table.
3. The method according to claim 2, wherein When the business requirement changes, the method further includes: According to the changed business requirement, select a target business pointing table from the current business pointing table and the historical business pointing table; and use the target business pointing table as the new current business pointing table to meet the changed business requirement.
4. The method according to claim 1, wherein The method further includes: If the first target temporary table corresponding to the first target data table is not detected, generate the first target temporary table corresponding to the first target data table, and write the first data to be synchronized into the first target temporary table.
5. The method according to claim 1, wherein The business requirement is a query requirement, and the requirement element information is time period information. The step of adjusting the first target temporary table based on the requirement element information corresponding to the business requirement to generate a first target business table includes: Based on the time period information of the query requirement, modify the table attribute information of the first target temporary table to generate a first target business table.
6. The method according to claim 5, wherein The table attribute information is the table name; the first target business table is generated in the following way: Based on the time period information of the query requirement, modify the table name of the first target temporary table to generate a first target business table; the table name of the first target business table includes the time period information.
7. A data synchronization device, characterized in that, The device includes: An acquisition module, configured to acquire all the first data to be synchronized from the first target data table that can meet the data synchronization requirement in the source data system according to the data synchronization requirement during the execution of the service. A writing module, configured to write the first data to be synchronized into the first target temporary table if the first target temporary table corresponding to the first target data table is detected. A generation module, configured to adjust the first target temporary table based on the requirement element information corresponding to the service requirement to generate a first target service table. A view module, configured to create a service query view according to the service requirement, and use the first target service table as the current service pointing table of the service query view to complete data synchronization, so that during the data synchronization process of the target service table, normal service queries can also be performed through the service query view; the service query view includes a query statement and a service pointing table that has a mapping relationship with the query statement. When the service requirement changes, the acquisition module is further configured to acquire all the second data to be synchronized from the second target data table that can meet the data synchronization requirement in the source data system according to the data synchronization requirement. The writing module is further configured to write the second data to be synchronized into the second target temporary table when the second target temporary table corresponding to the second target data table is detected. The generation module is further configured to adjust the second target temporary table based on the requirement element information corresponding to the service requirement to generate a second target service table. The view module is further configured to use the second target service table to replace the first target service table as the current service pointing table of the service query view.
8. An electronic device, characterized in that, It includes: A processor, a memory and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the data synchronization method according to any one of claims 1 to 6 are executed.
9. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium. When the computer program is run by the processor, the steps of the data synchronization method according to any one of claims 1 to 6 are executed.
Citation Information
Patent Citations
Data extraction method and device
CN112835879A
Method and device for importing data, electronic equipment and storage medium
CN114064783A
Data service construction method and device, electronic equipment and storage medium
CN114547189A