Data synchronization method, device and equipment based on multi-dimensional screening rule

Abstract

The invention provides a data synchronization method, device and equipment based on a multi-dimensional screening rule, and relates to the technical field of database management, and the method comprises the steps: obtaining the multi-dimensional screening rule configured by a target enterprise; the target enterprise is an enterprise of which the database is to be privately deployed; according to the multi-dimensional screening rule, screening target data from a target data table in a configured source database; generating a target SQL file according to the target data; and sending the target SQL file to a private deployment database corresponding to the target enterprise, so that the private deployment database completes target data synchronization by executing the target SQL file. The method supports multi-dimensional rule screening, and adapts to various business requirements; different screening rules are configured for different data table types, so that different data tables in the same database are synchronized according to different synchronization rules; executing and transmitting the synchronous script through SQL (Structured Query Language) merging and compression optimization; more flexible, more efficient and more reliable data synchronization is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database management, in particular to a data synchronization method and device based on multi-dimensional filtering rules and equipment. BACKGROUND

[0002] When an enterprise deploys a database privately, the database needs to be deployed independently on a local server and the original data needs to be synchronized. Enterprise data synchronization involves a large amount of data and different levels of business data. With the increasing trend of multi-tenant architecture, group deployment, and private government and enterprise systems, there are problems such as non-uniform data structures (field alias), complex table structures (including different business modules), large amounts of data (especially trajectory and log tables), and personalized synchronization requirements (each project company has different requirements) between systems. The existing synchronization methods that use full synchronization or single-dimensional filtering cannot solve the above problems and cannot meet the needs of efficient and accurate synchronization. SUMMARY

[0003] The purpose of the embodiments of the present application is to provide a data synchronization method, device and equipment based on multi-dimensional filtering rules, which can solve the problems of data redundancy, resource waste and low efficiency in the prior art, and can significantly improve the data synchronization efficiency.

[0004] In a first aspect, the present application provides a data synchronization method based on multi-dimensional filtering rules, which comprises:

[0005] Obtaining multi-dimensional filtering rules configured by a target enterprise; wherein the target enterprise is an enterprise to which a database is to be deployed privately;

[0006] According to the multi-dimensional filtering rules, target data is filtered from a target data table in a configured source database;

[0007] According to the target data, a target SQL file is generated;

[0008] The target SQL file is sent to a private deployment database corresponding to the target enterprise, so that the private deployment database completes target data synchronization by executing the target SQL file.

[0009] In an optional embodiment, the multi-dimensional filtering rules include different data table types and corresponding filtering rules for the data table types;

[0010] The data table types include a first data table type and a second data table type;

[0011] When the data table type is the first data table type, the screening rule includes: target screening fields corresponding to different screening dimensions and target screening conditions corresponding to each target screening field; different screening dimensions have different priorities;

[0012] The target screening field includes the enterprise ID of the target enterprise, the department IDs of different departments included in the target enterprise, and the employee IDs of different employees included in the target enterprise.

[0013] In an optional embodiment, before filtering the target data from the target data table in the configured source database according to the multi-dimensional filtering rule, the method further includes:

[0014] Filtering a data table containing the target screening field from a configured field list to obtain a first target data table; wherein the field list includes the table name, data table type, and fields contained in each data table in the source database whose data table type is the first data table type;

[0015] A data table of the second data table type is selected from the source database to obtain a second target data table.

[0016] In an optional embodiment, the multidimensional screening rule further includes: a time parameter; the target data includes first target data and second target data;

[0017] Filtering target data from a target data table in a configured source database according to the multi-dimensional filtering rules includes:

[0018] Filtering initial data from the first target data table according to the target screening field and the target screening condition;

[0019] If the data volume of the initial data is not greater than a configured first threshold, determining the initial data as the first target data;

[0020] If the data volume of the initial data is greater than a configured first threshold, first target data is filtered from the initial data according to the time parameter.

[0021] In an optional embodiment, filtering initial data from the first target data table according to the target screening field and the target screening condition includes:

[0022] From each target screening field in the first target data table, select the field to be screened with the highest priority in the screening dimension as the field to be screened;

[0023] Data that meets the screening rule corresponding to the field to be screened is screened from the first target data table to obtain initial data.

[0024] In an optional embodiment, filtering target data from a target data table in a configured source database according to the multi-dimensional filtering rule further includes:

[0025] Filtering second target data from the second target data table according to a filtering rule corresponding to the second data table type;

[0026] Generate a target SQL file based on the target data, including:

[0027] Generate a first SQL file according to the first target data;

[0028] Generate a second SQL file according to the second target data;

[0029] The first SQL file and the second SQL file are merged to obtain a target SQL file.

[0030] In a second aspect, the present invention provides a data synchronization device based on multi-dimensional screening rules, the device comprising:

[0031] An acquisition unit, configured to acquire a multi-dimensional screening rule configured for a target enterprise; wherein the target enterprise is an enterprise for which a database is to be privately deployed;

[0032] A screening unit, configured to screen target data from a target data table in a configured source database according to the multi-dimensional screening rule;

[0033] A generating unit, configured to generate a target SQL file according to the target data;

[0034] The synchronization unit is configured to send the target SQL file to the private deployment database corresponding to the target enterprise, so that the private deployment database completes target data synchronization by executing the target SQL file.

[0035] In a third aspect, the present invention provides an electronic device, comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0036] Memory for storing computer programs;

[0037] The processor is configured to implement any of the methods described in the foregoing embodiments when executing the program stored in the memory.

[0038] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of the aforementioned embodiments is implemented.

[0039] This application supports multi-dimensional rule filtering to adapt to various business needs; by configuring different filtering rules for different data table types, different data tables in the same database can be synchronized with different synchronization rules; for statistical data tables that only require incremental synchronization, incremental synchronization is implemented to reduce unnecessary data synchronization and improve performance; for rule data tables that require full synchronization, full synchronization is implemented to ensure data integrity; the execution and transmission of synchronization scripts are optimized through SQL merging and compression; and more flexible, efficient and reliable data synchronization is achieved.

[0040] This application can significantly improve data synchronization efficiency, and is particularly suitable for the private deployment scenarios of smart sanitation systems, providing strong technical support for system operation and maintenance.

[0041] This application has strong versatility: it can be applied to data synchronization scenarios with multiple databases and inconsistent field styles; high synchronization efficiency: incremental synchronization after filtering according to filtering conditions, avoiding invalid full data, and solving the problem of huge resource waste caused by full synchronization of existing technologies; strong maintainability: various synchronization rules, field normalization, and dimension definitions can be customized by users according to actual needs, solving the problem that synchronization scripts in existing technologies are difficult to maintain and cannot adapt to business changes; support for heterogeneous table structures: it can adapt to fields with different names but consistent semantics, such as department_id, orgId, accUnitId, solving the problem that synchronization logic cannot be reused due to different field naming and table structure differences in existing technologies; high engineering practicality: support for export merging, compression and packaging, suitable for cross-platform deployment; and realizes intelligent synchronization under heterogeneous database structures. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0043] Figure 1 A flowchart of a data synchronization method based on multi-dimensional screening rules provided in an embodiment of the present application;

[0044] Figure 2 A schematic diagram of the structure of a data synchronization device based on multi-dimensional screening rules provided in an embodiment of the present application;

[0045] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] With reference to the drawings and embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only some of the embodiments of the present application, but not all of the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts are within the scope of the present application.

[0047] The data synchronization method based on the multi-dimensional screening rule provided in the embodiments of the present application can be applied in a server or a terminal with strong computing capability. The server can be a physical server, a server cluster composed of multiple physical servers or a distributed system, or a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, content delivery network (CDN), and big data and artificial intelligence platform. The terminal can be a user equipment (UE) such as a mobile phone, a smart phone, a notebook computer, a digital broadcast receiver, a personal digital assistant (PDA), a tablet computer (PAD), a handheld device, a vehicle-mounted device, a wearable device, a computing device or other processing device connected to a wireless modem, a mobile station (MS), a mobile terminal (Mobile Terminal), etc. The terminal and the server can be directly or indirectly connected through wired or wireless communication, which is not limited in the present application.

[0048] The preferred embodiments of the present application are described below in conjunction with the accompanying drawings of the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application, and the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0049] Figure 1 A flowchart of a data synchronization method based on a multi-dimensional screening rule provided in the embodiments of the present application is shown in FIG. 1. As shown in FIG. 1, the method can include the following steps. Figure 1

[0050] In S110, a multi-dimensional screening rule configured by a target enterprise is obtained; and target data is screened from a target data table in a configured source database according to the multi-dimensional screening rule.

[0051] In the embodiments of the present application, a visual interface is provided; a user can complete the configuration of the multi-dimensional screening rule on the visual interface according to business requirements; and the user also needs to input the name of the database to complete the synchronization of the corresponding database.

[0052] ​In the embodiments of the present application, a plurality of sub-databases are provided in the source database, each of which corresponds to a functional module in the intelligent environmental sanitation management system; each of the sub-databases stores data tables of at least three types of data tables; the first type is data statistics type, which is used to store various operation data, including basic data, operation and maintenance data and historical data; the second type is rule configuration type, which includes configuration tables and metadata tables, and is used to store configuration information, business rules and the like; the third type is log type and table structure type, which is used to store operation logs and large amount of historical tables and table structures; the data statistics type is taken as the first data table type; the rule configuration type is taken as the second data table type; and the other data tables except the first and second data table types are taken as the third data table type.

[0053] In the embodiments of the present application, different filtering rules are configured for different types of data tables; when the data table type is the first data table type (i.e. the data statistics type), conditional synchronization is adopted, and the specific filtering rule includes: a first time parameter, target filtering fields corresponding to different filtering dimensions, and target filtering conditions corresponding to each target filtering field; the target filtering fields include enterprise ID of a target enterprise, department ID of different departments contained in the target enterprise, employee ID of different employees contained in the target enterprise, and ID of vehicles and other facilities contained in the target enterprise; the first time parameter is used to represent how long the statistical data in the past is retained;

[0054] When the data table type is the second data table type (i.e. the non-data statistics type), full-amount synchronization is adopted; the specific filtering rule can include a second time parameter, when the second time parameter is included, the second time parameter is used to represent how long the data in the past is retained; when the second time parameter is not included, it means that this time of synchronization is to synchronize all data;

[0055] When the data table type is the third data table type, the filtering rule is structure synchronization, which is used to obtain only the structure and built-in rules of the table, and does not synchronize any data.

[0056] In some embodiments of the present application, the user can change the type of different data tables and the corresponding filtering rules; when the target enterprise needs to synchronize log type tables such as operation logs, the type of the corresponding table can be modified to the data statistics type, or the filtering rules of the log type and the table structure type are set to conditional synchronization, i.e. the corresponding data can be filtered according to the conditions; when no specific filtering condition is set in this scenario, the corresponding log type table will be synchronized in full amount.

[0057] In the embodiments of the present application, the target filtering condition is to specifically retain data whose value under the target filtering field is what; for example, the target filtering field is company ID, and the target filtering condition is company 1234, i.e. only data with company ID 1234 is retained.

[0058] In the embodiment of the present application, the screening dimensions include enterprise dimension, department dimension and employee dimension; the priorities of different screening dimensions are different; the priority of the enterprise dimension is higher than that of the department dimension; the priority of the department dimension is higher than that of the employee dimension.

[0059] In another embodiment of the present application, the screening dimensions further include user, vehicle and route; and the corresponding target screening field is obtained by matching the company ID or employee ID from the vehicle management table or the dispatching table.

[0060] In the embodiment of the present application, the conditional synchronization is adopted for each type of data statistical table, so as to screen the relevant data of the target enterprise; and the full-amount synchronization is adopted for the configuration type data table or other data table, so as to ensure the completeness of the configuration rules and other data of the target enterprise, and to realize the high-efficiency and high-completeness data synchronization.

[0061] In the embodiment of the present application, the target data table includes a first target data table and a second target data table.

[0062] In the embodiment of the present application, the data table containing the target screening field is screened from the configured field list to obtain the first target data table; and the data table of the second data table type is selected from the source database to obtain the second target data table.

[0063] In another embodiment of the present application, the second target data table further includes the data table of the first data table type, which is not screened by the screening rule corresponding to the first data table type, so as to ensure the data synchronization completeness.

[0064] In the embodiment of the present application, the field list includes the table name, the data table type and the fields contained in each data table of the first data table type in the source database.

[0065] In the embodiment of the present application, the generation method of the field list includes:

[0066] All the fields contained in each data table of the first data table type in the source database are counted; the counted all the fields are de-duplicated; and the field list is generated according to the de-duplicated fields, the table name and the data table type of each data table.

[0067] In the embodiment of the present application, the target data includes first target data and second target data.

[0068] In the embodiment of the present application, the target data is screened from the target data table in the configured source database according to the multi-dimensional screening rule, including:

[0069] The initial data is screened from the first target data table according to the target screening field and the target screening condition;

[0070] If the data volume of the initial data is not greater than the configured first threshold, the initial data is determined as the first target data;

[0071] If the amount of the initial data is greater than a configured first threshold, the first target data is filtered from the initial data according to the time parameter;

[0072] The second target data is filtered from the second target data table according to the filtering rule corresponding to the second data table type.

[0073] In this embodiment of the present application, the first threshold is customized by the target enterprise based on actual business needs. When the amount of initial data exceeds the configured first threshold, it indicates that there is too much data to be synchronized. In this case, incremental synchronization can be performed on the filtered initial data according to the time window configured by the user, retaining only the most recent data to reduce redundancy. For example, if the user configures the time parameter to be 3 months and the first threshold to be 10GB, when the initial data exceeds 10GB, only the data from the most recent 3 months of the initial data will be retained as the first target data.

[0074] In an embodiment of the present application, when the user does not set the second time parameter, the second target data is all data in the second target data table; when the user sets the second time parameter, the data that meets the conditions is filtered according to the time parameter.

[0075] In the embodiment of the present application, the time window mechanism is combined to dynamically adjust the synchronization range to ensure the efficiency of incremental synchronization; by setting time parameters, multi-dimensional screening and independent table status management are supported to reduce unnecessary data synchronization.

[0076] In the embodiment of the present application, filtering initial data from the first target data table according to the target screening field and the target screening condition includes:

[0077] From each target screening field in the first target data table, select the field to be screened with the highest priority in the screening dimension as the field to be screened;

[0078] Data that meets the screening rules corresponding to the to-be-screened fields is screened from the first target data table to obtain initial data.

[0079] For example, when a data table contains both company ID and employee ID, the data table is filtered by company ID first. When the data table does not contain company ID and department ID, employee ID is used as the field to be filtered.

[0080] S120 . Generate a target SQL file based on the target data; and send the target SQL file to the private deployment database corresponding to the target enterprise, so that the private deployment database completes target data synchronization by executing the target SQL file.

[0081] In an embodiment of the present application, after obtaining the target SQL file method, the method further includes:

[0082] If the file size of the target SQL file is greater than the configured second threshold, the target SQL file is compressed and sent to the private deployment database corresponding to the target enterprise.

[0083] In the embodiment of the present application, multiple generated SQL files are merged to form a complete executable SQL script; the SQL files are compressed to simplify the data deployment process and facilitate transmission and deployment. Specifically, the merge_sql and merge_and_compress_sql methods can be used to merge and compress the SQL scripts.

[0084] This application implements conditional synchronization, full synchronization, and skip synchronization through the above technical solution; conditional synchronization requires not only field hits but also value hits to generate export statements with WHERE xxx IN(...); full synchronization is the full export of certain configuration tables; skip synchronization means that log tables and large historical tables can skip data export, retaining only the structure or completely excluding it.

[0085] In some embodiments of the present application, the method further includes: generating a unique serial number for the target SQL file obtained for each data synchronization; the serial number includes information such as the company ID and synchronization date of the target enterprise; associating the target SQL file with the target data; when synchronizing data, the target enterprise can determine whether the corresponding target data is the latest version based on the serial number of the target SQL file. When the target data corresponding to the target SQL file is not the latest version, the data is re-synchronized, or merged with the latest version of the target data, or conflict resolution is performed; thereby effectively resolving conflict problems in the data synchronization process and ensuring that the branch obtains the latest and consistent data.

[0086] In some embodiments of the present application, the method also includes: setting a trigger for the rule configuration data table in each sub-database in the source database or the third target data table in the source database selected by the target enterprise, and the trigger is used to generate a data change message when the third target data table changes, and send the generated data change message to the configured message queue, and the message queue corresponds to a queue ID; when the data change message is stored in the message queue, according to the comparison table of different queue IDs and different company IDs configured, the target enterprise corresponding to the message queue is matched, and the data change message is sent to the private deployment database configuration consumer of the target enterprise, thereby realizing automatic synchronization of rule configuration data and ensuring the stability, sequence, flexibility and reliability of data synchronization.

[0087] In another embodiment of the present application, a data synchronization method based on multi-dimensional screening rules includes:

[0088] Step 1: Field analysis and screening:

[0089] Extract all fields from the target database and determine the fields that need to be synchronized through field analysis; manually or automatically filter the list of fields of interest;

[0090] Step 2: Conditional filtering synchronization:

[0091] According to the filtering rules, filter the target data table by field name and conditions; export the data of each table that meets the conditions and generate an SQL file;

[0092] Step 3, incremental synchronization:

[0093] For large tables or tables with large data volumes, synchronize data within the most recent T time period according to the time window; generate SQL scripts for incremental synchronization;

[0094] Step 4: Full synchronization:

[0095] For tables that are not filtered (including rule configuration tables, etc.), perform full data synchronization to ensure data coverage;

[0096] Step 5: SQL merge and compression:

[0097] Merge all generated SQL files into a complete SQL script; compress the SQL files to optimize transmission and execution efficiency.

[0098] In an embodiment of the present application, the intelligent sanitation management system includes:

[0099] The analysis module is used to read the database structure, extract and normalize all field names (such as deptId, department_id);

[0100] Dimension configuration module, used to dynamically extract filter values ​​based on dimensions such as company, department, employee, and vehicle;

[0101] The script generation module is used to generate SQL scripts based on field hits and policy selection synchronization mode (conditional filtering / full / skip);

[0102] Compression module, used to merge all exported SQL files and compress them into a deployment package;

[0103] Auxiliary control module, used for manual control of specific business tables (such as log skipping and large table structure export).

[0104] In an embodiment of the present application, each database of the intelligent sanitation management system can also be configured separately: skip the exported log table, backup table, historical track sub-table, etc.; specific field names such as driverId, regionId can be customized for a single database; for basic dependent tables (such as organizational structure), "priority export" is supported to avoid synchronous deadlock.

[0105] Corresponding to the above method, the embodiment of the present application also provides a data synchronization device based on multi-dimensional screening rules, such as Figure 2 As shown, the data synchronization device based on multi-dimensional screening rules includes:

[0106] An acquisition unit 210 is configured to acquire a multi-dimensional screening rule configured by a target enterprise; wherein the target enterprise is an enterprise whose database is to be privately deployed;

[0107] A screening unit 220 is configured to screen target data from a target data table in a configured source database according to multi-dimensional screening rules;

[0108] A generating unit 230 is configured to generate a target SQL file based on the target data;

[0109] The synchronization unit 240 is configured to send the target SQL file to the private deployment database corresponding to the target enterprise, so that the private deployment database completes target data synchronization by executing the target SQL file.

[0110] The functions of each functional unit of the data synchronization device based on multi-dimensional screening rules provided in the above embodiments of the present application can be realized through the above-mentioned method steps. Therefore, the specific working process and beneficial effects of each unit in the data synchronization device based on multi-dimensional screening rules provided in the embodiments of the present application will not be repeated here.

[0111] The present application also provides an electronic device, such as Figure 3 As shown, it includes a processor 310 , a communication interface 320 , a memory 330 and a communication bus 340 , wherein the processor 310 , the communication interface 320 , and the memory 330 communicate with each other via the communication bus 340 .

[0112] Memory 330, for storing computer programs;

[0113] The processor 310 is configured to execute the program stored in the memory 330 by performing the following steps:

[0114] Obtaining multi-dimensional screening rules configured for a target enterprise; wherein the target enterprise is an enterprise whose database is to be privatized;

[0115] Filter target data from the target data table in the configured source database according to multi-dimensional filtering rules;

[0116] Generate target SQL file based on target data;

[0117] The target SQL file is sent to the private deployment database corresponding to the target enterprise, so that the private deployment database completes the target data synchronization by executing the target SQL file.

[0118] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0119] The communication interface is used for communication between the above electronic device and other devices.

[0120] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0121] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0122] The implementation methods and beneficial effects of the various components of the electronic device in the above embodiments to solve the problems can be found in Figure 1 The various steps in the embodiment shown are implemented, therefore, the specific working process and beneficial effects of the electronic device provided by the embodiment of the present application are not repeated here.

[0123] In another embodiment provided in the present application, a computer-readable storage medium is also provided, which stores instructions. When the computer-readable storage medium is run on a computer, it enables the computer to execute the data synchronization method based on multidimensional screening rules described in any of the above embodiments.

[0124] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the data synchronization method based on multi-dimensional screening rules described in any one of the above embodiments.

[0125] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the embodiments of the present application can be implemented in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the embodiments of the present application can be implemented in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0126] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0127] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0129] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic creative concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0130] Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present application without departing from the spirit and scope of the embodiments of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims and their equivalents, the embodiments of the present application are also intended to include these modifications and variations.

Claims

1. A data synchronization method based on multi-dimensional screening rules, characterized in that: The method comprises: Obtaining multi-dimensional screening rules configured for a target enterprise; wherein the target enterprise is an enterprise for which a database is to be privately deployed; Filter target data from a target data table in a configured source database according to the multidimensional filtering rules; Generate a target SQL file according to the target data; The target SQL file is sent to the private deployment database corresponding to the target enterprise, so that the private deployment database completes target data synchronization by executing the target SQL file.

2. The method according to claim 1, wherein The multidimensional screening rules include different data table types and screening rules corresponding to the corresponding data table types; The data table types include: a first data table type and a second data table type; When the data table type is the first data table type, the screening rule includes: target screening fields corresponding to different screening dimensions and target screening conditions corresponding to each target screening field; different screening dimensions have different priorities; The target screening field includes the enterprise ID of the target enterprise, the department IDs of different departments included in the target enterprise, and the employee IDs of different employees included in the target enterprise.

3. The method according to claim 2, wherein Before filtering target data from a target data table in a configured source database according to the multi-dimensional filtering rule, the method further includes: Filtering a data table containing the target screening field from a configured field list to obtain a first target data table; wherein the field list includes the table name, data table type, and fields contained in each data table in the source database whose data table type is the first data table type; A data table of the second data table type is selected from the source database to obtain a second target data table.

4. The method according to claim 3, wherein The multi-dimensional screening rule further includes: a time parameter; the target data includes first target data and second target data; Filtering target data from a target data table in a configured source database according to the multi-dimensional filtering rules includes: Filtering initial data from the first target data table according to the target screening field and the target screening condition; If the data volume of the initial data is not greater than a configured first threshold, determining the initial data as the first target data; If the data volume of the initial data is greater than a configured first threshold, first target data is filtered from the initial data according to the time parameter.

5. The method according to claim 4, wherein Filtering initial data from the first target data table according to the target screening field and the target screening condition includes: From each target screening field in the first target data table, select the field to be screened with the highest priority in the screening dimension as the field to be screened; Data that meets the screening rule corresponding to the field to be screened is screened from the first target data table to obtain initial data.

6. The method according to claim 4, wherein Filtering target data from a target data table in a configured source database according to the multi-dimensional filtering rules also includes: Filtering second target data from the second target data table according to a filtering rule corresponding to the second data table type; Generate a target SQL file based on the target data, including: Generate a first SQL file according to the first target data; Generate a second SQL file according to the second target data; The first SQL file and the second SQL file are merged to obtain a target SQL file.

7. A data synchronization device based on multi-dimensional screening rules, characterized in that: The device comprises: An acquisition unit, configured to acquire a multi-dimensional screening rule configured for a target enterprise; wherein the target enterprise is an enterprise for which a database is to be privately deployed; A screening unit, configured to screen target data from a target data table in a configured source database according to the multi-dimensional screening rule; A generating unit, configured to generate a target SQL file according to the target data; The synchronization unit is configured to send the target SQL file to the private deployment database corresponding to the target enterprise, so that the private deployment database completes target data synchronization by executing the target SQL file.

8. An electronic device, characterized in that: The electronic device includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; A processor, configured to implement the method according to any one of claims 1 to 6 when executing a program stored in a memory.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images