Data synchronization detection method and device

Abstract

The present application discloses a data synchronization detection method and device, which belongs to the field of computer technology. The method includes executing an offline data extraction timing task, wherein the offline data extraction timing task is used to synchronize the data in the table to be synchronized to the offline data warehouse table at a preset time; calling a database query application program interface API to obtain the value returned by the database query API; performing data synchronization detection on the table to be synchronized and the offline data warehouse table according to the value returned by the database query API to obtain a data synchronization detection result.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and specifically relates to a data synchronization detection method and device. Background Art

[0002] With the widespread application of big data technology, offline data synchronization is becoming more and more common, especially extracting data from databases offline to data warehouses for storage. Synchronized data needs to be checked for success and for changes in the database table structure to avoid affecting data synchronization.

[0003] The existing detection method is to determine whether the data synchronization is successful based on whether the return code of the synchronization data task is 0. When the database table structure changes, the offline data warehouse table structure is manually adjusted according to the database table structure.

[0004] Judging whether data extraction is successful based on the return code of the synchronization data task is not accurate enough. In addition, when the database table structure changes, the data warehouse administrator cannot be notified in time, resulting in data synchronization failure. The database administrator needs to be notified in time, and the data warehouse administrator also needs to manually adjust the offline data warehouse table structure, which has a high maintenance cost. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a data synchronization detection method and device, which can solve the problems of low accuracy of data synchronization detection and high manual maintenance cost during data synchronization.

[0006] In a first aspect, an embodiment of the present application provides a data synchronization detection method, the method comprising:

[0007] Execute an offline data extraction scheduled task, wherein the offline data extraction scheduled task is used to synchronize the data in the table to be synchronized to the offline data warehouse table at a preset time interval;

[0008] Calling a database query application program interface (API) to obtain a value returned by the database query API;

[0009] According to the value returned by the database query API, a data synchronization test is performed on the table to be synchronized and the offline data warehouse table to obtain a data synchronization test result.

[0010] In a second aspect, an embodiment of the present application provides a data synchronization detection device, the device comprising:

[0011] A timing extraction module, used to execute an offline data extraction timing task, wherein the offline data extraction timing task is used to synchronize the data in the table to be synchronized to the offline data warehouse table at a preset time interval;

[0012] A data query module is used to call a database query application program interface (API) to obtain a value returned by the database query API;

[0013] The synchronization detection module is used to perform data synchronization detection on the table to be synchronized and the offline data warehouse table according to the value returned by the database query API to obtain a data synchronization detection result.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0015] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0017] In an embodiment of the present application, data is synchronized to an offline data warehouse table by executing an offline data extraction scheduled task, and then data synchronization detection is performed through the return value of the database query API, thereby improving the accuracy of data extraction detection and reducing the manual maintenance cost during the data synchronization process. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flowchart of the data synchronization detection method provided by the present application;

[0019] Figure 2 It is a flowchart of updating the table structure of an offline data warehouse table provided by this application;

[0020] Figure 3 It is a flowchart of creating the offline data warehouse table provided by this application;

[0021] Figure 4 A schematic diagram of the structure of the data synchronization detection device provided by this application;

[0022] Figure 5 A schematic diagram of the structure of the electronic device provided for this application;

[0023] Figure 6 A schematic diagram of the hardware structure of the electronic device provided in this application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0025] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0026] The following is a detailed description of the data synchronization detection method provided by the embodiment of the present application through some embodiments and their application scenarios in conjunction with the accompanying drawings. The data synchronization detection method provided by the embodiment of the present application is executed by an electronic device, and the electronic device mentioned in the embodiment of the present application includes but is not limited to a mobile phone, a tablet computer, a computer, a wearable device, etc.

[0027] Figure 1 A flow chart of a data synchronization detection method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method comprises the following steps:

[0028] Step 100: Execute an offline data extraction scheduled task, wherein the offline data extraction scheduled task is used to synchronize the data in the table to be synchronized to the offline data warehouse table at a preset time interval.

[0029] Optionally, execute the offline data extraction scheduled task to synchronize data. The parameters that need to be passed in include: the table name of the table to be synchronized, the database type of the table to be synchronized, the database IP and port number of the table to be synchronized, and the table name and where condition of the offline data warehouse table.

[0030] The where condition is used for offline data extraction judgment. The offline data extraction scheduled task synchronizes the data in the table to be synchronized to the offline data warehouse table at preset intervals according to the input parameters.

[0031] Step 101: Call a database query application program interface (API) to obtain a value returned by the database query API.

[0032] Optionally, the parameters that need to be passed in when calling the database query application programming interface (API) include: operation type, table name of the table to be synchronized, database type of the table to be synchronized, database IP and port number of the table to be synchronized, table name of the offline data warehouse table and where condition.

[0033] The operation type indicates the query operation type of the database query API, for example, table creation, primary key determination, or quantity determination.

[0034] Optionally, the electronic device calls the database query API, and obtains the value returned by the database query API corresponding to the parameter passed in according to the parameter passed in. The return value is the data information in the table to be synchronized and the offline data warehouse table, which can reflect whether the data information in the table to be synchronized and the offline data warehouse table is consistent.

[0035] For example, if the operation type in the passed parameters is primary key judgment, the value returned by the database query API is the maximum and minimum values ​​of the primary keys of the table to be synchronized and the offline data warehouse table; or, if the operation type in the passed parameters is quantity judgment, the value returned by the database query API is the number of records that meet the where conditions in the table to be synchronized and the offline data warehouse table.

[0036] Step 102: According to the value returned by the database query API, a data synchronization test is performed on the table to be synchronized and the offline data warehouse table to obtain a data synchronization test result.

[0037] Optionally, according to the value returned by the database query API, by comparing whether the information in the table to be synchronized and the offline data warehouse table is consistent, a data synchronization test is performed on the table to be synchronized and the offline data warehouse table to obtain a data synchronization test result.

[0038] In an embodiment of the present application, data is synchronized to an offline data warehouse table by executing an offline data extraction scheduled task, and then data synchronization detection is performed through the return value of the database query API, thereby improving the accuracy of data synchronization detection and reducing the manual maintenance cost during the data synchronization process.

[0039] Optionally, the value returned by the database query API includes:

[0040] When the operation type in the parameter passed in when calling the database query API is quantity judgment, the value returned by the database query API is the number of records that meet the conditions in the table to be synchronized and the offline data warehouse table;

[0041] For example, when the database query API is called, the operation type in the parameter passed in is quantity judgment, and the where condition specifies the query condition. The value returned by the API is the number of records that meet the where condition in the table to be synchronized and the offline data warehouse table respectively.

[0042] When the operation type in the parameters passed in when calling the database query API is primary key judgment, the value returned by the database query API is the primary key value range of the table to be synchronized and the offline data warehouse table.

[0043] For example, if the operation type in the parameters passed in when calling the database query API is primary key judgment, the maximum and minimum values ​​of the primary keys in the table to be synchronized and the offline data warehouse table are queried to obtain the primary key value range of the table to be synchronized and the offline data warehouse table.

[0044] In an embodiment of the present application, different query operations are performed according to the operation type in the parameters passed in when calling the database query API, different return values ​​are obtained, and data synchronization detection is performed based on the return value, thereby improving the accuracy of data synchronization detection and reducing the difficulty of data synchronization detection.

[0045] Optionally, performing data synchronization detection on the table to be synchronized and the offline data warehouse table according to the value returned by the database query API to obtain a data synchronization detection result includes:

[0046] When the number of records that meet the conditions in the table to be synchronized is equal to the number of records that meet the conditions in the offline data warehouse table, and the primary key value range of the table to be synchronized is the same as the primary key value range of the offline data warehouse table, the data synchronization detection result is determined to be successful data synchronization.

[0047] Optionally, if the number of records that meet the conditions in the table to be synchronized is equal to the number of records that meet the conditions in the offline data warehouse table, and the primary key value range of the table to be synchronized is the same as the primary key value range of the offline data warehouse table, then the data in the table to be synchronized and the offline data warehouse table are considered to be consistent, and the data synchronization detection result is that the data synchronization is successful.

[0048] In the embodiment of the present application, a data synchronization detection result is obtained based on the number of qualified records in the table to be synchronized and the offline data warehouse table and the range of primary key values, thereby improving the accuracy of the data synchronization detection and reducing the difficulty of the data synchronization detection.

[0049] Optionally, before executing the offline data extraction scheduled task, the method further includes: updating the table structure of the offline data warehouse table. Figure 2 : is a flow chart of updating the table structure of an offline data warehouse table provided in an embodiment of the present application, such as Figure 2 As shown, updating the table structure of the offline data warehouse table includes the following sub-steps:

[0050] Step 200: Obtain meta information of the database management platform, and determine whether the table to be synchronized has undergone a structural change according to the meta information;

[0051] The meta information of the database management platform records each change operation of the database. Therefore, it is possible to determine whether the structure of the table to be synchronized has changed through the meta information of the database management platform.

[0052] Optionally, the following parameters need to be passed in to obtain the metadata of the database management platform: the table name of the table to be synchronized, the database IP and port number of the table to be synchronized, and the table name of the offline data warehouse table. Based on the passed parameters, the metadata of the database management platform is obtained, and based on the metadata, it is determined whether the structure of the table to be synchronized has changed.

[0053] For example, a new field is added to the table to be synchronized. At this time, it can be determined according to the meta information that the structure of the table to be synchronized has been changed.

[0054] Step 201: When the structure of the table to be synchronized is changed, generate a table structure change structured query language SQL according to the meta information, execute the table structure change SQL in the offline data warehouse table, and update the table structure of the offline data warehouse table.

[0055] Optionally, if it is determined based on the metadata of the database management platform that a structural change has occurred in the table to be synchronized, a table structure change SQL is generated based on the metadata, and the table structure change SQL records the structural change content of the table to be synchronized. The table structure change SQL is executed in the offline data warehouse table to update the table structure of the offline data warehouse table. After the update, the table structure of the offline data warehouse table and the table structure of the table to be synchronized remain synchronized.

[0056] In an embodiment of the present application, before executing an offline data extraction scheduled task, the table structure of the offline data warehouse table is updated. When the table structure of the table to be synchronized changes, the table structure of the offline data warehouse table is automatically updated, thereby avoiding the defect of untimely update caused by manual adjustment of the offline data warehouse table structure. At the same time, the table structure of the offline data warehouse table is updated so that the table structure of the offline data warehouse table and the table structure of the table to be synchronized are consistent, thereby improving the success rate of executing offline data extraction scheduled tasks for data synchronization.

[0057] Optionally, before executing the offline data extraction scheduled task, the method further includes:

[0058] Determine whether the offline data warehouse table exists.

[0059] Optionally, the data synchronization task script determines whether there is an offline data warehouse table corresponding to the table to be synchronized.

[0060] If the offline data warehouse table does not exist, create the offline data warehouse table.

[0061] It is understandable that if the offline data warehouse table does not exist, data synchronization cannot be performed and the offline data warehouse table needs to be created.

[0062] In an embodiment of the present application, before executing the offline data extraction scheduled task, it is determined whether the offline data warehouse table exists. If the offline data warehouse table does not exist, the offline data warehouse table is created, thereby avoiding data synchronization failure due to the non-existence of the offline data warehouse table and improving the success rate of data synchronization.

[0063] Figure 3 is a schematic diagram of a process for creating the offline data warehouse table provided in an embodiment of the present application, such as Figure 3 As shown, optionally, the creating of the offline data warehouse table includes the following sub-steps:

[0064] Step 300: Acquire the field information, table attribute information and the first table creation SQL of the table to be synchronized by calling the database query API, wherein the operation type in the parameters passed in when calling the database query API is table creation.

[0065] Optionally, when calling the database query API, the parameters that need to be passed in include: operation type, table name of the table to be synchronized, database type of the table to be synchronized, database IP and port number of the table to be synchronized, table name and where condition of the offline data warehouse table; among which, the operation type in the parameters passed in when calling the database query API is table creation.

[0066] The database query API automatically obtains the table creation SQL of the table to be synchronized, that is, the first table creation SQL, as well as the field information and table attribute information of the table to be synchronized according to the input parameters.

[0067] Step 301: Generate a second table creation SQL of the offline data warehouse table according to the information of the offline data warehouse table, the field information of the table to be synchronized, the table attribute information and the first table creation SQL.

[0068] According to the information of the offline data warehouse table returned by the database query API, the field information of the table to be synchronized, the table attribute information and the first table creation SQL, a second table creation SQL is automatically generated to create the offline data warehouse table.

[0069] Step 302: Execute the second table creation SQL of the offline data warehouse table to create the offline data warehouse table.

[0070] Optionally, the second table creation SQL is a table creation SQL for an offline data warehouse table, and the offline data warehouse table is created by executing the second table creation SQL.

[0071] In the embodiment of the present application, by calling the database query API, the second table creation SQL for creating the offline data warehouse table is automatically generated according to the first table creation SQL for creating the table to be synchronized, thereby improving the efficiency and accuracy of creating the offline data warehouse table and further improving the accuracy of data synchronization.

[0072] Optionally, the data synchronization detection method further includes:

[0073] When the data synchronization detection result is that the data synchronization is unsuccessful, an alarm service is called to output an alarm prompt of data synchronization failure.

[0074] Optionally, if data synchronization failure is detected, the alarm service is called to output an alarm prompt of data synchronization failure for operation and maintenance personnel to view and refer to.

[0075] Optionally, the alarm prompt is location and quantity information of inconsistent data between the table to be synchronized and the offline data warehouse table.

[0076] In the embodiment of the present application, when the data synchronization detection result is that the data synchronization is unsuccessful, an alarm prompt of the data synchronization failure is output in time, thereby reducing the maintenance cost during the data synchronization process.

[0077] It should be noted that the data synchronization detection method provided in the embodiment of the present application can be executed by a data synchronization detection device, or a control module in the data synchronization detection device for executing the data synchronization detection method. In the embodiment of the present application, the data synchronization detection device provided in the embodiment of the present application is described by taking the data synchronization detection device executing the data synchronization detection method as an example.

[0078] Figure 4 A data synchronization detection device provided in an embodiment of the present application, such as Figure 4 As shown, the device includes a timing extraction module 410, a data query module 420 and a synchronization detection module 430, wherein:

[0079] A timed extraction module 410, used to execute an offline data extraction timed task, wherein the offline data extraction timed task is used to synchronize the data in the table to be synchronized to the offline data warehouse table at a preset time interval;

[0080] The data query module 420 is used to call the database query application program interface API to obtain the value returned by the database query API;

[0081] The synchronization detection module 430 is used to perform data synchronization detection on the table to be synchronized and the offline data warehouse table according to the value returned by the database query API to obtain a data synchronization detection result.

[0082] In an embodiment of the present application, data is synchronized to an offline data warehouse table by executing an offline data extraction scheduled task, and then data synchronization detection is performed through the return value of the database query API, thereby improving the accuracy of data synchronization detection and reducing the manual maintenance cost during the data synchronization process.

[0083] Optionally, the value returned by the database query API includes:

[0084] When the operation type in the parameter passed in when calling the database query API is quantity judgment, the value returned by the database query API is the number of records that meet the conditions in the table to be synchronized and the offline data warehouse table;

[0085] When the operation type in the parameters passed in when calling the database query API is primary key judgment, the value returned by the database query API is the primary key value range of the table to be synchronized and the offline data warehouse table.

[0086] Optionally, the synchronization detection module is used to:

[0087] When the number of records that meet the conditions in the table to be synchronized is equal to the number of records that meet the conditions in the offline data warehouse table, and the primary key value range of the table to be synchronized is the same as the primary key value range of the offline data warehouse table, the data synchronization detection result is determined to be successful data synchronization.

[0088] Optionally, a table structure updating module is further included, which is used before executing the offline data extraction scheduled task:

[0089] Obtaining meta information of the database management platform, and determining whether the table to be synchronized has undergone a structural change according to the meta information;

[0090] When the structure of the table to be synchronized is changed, a table structure change structured query language SQL is generated according to the meta information, and the table structure change SQL is executed in the offline data warehouse table to update the table structure of the offline data warehouse table.

[0091] Optionally, a warehouse table creation module is also included, which is used to:

[0092] Determine whether the offline data warehouse table exists;

[0093] If the offline data warehouse table does not exist, create the offline data warehouse table.

[0094] Optionally, the creating the offline data warehouse table includes:

[0095] By calling the database query API, the field information, table attribute information and the first table creation SQL of the table to be synchronized are obtained, wherein the operation type in the parameter passed in when calling the database query API is table creation;

[0096] Generate a second table creation SQL of the offline data warehouse table according to the information of the offline data warehouse table, the field information of the table to be synchronized, the table attribute information and the first table creation SQL;

[0097] Execute the second table creation SQL of the offline data warehouse table to create the offline data warehouse table.

[0098] Optionally, an alarm prompt module is also included, which is used to:

[0099] When the data synchronization detection result is that the data synchronization is unsuccessful, an alarm service is called to output an alarm prompt of data synchronization failure.

[0100] The data synchronization detection device in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. The non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0101] The data synchronization detection device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0102] The data synchronization detection device provided in the embodiment of the present application can achieve Figures 1 to 3 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0103] Alternatively, if Figure 5As shown, an embodiment of the present application also provides an electronic device 500, including a processor 501, a memory 502, and a program or instruction stored in the memory 502 and executable on the processor 501. When the program or instruction is executed by the processor 501, each process of the above-mentioned data synchronization detection method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0104] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0105] Figure 6 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0106] The electronic device 600 includes but is not limited to: a radio frequency unit 601, a network module 602, an audio output unit 603, an input unit 604, a sensor 605, a display unit 606, a user input unit 607, an interface unit 608, a memory 609, and a processor 610.

[0107] Those skilled in the art will appreciate that the electronic device 600 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 610 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 6 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0108] The processor 610 is used for:

[0109] Execute an offline data extraction scheduled task, wherein the offline data extraction scheduled task is used to synchronize the data in the table to be synchronized to the offline data warehouse table at a preset time interval;

[0110] Calling a database query application program interface (API) to obtain a value returned by the database query API;

[0111] According to the value returned by the database query API, a data synchronization test is performed on the table to be synchronized and the offline data warehouse table to obtain a data synchronization test result.

[0112] In an embodiment of the present application, data is synchronized to an offline data warehouse table by executing an offline data extraction scheduled task, and then data synchronization detection is performed through the return value of the database query API, thereby improving the accuracy of data extraction detection and reducing the manual maintenance cost during the data synchronization process.

[0113] Optionally, the value returned by the database query API includes:

[0114] When the operation type in the parameter passed in when calling the database query API is quantity judgment, the value returned by the database query API is the number of records that meet the conditions in the table to be synchronized and the offline data warehouse table;

[0115] When the operation type in the parameters passed in when calling the database query API is primary key judgment, the value returned by the database query API is the primary key value range of the table to be synchronized and the offline data warehouse table.

[0116] Optionally, performing data synchronization detection on the table to be synchronized and the offline data warehouse table according to the value returned by the database query API to obtain a data synchronization detection result includes:

[0117] When the number of records that meet the conditions in the table to be synchronized is equal to the number of records that meet the conditions in the offline data warehouse table, and the primary key value range of the table to be synchronized is the same as the primary key value range of the offline data warehouse table, the data synchronization detection result is determined to be successful data synchronization.

[0118] Optionally, the processor 610 is further configured to:

[0119] Obtaining meta information of the database management platform, and determining whether the table to be synchronized has undergone a structural change according to the meta information;

[0120] When the structure of the table to be synchronized is changed, a table structure change structured query language SQL is generated according to the meta information, and the table structure change SQL is executed in the offline data warehouse table to update the table structure of the offline data warehouse table.

[0121] Optionally, the processor 610 is further configured to:

[0122] Determine whether the offline data warehouse table exists;

[0123] If the offline data warehouse table does not exist, create the offline data warehouse table.

[0124] Optionally, the creating the offline data warehouse table includes:

[0125] By calling the database query API, the field information, table attribute information and the first table creation SQL of the table to be synchronized are obtained, wherein the operation type in the parameter passed in when calling the database query API is table creation;

[0126] Generate a second table creation SQL of the offline data warehouse table according to the information of the offline data warehouse table, the field information of the table to be synchronized, the table attribute information and the first table creation SQL;

[0127] Execute the second table creation SQL of the offline data warehouse table to create the offline data warehouse table.

[0128] Optionally, the processor 610 is further configured to:

[0129] When the data synchronization detection result is that the data synchronization is unsuccessful, an alarm service is called to output an alarm prompt of data synchronization failure.

[0130] It should be understood that in the embodiment of the present application, the input unit 604 may include a graphics processor (GPU) 641 and a microphone 642, and the graphics processor 641 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 606 may include a display panel 661, and the display panel 661 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 607 includes a touch panel 671 and other input devices 672. The touch panel 671 is also called a touch screen. The touch panel 671 may include two parts: a touch detection device and a touch controller. Other input devices 672 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here. The memory 609 may be used to store software programs and various data, including but not limited to applications and operating systems. The processor 610 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and applications, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 610.

[0131] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned data synchronization detection method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0132] The processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0133] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned data synchronization detection method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0134] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0135] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0136] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0137] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A data synchronization detection method, characterized in that: include: Execute an offline data extraction scheduled task, wherein the offline data extraction scheduled task is used to synchronize the data in the table to be synchronized to the offline data warehouse table according to the first parameter input at a preset time interval; Calling a database query application program interface API, and obtaining a value corresponding to the second parameter returned by the database query API according to a second parameter passed in, wherein the second parameter includes an operation type and the first parameter, and the operation type indicates a query operation type of the database query API, and the operation type includes at least one of primary key determination and quantity determination of records meeting the condition; According to the value corresponding to the second parameter returned by the database query API, data synchronization detection is performed on the table to be synchronized and the offline data warehouse table to obtain a data synchronization detection result.

2. The data synchronization detection method according to claim 1, characterized in that: The values ​​returned by the database query API include: When the operation type in the parameter passed in when calling the database query API is quantity judgment, the value returned by the database query API is the number of records that meet the conditions in the table to be synchronized and the offline data warehouse table; When the operation type in the parameters passed in when calling the database query API is primary key judgment, the value returned by the database query API is the primary key value range of the table to be synchronized and the offline data warehouse table.

3. The data synchronization detection method according to claim 1, characterized in that: Before executing the offline data extraction scheduled task, the method further includes: Obtaining meta information of the database management platform, and determining whether the table to be synchronized has undergone a structural change according to the meta information; When the structure of the table to be synchronized is changed, a table structure change structured query language SQL is generated according to the meta information, and the table structure change SQL is executed in the offline data warehouse table to update the table structure of the offline data warehouse table.

4. The data synchronization detection method according to claim 1, characterized in that: Before executing the offline data extraction scheduled task, the method further includes: Determine whether the offline data warehouse table exists; If the offline data warehouse table does not exist, create the offline data warehouse table.

5. The data synchronization detection method according to claim 4, characterized in that: The creating of the offline data warehouse table includes: By calling the database query API, the field information, table attribute information and the first table creation SQL of the table to be synchronized are obtained, wherein the operation type in the parameter passed in when calling the database query API is table creation; Generate a second table creation SQL of the offline data warehouse table according to the information of the offline data warehouse table, the field information of the table to be synchronized, the table attribute information and the first table creation SQL; Execute the second table creation SQL of the offline data warehouse table to create the offline data warehouse table.

6. A data synchronization detection device, characterized in that: include: A timing extraction module, used to execute an offline data extraction timing task, wherein the offline data extraction timing task is used to synchronize the data in the table to be synchronized to the offline data warehouse table according to the first parameter input at a preset time interval; a data query module, configured to call a database query application program interface API, and obtain a value corresponding to the second parameter returned by the database query API according to a second parameter passed in, wherein the second parameter includes an operation type and the first parameter, wherein the operation type indicates a query operation type of the database query API, and the operation type includes at least one of primary key determination and quantity determination of records meeting a condition; The synchronization detection module is used to perform data synchronization detection on the table to be synchronized and the offline data warehouse table according to the value corresponding to the second parameter returned by the database query API to obtain a data synchronization detection result.

7. The data synchronization detection device according to claim 6, characterized in that: The values ​​returned by the database query API include: When the operation type in the parameter passed in when calling the database query API is quantity judgment, the value returned by the database query API is the number of records that meet the conditions in the table to be synchronized and the offline data warehouse table; When the operation type in the parameters passed in when calling the database query API is primary key judgment, the value returned by the database query API is the primary key value range of the table to be synchronized and the offline data warehouse table.

8. The data synchronization detection device according to claim 6, characterized in that: Also includes table structure update module for: Obtaining meta information of the database management platform, and determining whether the table to be synchronized has undergone a structural change according to the meta information; When the structure of the table to be synchronized is changed, a table structure change structured query language SQL is generated according to the meta information, and the table structure change SQL is executed in the offline data warehouse table to update the table structure of the offline data warehouse table.

9. The data synchronization detection device according to claim 6, characterized in that: Also includes warehouse table creation modules for: Determine whether the offline data warehouse table exists; If the offline data warehouse table does not exist, create the offline data warehouse table.

10. The data synchronization detection device according to claim 9, characterized in that: The creating of the offline data warehouse table includes: By calling the database query API, the field information, table attribute information and the first table creation SQL of the table to be synchronized are obtained, wherein the operation type in the parameter passed in when calling the database query API is table creation; Generate a second table creation SQL of the offline data warehouse table according to the information of the offline data warehouse table, the field information of the table to be synchronized, the table attribute information and the first table creation SQL; Execute the second table creation SQL of the offline data warehouse table to create the offline data warehouse table.

Images