Data synchronization method, device, equipment and medium

Abstract

The present application discloses a data synchronization method, apparatus, device and medium, wherein the method includes: receiving a first query request sent by a synchronization client, the first query request is used to query target data for data synchronization, the first query request includes a data screening condition and a query method, the data screening condition is used to screen the target data to be synchronized, the data screening condition includes a composite index, and the composite index includes a time field and a serial number field; according to the first query request, querying the target data for data synchronization from the database; returning the target data to the synchronization client, so that the synchronization client writes the target data into the database of the synchronization client to realize data synchronization, thereby realizing stable and fast data synchronization.

Description

Technical Field

[0001] The present disclosure generally relates to the field of database technology, specifically to the field of data synchronization technology, and more particularly to a data synchronization method, apparatus, device, and medium. Background Art

[0002] With the development of technologies such as the Internet of Things, big data, and artificial intelligence, data is growing explosively. Stable and rapid data synchronization is an urgent issue in this field. Summary of the Invention

[0003] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide a data synchronization method, apparatus, device and medium that can achieve stable and fast data synchronization.

[0004] In a first aspect, an embodiment of the present application provides a data synchronization method, including:

[0005] Receive a first query request sent by a synchronization client, the first query request is used to query target data for data synchronization, the first query request includes a data filtering condition and a query method, the data filtering condition is used to filter the target data to be synchronized, the data filtering condition includes a composite index, and the composite index includes a time field and a sequence number field;

[0006] querying the target data for data synchronization from a database according to the first query request;

[0007] The target data is returned to the synchronization client, so that the synchronization client writes the target data into the database of the synchronization client to realize data synchronization.

[0008] In a second aspect, an embodiment of the present application provides a data synchronization device, including:

[0009] a receiving module, configured to receive a first query request sent by a synchronization client, the first query request being used to query target data for data synchronization, the first query request including a data screening condition and a query mode, the data screening condition being used to screen the target data to be synchronized, the data screening condition including a composite index including a time field and a sequence number field;

[0010] a screening module, configured to query the target data for data synchronization from a database according to the first query request;

[0011] The feedback module is configured to return the target data to the synchronization client, so that the synchronization client writes the target data into a database of the synchronization client to achieve data synchronization.

[0012] In some embodiments, the data filtering condition includes a first compound index and a second compound index connected by an OR operation, the first compound index includes the time field, which is used to filter data generated after the generation time of the last data synchronized last time, and the second compound index includes the time field and the serial number field, which is used to filter data generated at the same time as the last data synchronized last time but sorted after the last data synchronized last time.

[0013] In some embodiments, the data screening condition may further include a classification sub-condition, and the classification sub-condition is used to perform secondary screening of candidate data. When the data screening condition does not include a classification sub-condition or the data screening condition includes a classification sub-condition and the time for querying the target data is less than or equal to a preset time, the query method adopts direct query; the screening module is further used to:

[0014] Sending the data screening condition to the database, the database is used to query according to the composite index in the data screening condition to obtain the candidate data, and to perform a secondary screening on the candidate data according to the classification sub-condition to obtain the target data that meets the classification sub-condition;

[0015] The target data filtered out by the database according to the data filtering condition is received.

[0016] In some embodiments, the data screening condition may further include a classification sub-condition, and the classification sub-condition is used to perform secondary screening of the candidate data. When the data screening condition includes the classification sub-condition and the time for querying the target data is greater than a preset time, the query method is a proxy query, and the screening module is further used to:

[0017] Splitting the data screening condition into the composite index and the classification sub-condition;

[0018] sending the composite index to the database;

[0019] receiving candidate data screened by the database according to the composite index;

[0020] The candidate data are classified and screened according to the classification sub-conditions to obtain the target data that meets the classification sub-conditions.

[0021] In some embodiments, the composite index further includes a read data volume. When the data volume of the candidate data is greater than the read data volume, the screening module is further configured to:

[0022] Update the composite index according to the time field and sequence number field corresponding to the last piece of candidate data currently received;

[0023] The updated composite index is sent to the database.

[0024] In some embodiments, the screening module is further configured to:

[0025] Increasing the time field corresponding to the last piece of candidate data by 1 time granularity to obtain an updated time field;

[0026] Constructing a third composite index using the updated time field, and constructing a fourth composite index using an AND operation between the time field corresponding to the last piece of candidate data and the sequence number field corresponding to the last piece of candidate data;

[0027] An OR operation is performed on the third composite index and the fourth composite index to obtain the updated composite index.

[0028] In some embodiments, the feedback module is further configured to:

[0029] receiving a second query request, where the second query request is used to query a screening status of the target data;

[0030] When a preset condition is met, the target data is returned to the synchronization client in response to the second query request, wherein the preset condition includes at least one of filtering out the target data and completing the screening of the candidate data.

[0031] In some embodiments, the data synchronization device further includes: a standardization module.

[0032] The standardization module is further used to: perform standardization processing on the data in the database according to the modification time and modification order of the data in the database, and obtain a time field label and a sequence number field label corresponding to each data in the database.

[0033] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the embodiment of the present application when executing the program.

[0034] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, which, when executed by a processor, implements the method described in the embodiment of the present application.

[0035] The present application receives a first query request including data screening conditions and query methods sent by a synchronization client, and uses a composite index consisting of a time field and a serial number field in the data screening conditions to screen target data, so that the screened target data can include both data that meets the time field requirements and data that meets the serial number requirements, and then returns the target data to the synchronization client, so that the synchronization client writes the target data into the database of the synchronization client to realize data synchronization, effectively avoiding the problem of missing data or repeated synchronization during the data synchronization process, and greatly improving the reliability of the online data synchronization process.

[0036] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0038] Figure 1 A schematic diagram of a scenario of a data synchronization method provided by an embodiment of the present invention;

[0039] Figure 2 A flowchart of a data synchronization method proposed in an embodiment of the present application;

[0040] Figure 3 A flowchart of another data synchronization method proposed in an embodiment of the present application;

[0041] Figure 4 A flowchart of another data synchronization method proposed in an embodiment of the present application;

[0042] Figure 5 A flowchart of another data synchronization method proposed in an embodiment of the present application;

[0043] Figure 6 A flowchart of another data synchronization method proposed in an embodiment of the present application;

[0044] Figure 7 A flowchart of another data synchronization method proposed in an embodiment of the present application;

[0045] Figure 8 A block diagram of a data synchronization device proposed in an embodiment of the present application;

[0046] Figure 9 A block diagram of another data synchronization device proposed in an embodiment of the present application;

[0047] Figure 10It is a structural diagram of a computer system of an electronic device or server suitable for implementing the embodiments of the present application. DETAILED DESCRIPTION

[0048] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the invention are shown in the accompanying drawings.

[0049] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0050] All Internet applications have the need for data synchronization, especially when capabilities are integrated and big data services are jointly built.

[0051] For example, the school system contains information about all teachers, students, classes, teaching, and exams in the school. Data visualization developers need this data to develop large-screen applications. Due to network permissions and data privacy issues, they cannot directly use the school database and need to synchronize a copy of the data from the school for visualization.

[0052] Since some data is constantly changing, such as students' test scores, attendance and dismissal status, etc., online incremental synchronization of the changing data is required.

[0053] However, in the related art, synchronization methods such as paging synchronization and offline synchronization are usually used. Among them, paging synchronization is to pull the data table with the specified offset and size. However, as the offset increases, the time of each query will increase linearly, and the overhead on the database will also increase linearly. In offline synchronization, the data table is exported as a csv or json file, and then transferred using a physical carrier such as a USB flash drive / using an FTP server / on-cloud object storage / instant messaging tools such as WeChat, and then the data is manually imported at the destination. In most scenarios where offline synchronization has to be used, manual operation is required, which not only has large delays and a large workload, but also, when the amount of data is large, the exported file is also large, and a large file is difficult to handle in any computer system. In addition, large files stored in the cloud will incur a lot of fees. Over time, a large number of fragmented files will accumulate, which will be difficult to manage later. The efficiency is limited by the disk read and write performance, and the synchronization efficiency is difficult to improve.

[0054] Based on this, embodiments of the present application provide a data synchronization method, apparatus, device, and medium, wherein the data synchronization apparatus can be specifically integrated into a database management system.

[0055] See also Figure 1 , which is a schematic diagram of a scenario for the data synchronization method provided in an embodiment of the present invention. This scenario may include a synchronization client 1, a database management system 2, and a database 3. The synchronization client 1 is the target terminal, primarily used to provide data synchronization requirements and receive target data to be synchronized. The database management system 2 is used to standardize the data in the database and provide proxy services for massive data synchronization and / or sparse data synchronization. Specifically, a proxy module is established in the database management system 2 to replace the synchronization client in interacting with the database 3. Accessing the database 3 via the intranet reduces data latency and ensures that data information can be returned within a reasonable time.

[0056] The database management system 2 can be installed in a server, which can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The synchronization client 1 can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, in-vehicle terminal, smart TV, etc., but is not limited to these. The synchronization client 1 and the database management system 2 can be connected directly or indirectly via wired or wireless communication, which is not limited in this application.

[0057] Optionally, as disclosed in the data synchronization method of this application, these data can be stored on the blockchain.

[0058] In order to describe this application more clearly, the following is an explanation of relevant technical terms:

[0059] A database, in short, can be thought of as a digital filing cabinet—a place where electronic files are stored, allowing users to add, query, update, and delete data. A database is a collection of data stored in a specific way, shared by multiple users, with minimal redundancy, and independent of applications.

[0060] A database management system (DBMS) is a computer software system designed for managing databases, typically providing basic functions such as storage, retrieval, security, and backup. DBMSs can be categorized by the database model they support, such as relational or XML (Extensible Markup Language); by the type of computer they support, such as server clusters or mobile phones; by the query language they use, such as SQL (Structured Query Language) or XQuery; by performance priorities, such as maximum scale or maximum speed; or by other classification methods. Regardless of the classification method used, some DBMSs can cross categories, for example, supporting multiple query languages ​​simultaneously.

[0061] Data synchronization: Migrate data from one database to another to make the data in the two databases consistent.

[0062] Offline synchronization: In contrast to online synchronization, this approach typically involves exporting source data into files, transferring the files over the network, and then importing them on the destination end. This approach typically takes a long time and synchronization tasks are slow to complete.

[0063] Online synchronization: Usually, the DB (datebase, database file) of both parties is online (accessible through network calls), and the data synchronization module directly reads the data source and writes it to the destination.

[0064] Full synchronization: Data synchronization reads all data and writes it to the destination each time.

[0065] Incremental synchronization: When synchronizing data, a full synchronization is performed only for the first time. Each subsequent synchronization only reads and writes the modified or newly added content to the destination.

[0066] Massive data: extremely large amounts of data. A data table may contain hundreds of millions of data items.

[0067] Supports filtering conditions: you can synchronize only data that meets the specified conditions.

[0068] Figure 2 This is a flow chart of a data synchronization method proposed in an embodiment of the present application. Figure 2 As shown, the data synchronization method includes the following steps:

[0069] Step 101: Receive a first query request sent by a synchronization client. The first query request is used to query target data for data synchronization. The first query request includes data filtering conditions and a query method. The data filtering conditions are used to filter target data that needs to be synchronized. The data filtering conditions include a composite index. The composite index includes a time field and a serial number field.

[0070] The data screening conditions may include at least one composite index. If the composite index only includes a time field, that is, the index matches all sequence number fields corresponding to the included time field. If the composite index only includes a sequence number field, that is, the index matches data with the sequence number generated at any time. A composite index may also include both a time field and a sequence number field, that is, the index matches data that matches both the included time field and the sequence number field. It should be understood that when a composite index only includes either a time field or a sequence number field, it can also be understood as an abbreviation for the other field.

[0071] It should be noted that, before receiving the first query request in step 101, the process also includes normalizing the data in the database based on the modification time and modification order of the data in the database, thereby obtaining a time field label and a sequence number field label corresponding to each data item in the database. Adding, modifying, and deleting data are all considered data modification operations. Specifically, data normalization is performed based on the time and order in which the data is added, modified, or deleted.

[0072] It should be understood that the purpose of standardizing the data in the database in this application is to ensure that each data item contains a tag for the time field mtime and a tag for the sequence number field id, specifically in accordance with the following rules:

[0073]

[0074] Specifically, for the data already stored in the database, a time field label and a sequence number field label corresponding to each data are generated according to the time node when the data is stored and the order in which it is stored.

[0075] Among them, this application also provides a standard "add, delete, modify and query" interface for data. The external party accesses the data through the data synchronization device proposed in this application, so that the id and mtime can maintain their logic throughout the life cycle of the data service, that is, the id and mtime corresponding to the data that is "added, deleted, modified and query" can also meet the standardization rules. In other words, the data that is added and modified will generate corresponding time field labels and serial number field labels when it is added or modified.

[0076] This application is applicable to general-purpose databases such as MySQL, MongoDB, and PostgreSQL. Furthermore, some databases can achieve standardization through built-in configuration commands. For example, MySQL can automatically conform to this standard by setting mtime to [ON UPDATE CURRENT_TIMESTAMP] using the auto-incrementing primary key ID. In other words, general-purpose databases typically have standardization capabilities, and using the database's own standardization commands, the corresponding time and sequence number fields can be automatically generated for the data in the database.

[0077] It should be understood that since data synchronization is an indispensable part of the Internet field, and considering its universality, it is impossible to create an index specifically for each synchronization task. Therefore, this application designs a composite index that combines time and serial number. By managing these two fields, online incremental synchronization of massive data can be completed with high performance and high reliability.

[0078] Step 102: Query target data for data synchronization from a database according to a first query request.

[0079] Step 103: Return the target data to the synchronization client, so that the synchronization client writes the target data into the database of the synchronization client to achieve data synchronization.

[0080] That is to say, after using the data filtering conditions containing compound indexes to filter out the target data for data synchronization from the database, the database management system returns the target data to the synchronization client. The synchronization client receives the target data returned by the database management system and writes the target data into its own database, thereby realizing data synchronization between the synchronization client and the database.

[0081] The synchronization client may store each data in the target data in a corresponding storage location in the database of the synchronization client according to its own preset data storage rules.

[0082] Specifically, the present application receives a first query request including data screening conditions and query methods sent by a synchronization client, and uses a composite index consisting of a time field and a serial number field in the data screening conditions to screen the target data, so that the screened target data can include both data that meets the time field requirements and data that meets the serial number requirements, and then returns the target data to the synchronization client, so that the synchronization client writes the target data into the database of the synchronization client to achieve data synchronization, effectively avoiding the problem of missing data or repeated synchronization during the data synchronization process, and greatly improving the reliability of the online data synchronization process.

[0083] Furthermore, the data filtering condition includes a first compound index and a second compound index connected by an OR operation, the first compound index includes a time field, which is used to filter data generated after the time when the last data synchronized last was generated, and the second compound index includes a time field and a sequence number field, which is used to filter data generated at the same time as the last data synchronized last but sorted after the last data synchronized last.

[0084] The generation time includes the time when the data is created and / or updated. Optionally, the time when the data is generated is recorded in seconds.

[0085] As a feasible embodiment, the time field and the sequence number field can be managed by the following rules to form data screening conditions:

[0086]

[0087]

[0088] Among them, for general online incremental synchronization, two general filtering conditions are ORed, and other conditions (sorting conditions, specified index, read quantity) can be spliced ​​normally in the query request, thus forming a general online incremental synchronization composite index. When using it, only the specific time field and serial number field need to be continuously updated to achieve continuous online incremental synchronization.

[0089] For example, according to the above rules, the first query request written in SQL can be: select * from t_student force index (mtime_id) where (mtime>1614592239) or (mtime=1614592239andid>'53102b43bf1044ed8b0ba36b') order by mtime ASC, id ASC limit 100; the first query request written in MongoDB can be: db.student.find({$or:[{mtime:{$gt:1614592239}},{mtime:1614592239,_id:{$gt:'53102b43bf1044ed8b0ba36b'}}]}).sort({mtime:1,_id:1}).limit(100).hint({mtime:1,_id:1}).

[0090] Specifically, it can be expressed as: query the data created or modified after the time 1614592239 from the student data table, or the data created or rewritten at the time 1614592239 but sorted after '53102b43bf1044ed8b0ba36b'. The queried data is first sorted by the update timestamp mtime, and then sorted by the serial number id, and 100 data results are returned.

[0091] For example, an index that includes both the time field and the sequence number field can be visualized as follows: all data is first sorted by the time field mtime. For data with the same time field mtime, it is sorted by the sequence number id to obtain an array, as shown in the following table:

[0092]

[0093] Among them, the "or" condition is set to connect the first composite index and the second composite index so that all first composite indexes and second composite indexes can hit the index, so the query efficiency has reached the optimal level. Take mtime=2, id=e (that is, the last data was synchronized to Zhao Liu's location) as an example.

[0094] For the first composite index: mtime>2, the search starts from the position with the update timestamp 3, and the two pieces of data, Feng Shi and Xiao Shiyi, are obtained.

[0095] For the second composite index: mtime=2and id>'e', the scan starts from the first position with an update timestamp of 2 and a sequence number after e, and the data from Monday to Saturday is obtained.

[0096] It can be seen that using the composite index designed in this application, it is possible to directly query the data after the previous query by hitting the index. That is, the next query can synchronize data from the position where mtime=3, id=b, effectively avoiding data omissions while ensuring the performance of each data synchronization. For example, using the composite index proposed in this application, because the index is hit by the time field, it is possible to synchronize data that has already been synchronized but has been updated again, thus avoiding data omissions.

[0097] Moreover, because the index is hit, there is no need to reorder the target data, the search can be directly reached in 0(logN), and all scanned indexes are valid. Therefore, no matter how large the data to be synchronized is, the response time for each data synchronization can be fully guaranteed.

[0098] Optionally, to avoid server time errors, the latest synchronized time can be limited to a period of time before the current time, for example, 3 seconds, to ensure that data is not missed due to time errors. In other words, when performing real-time data synchronization, data generated 3 seconds before and after the current time can be synchronized. For example, if the current time is 5 seconds, the first time field can be mtime=2, and the second time field can be mtime=1.

[0099] It should be noted that data synchronization also includes conditional data synchronization, for example, only synchronizing target data that meets certain conditions.

[0100] For example, add the data type attribute. Type is a binary classification of data, that is, the data type can be 0 or 1. The amount of data to be read is 3. The data list is as follows:

[0101]

[0102] That is to say, in order to complete the query task, the database usually needs to scan the entire data table. When the numbers n and m are small, for example, when m, n≤100, the time taken by the database to scan the entire data table and filter out the target data can be within the set return time (for example, when the set return time is 5 seconds), the database can complete the data table scan within 5 seconds and successfully complete the query requirements of the first query request.

[0103] However, as n and m increase, the database's scanning time for the entire data table gradually increases. The time it takes for the synchronization client to receive the target data returned by the database management system or the database is extended as n and m increase, and may even exceed the preset time (5 seconds), thereby causing the synchronization client to freeze. At this time, a proxy query can be used to maintain communication with the synchronization client through the proxy module of the data synchronization device to prevent the synchronization client from freezing.

[0104] Optionally, before selecting a proxy query, the read quantity per batch set in the first query request can be rewritten to reduce the time the database spends scanning the database table. For example, under normal circumstances, the read quantity per batch can be set to 1,000. When performing conditional screening, the read quantity per batch can be set to 100 or lower. This reduces the time it takes for the database to scan and read the target data each time, ensuring that the target data can be returned within the set time and preventing pseudo-dead events caused by return timeouts. The reduction in the read quantity per batch can be adjusted linearly, but the read quantity per batch should be greater than or equal to 1 and less than or equal to 500.

[0105] If the read volume of each batch has been reduced to 1, the target data still cannot be returned within the set time. For example, n is 1 million and m is 100 million. At this time, the database needs to scan hundreds of millions of data, which will obviously exceed the set time (5 seconds). That is, in the current situation, a pseudo-death phenomenon caused by the return time timeout will inevitably occur. At this time, proxy query can be used.

[0106] It should be understood that the filtering conditions can be sent to the database using the synchronization client first to check whether the database data return time has timed out. If it still times out when the read quantity drops to 1, the query mode is set to proxy query in the first query request.

[0107] That is to say, the data screening conditions may also include classification sub-conditions, which are used to perform secondary screening of candidate data. When the data screening conditions do not include classification sub-conditions or the data screening conditions include classification sub-conditions and the time for querying the target data is less than or equal to the preset time, the query method adopts direct query.

[0108] like Figure 3 As shown, when the query mode can be a direct query, querying the target data for data synchronization from the database according to the first query request includes:

[0109] In step 1021, the data synchronization device sends the data screening conditions to the database, which is used to query according to the composite index in the data screening conditions to obtain candidate data, and to perform secondary screening on the candidate data according to the classification sub-conditions to obtain target data of the composite classification sub-conditions.

[0110] Step 1022: The data synchronization device receives target data filtered out by the database according to the data filtering condition.

[0111] That is to say, the data synchronization device receives a first query request sent by the target terminal, and determines to use a direct query method to query the data screening conditions according to the query method in the first query request. At this time, the data synchronization device sends the data screening conditions in the first query request to the database. When the data screening conditions do not include classification conditions, the database performs data queries according to the first composite index and the second composite index in the data screening conditions, respectively, to obtain a first target data set corresponding to the first composite index and a second target data set corresponding to the second composite index. Then, the data in the first target data set and the second target data set are first sorted according to the time field, and then the multiple data corresponding to each time field are sorted according to the serial number field to obtain candidate data. From the candidate data, data of a number corresponding to the read quantity is selected and fed back to the data synchronization device as the final target data. Then, the data synchronization device returns the target data to the synchronization client, thereby achieving the purpose of synchronizing the data in the database to the synchronization client.

[0112] Alternatively, when the data screening conditions include classification sub-conditions and the query time is less than or equal to the preset time, the database first performs a data query based on the first composite index and the second composite index in the data screening conditions, respectively, to obtain a first target data set corresponding to the first composite index and a second target data set corresponding to the second composite index. The first target data set and the second target data set are then subjected to a secondary screening based on the classification sub-conditions to obtain the final target data. The target data is then sorted by the time field and then the sequence number field, and the sorting results are fed back to the synchronization client. Alternatively, the first target data set and the second target data set may be sorted to obtain candidate data, and then the sorting results may be subjected to a secondary screening. This application will not elaborate on this further.

[0113] Optionally, the first query request may further include address information of the synchronization client. After querying the target data, the database may directly return the target data to the synchronization client according to the address information of the synchronization client.

[0114] However, when the data screening conditions include classification sub-conditions and the time for querying the target data is greater than the preset time, it means that the synchronization client will inevitably be in a state of suspended animation because it cannot receive the return information within the preset time. Therefore, a proxy query can be used to query, so that the proxy module in the data synchronization device can maintain normal communication with the synchronization client to prevent the synchronization client from being suspended. At the same time, the proxy module waits for the database to query the data according to the screening conditions, or the proxy module uses the classification sub-conditions to perform a secondary screening on the candidate data filtered out according to the first composite index and the second composite index to ensure the reliability of the data query.

[0115] Furthermore, if Figure 4 As shown, the data screening conditions may also include classification sub-conditions, which are used to perform secondary screening of candidate data. When the data screening conditions include classification sub-conditions and the time for querying the target data is greater than the preset time, the query method is a proxy query, and according to the first query request, the target data for data synchronization is queried from the database.

[0116] Querying the database for target data for data synchronization according to the first query request includes:

[0117] Step 1023: Split the data screening condition into a composite index and classification sub-conditions.

[0118] It should be understood that the classification sub-conditions can be set according to actual conditions. When performing general incremental synchronization, the classification sub-conditions can be unrestricted conditions.

[0119] Step 1024: Send the composite index to the database.

[0120] Step 1025: Receive candidate data selected by the database based on the composite index.

[0121] It should be understood that, since the composite index mainly includes the time field and the sequence number field, the composite index can be used to hit and obtain a certain number of candidate data that meet the time range and sequence range.

[0122] It's important to note that when using proxy queries, you can filter and synchronize data during incremental synchronization, meaning you can conditionally filter incremental data before synchronizing it. You can also filter for target data, meaning the filtered database table can contain previously synchronized data. For example, if database table A has already synchronized first target data that meets the first classification sub-condition, then you can synchronize table A again based on the second classification sub-condition to synchronize second target data that meets the second classification sub-condition. The second classification sub-condition can be different from, the same as, or partially the same as the first classification sub-condition. In other words, proxy queries are not limited to incremental synchronization.

[0123] Step 1026: Classify and filter the candidate data according to the classification sub-conditions to obtain target data that meets the classification sub-conditions.

[0124] When conditionally filtering data, a composite index is usually used to limit the time range and serial number range of the data to be filtered. The data synchronization device first uses the composite index to obtain candidate data that meets the time range and serial number range, and then uses the classification sub-conditions in the local agent module of the data synchronization device to perform a secondary screening on the candidate data to obtain the final target data.

[0125] Specifically, the data synchronization device receives a first query request sent by the synchronization client, and then splits the data screening conditions in the first query request into a compound index and a classification sub-condition, wherein the compound index includes a first compound index and a second compound index. The data synchronization device sends the compound index to the database so that the database obtains the first target data set and the second target data set according to the first compound index and the second compound index query respectively. Then the database sorts the data in the first target data set and the second target data set in the order of time first and then serial number to obtain candidate data, and returns the candidate data to the data synchronization device. The data synchronization device receives the candidate data returned by the database, and performs a secondary screening on the candidate data according to the classification sub-condition, filters out the data that meets the classification sub-condition as the final target data, and feeds back the target data to the synchronization client.

[0126] Therefore, the present application utilizes the data synchronization device in the database system to access the database, effectively improving the access speed of massive data based on the database intranet and effectively reducing the delay time of data acquisition. Moreover, the use of the data synchronization device in the database system to perform a secondary screening of candidate data can effectively reduce the database's computational load and effectively improve the overall speed of data screening.

[0127] As a feasible embodiment, the composite index also includes the read data volume, where the read data volume is the number of candidate data that can be read from the database in each batch when the candidate data is read from the database in batches.

[0128] When the amount of candidate data is greater than the amount of read data, the composite index is sent to the database, including: updating the composite index according to the time field and sequence number field corresponding to the last candidate data currently received, and sending the updated composite index to the database.

[0129] In other words, when the amount of candidate data exceeds the amount of data to be read, it means that the candidate data cannot be read from the database to the data synchronization device all at once. In other words, the data synchronization device needs to synchronize the candidate data to the data synchronization device multiple times. In this case, the multiple data synchronization process can be converted into a continuous online incremental synchronization operation. Therefore, the composite index of the next data synchronization needs to be updated according to the time field and sequence number field of the last data after each synchronization to achieve the purpose of continuously synchronizing candidate data from the database.

[0130] Furthermore, if Figure 5 As shown, the composite index is updated, including:

[0131] Step 201: Increase the time field corresponding to the last piece of candidate data by one time granularity to obtain an updated time field.

[0132] The time granularity is the time level used when normalizing database data. For example, when data is normalized using the second-level time, one time granularity can correspond to one second. For example, when the time field corresponding to the last candidate data is mtime=2, the updated time field is mtime=3. Alternatively, when data is normalized using the millisecond level, one time granularity can correspond to one millisecond. In this case, the time field corresponding to the last candidate data is mtime=2ms, and the updated time field is mtime=3ms. In this embodiment, 2s and 3s refer to the second second and third second, and 2ms and 3ms refer to the second millisecond and third millisecond.

[0133] Step 202 : construct a third composite index using the updated time period, and construct a fourth composite index using an AND operation between the time field corresponding to the last candidate data and the updated sequence number field.

[0134] It should be noted that the third composite index is used to filter data generated after the last candidate data is generated, and the fourth composite index is used to filter data generated at the same time as the last candidate data and sorted after the last data.

[0135] Step 203: Perform an OR operation on the third composite index and the fourth composite index to obtain an updated composite index.

[0136] That is to say, by updating the composite index and obtaining the third composite index and the fourth composite index, the next synchronization requirement can be effectively converted to start from the last candidate data synchronized last time, effectively ensuring that the candidate data synchronized using the updated composite index can remain continuous with the candidate data synchronized last time in terms of time and serial number, without any data omission.

[0137] Specifically, taking data standardization using second-level time and English letter sequence as an example, when the time field corresponding to the last candidate data is 2 and the sequence number is a, that is, mtime=2, the updated time field is mtime=3. Therefore, the third compound index constructed is mitime>3, and the fourth compound index is mtime=2and id>a.

[0138] Therefore, the present application utilizes the updated composite index to ensure that candidate data are synchronized from the database in sequence according to time sequence and serial number sequence during multiple data synchronization processes.

[0139] As an optional embodiment, Figure 6 As shown, the target data is returned to the synchronization client, including:

[0140] Step 301, receive a first query request, the first query request is used to query the target data for data synchronization, the first query request includes data filtering conditions and query methods, the data filtering conditions are used to filter the target data that needs to be synchronized, the data filtering conditions include a composite index, and the composite index includes a time field and / or a serial number field.

[0141] Step 302: Query target data for data synchronization from a database according to a first query request.

[0142] Step 303: Receive a second query request, where the second query request is used to query the screening status of the target data.

[0143] Step 304 : When a preset condition is met, respond to the second query request and return the target data to the synchronization client, wherein the preset condition includes at least one of screening out the target data and completing screening of candidate data.

[0144] That is to say, after receiving the first query request, the data synchronization device queries the target data for data synchronization from the database according to the first query request, and then the target terminal can send a second query request to the data synchronization device at a certain frequency to query the status of data synchronization. When the data synchronization device filters out at least one target data or completes the screening of all data corresponding to the composite index, it responds to the second query request and returns the screening result.

[0145] Among them, filtering out the target data may include filtering out all target data, or may include filtering out at least one target data, that is, the data synchronization device may return the target data to the target terminal based on the second query request each time a target data is filtered out, or wait for the completion of the screening of all candidate data, that is, when the data screening conditions are executed (there will be no more target data), return the target data to the target terminal based on the second query request.

[0146] As a specific embodiment, Figure 7 As shown, the data synchronization method includes:

[0147] Step 401: The target terminal initiates a data synchronization task by carrying a first query request including a data screening condition and a query method.

[0148] Step 402: The data synchronization device returns task confirmation information to the target terminal.

[0149] Step 403: The data synchronization device uses the composite index in the data screening condition to initiate a data synchronization request to the database.

[0150] Step 404: The data synchronization device receives the candidate data returned by the database.

[0151] In step 405 , the data synchronization device performs a secondary screening on the candidate data returned in the current batch using the classification sub-conditions.

[0152] The screening result may include obtaining target data and not obtaining target data. When the target data is obtained, the filtered target data is cached.

[0153] Step 406: The data synchronization device updates the composite index and initiates a data synchronization request to the database using the updated composite index.

[0154] In step 407 , the data synchronization device performs a secondary screening on the candidate data returned in the current batch using the classification sub-conditions.

[0155] Steps 406 and 407 are repeatedly executed until the screening conditions are completely executed.

[0156] It should be understood that the composite index in step 403 includes the time field and serial number field of the initial data sent by the target terminal for data scanning, and step 406 is the second sending of the composite index after the initial data synchronization, that is, the time field and serial number field of the current initial position data need to be updated according to the candidate data returned in step 404. The subsequent repeated steps also update the composite index according to the candidate data returned last time to achieve the effect of synchronizing the data in the database in sequence.

[0157] Step 408: The target terminal sends a second query request to the data synchronization device after the first preset time.

[0158] Step 409: The data synchronization device returns unfinished status information to the target terminal.

[0159] Step 410: The target terminal sends a second query request to the data synchronization device after a second preset time.

[0160] The second preset time is less than or equal to the first preset time. For example, the first preset time may be 5 seconds, and the second preset time may be 1 second.

[0161] Step 411: The data synchronization device returns unfinished status information to the target terminal.

[0162] Steps 410 and 411 are repeatedly executed until the completion status information returned by the data synchronization device is received, and the target data returned by the data synchronization device is received.

[0163] There is no time sequence between steps 403 - 407 and steps 408 - 411 , that is, any step in steps 403 - 407 may be performed at the same time as any step in steps 408 - 411 .

[0164] It should be noted that after completing the screening, the data synchronization device can store the cached target data in the release queue. When the storage time of the target data reaches the preset time or queues to the release position, the data synchronization device deletes the stored target data to complete the release.

[0165] To sum up, the present application receives a first query request including data screening conditions and query methods sent by a synchronization client, and uses a composite index consisting of a time field and a serial number field in the data screening conditions to screen the target data, so that the screened target data can include both data that meets the time field requirements and data that meets the serial number requirements, and then returns the target data to the synchronization client, so that the synchronization client writes the target data into the database of the synchronization client to realize data synchronization, effectively avoiding the problem of missing data or repeated synchronization during the data synchronization process, and greatly improving the reliability of the online data synchronization process.

[0166] It should be noted that although the operations of the present method are described in a particular order in the drawings, this does not require or imply that the operations must be performed in this particular order, or that all illustrated operations must be performed to achieve desirable results.

[0167] Further, Figure 8 A block diagram of a data synchronization device proposed in an embodiment of the present application.

[0168] like Figure 8 As shown, the data synchronization device 10 includes:

[0169] A receiving module 11 is configured to receive a first query request sent by a synchronization client, the first query request being used to query target data for data synchronization, the first query request including a data screening condition and a query method, the data screening condition being used to screen target data to be synchronized, the data screening condition including a composite index including a time field and a sequence number field;

[0170] A screening module 12 is configured to query the database for target data for data synchronization according to the first query request;

[0171] The feedback module 13 is configured to return the target data to the synchronization client, so that the synchronization client writes the target data into a database of the synchronization client to achieve data synchronization.

[0172] In some embodiments, the data filtering condition includes a first compound index and a second compound index connected by an OR operation, the first compound index includes a time field, which is used to filter data generated after the time when the last data synchronized last was generated, and the second compound index includes a time field and a sequence number field, which is used to filter data generated at the same time as the last data synchronized last but sorted after the last data synchronized last.

[0173] In some embodiments, the data screening condition may further include a classification sub-condition, which is used to perform a secondary screening of the candidate data. When the data screening condition does not include a classification sub-condition or the data screening condition includes a classification sub-condition and the time for querying the target data is less than or equal to a preset time, the query method adopts direct query;

[0174] Querying the database for target data for data synchronization according to the first query request includes:

[0175] The data screening conditions are sent to the database, which is used to query the candidate data based on the composite index in the data screening conditions, and to perform a secondary screening on the candidate data based on the classification sub-conditions to obtain the target data that meets the classification sub-conditions;

[0176] Receive the target data filtered by the database according to the data filtering conditions.

[0177] In some embodiments, the data screening condition may further include a classification sub-condition, which is used to perform secondary screening of candidate data. When the data screening condition includes a classification sub-condition and the time to query the target data is greater than a preset time, the query method is a proxy query, and the screening module 12 is further used to:

[0178] Split data filtering conditions into compound indexes and classification sub-conditions;

[0179] Send the composite index to the database;

[0180] Receive candidate data filtered by the database based on the composite index;

[0181] The candidate data are classified and screened according to the classification sub-conditions to obtain the target data that meets the classification sub-conditions.

[0182] In some embodiments, the screening module 12 is further configured to:

[0183] Update the composite index based on the time field and sequence number field corresponding to the last candidate data currently received;

[0184] Send the updated composite index to the database.

[0185] In some embodiments, the screening module 12 is further configured to:

[0186] Increase the time field corresponding to the last candidate data by 1 time granularity to obtain the updated time field;

[0187] A third composite index is constructed using the updated time field, and a fourth composite index is constructed using an AND operation between the time field corresponding to the last candidate data and the sequence number field corresponding to the last candidate data;

[0188] An OR operation is performed on the third composite index and the fourth composite index to obtain an updated composite index.

[0189] In some embodiments, the receiving module 11 is further configured to: receive a second query request, the second query request being configured to query a screening status of the target data;

[0190] The feedback module 13 is further configured to respond to the second query request and return the target data to the synchronization client when a preset condition is met, wherein the preset condition includes at least one of screening out the target data and completing screening of candidate data.

[0191] In some embodiments, the data synchronization device 10 further includes:

[0192] The standardization module 14 is used to perform standardization processing on the data in the database.

[0193] To sum up, the present application receives a first query request including data screening conditions and query methods sent by a synchronization client, and uses a composite index consisting of a time field and a serial number field in the data screening conditions to screen the target data, so that the screened target data can include both data that meets the time field requirements and data that meets the serial number requirements, and then returns the target data to the synchronization client, so that the synchronization client writes the target data into the database of the synchronization client to realize data synchronization, effectively avoiding the problem of missing data or repeated synchronization during the data synchronization process, and greatly improving the reliability of the online data synchronization process.

[0194] It should be understood that the units or modules described in the data synchronization device 10 are the same as those in the reference Figure 2 The various steps in the described method correspond to each other. Therefore, the operations and features described above for the method are also applicable to the data synchronization device 10 and the units contained therein, and will not be repeated here. The data synchronization device 10 can be pre-implemented in the browser or other security applications of the electronic device, or loaded into the browser or its security application of the electronic device by downloading or other means. The corresponding units in the data synchronization device 10 can cooperate with the units in the electronic device to implement the solution of the embodiment of the present application.

[0195] The several modules or units mentioned in the detailed description above are not necessarily divided into one module or unit. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0196] It should be noted that for details not disclosed in the data synchronization device of the embodiment of the present application, please refer to the details disclosed in the above embodiments of the present application, which will not be repeated here.

[0197] Reference below Figure 10 , Figure 10 A schematic diagram of the structure of a computer system of an electronic device or server suitable for implementing the embodiments of the present application is shown.

[0198] like Figure 10 As shown, the computer system includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1002 or the program loaded from the storage part 1008 into the random access memory (RAM) 1003. Various programs and data required for the operation instructions of the system are also stored in the RAM 1003. The CPU 1001, ROM 1002 and RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0199] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, and the like; an output section 1007 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 1008 including a hard disk; and a communication section 1009 including a network interface card such as a LAN card or a modem. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 1010 as needed, so that computer programs read therefrom can be installed into the storage section 1008 as needed.

[0200] In particular, according to the embodiment of the present application, the above reference flow chart Figure 2 The described process can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 1009, and / or installed from a removable medium 1011. When the computer program is executed by the central processing unit (CPU) 1001, the above-mentioned functions defined in the system of the present application are executed.

[0201] It should be noted that the computer-readable medium shown in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical cable, RF, or any suitable combination thereof.

[0202] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operating instructions of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the aforementioned module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than the order marked in the accompanying drawings. For example, the boxes represented by two connections can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operating instruction, or can be implemented using a combination of dedicated hardware and computer instructions.

[0203] The units or modules involved in the embodiments described in the present application may be implemented by software or by hardware. The units or modules described may also be provided in a processor, for example, they may be described as: a processor including a receiving module and a screening module. The names of these units or modules do not, in certain cases, constitute a limitation on the units or modules themselves. For example, the receiving module may also be described as "receiving a first query request, the first query request being used to query target data for data synchronization, the first query request including data screening conditions and a query method, the data screening conditions being used to screen the target data to be synchronized, the data screening conditions including a composite index, the composite index including a time field and / or a serial number field".

[0204] As another aspect, the present application further provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments, or may exist independently and not be incorporated into the electronic device. The computer-readable storage medium stores one or more programs, which, when used by one or more processors, execute the data synchronization method described in the present application.

[0205] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the aforementioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A data synchronization method, characterized in that: include: Receive a first query request sent by a synchronization client, the first query request is used to query target data for data synchronization, the first query request includes a data screening condition and a query method, the data screening condition is used to screen the target data to be synchronized, the data screening condition includes a composite index and a classification sub-condition, and the composite index includes a time field and a sequence number field; According to the first query request, the target data for data synchronization is queried from the database; the classification sub-condition is used to perform secondary screening of candidate data; the candidate data is obtained by querying the database according to the composite index, and when the time for querying the target data is less than or equal to the preset time, the query mode is direct query, and the target data is obtained by the database performing secondary screening of the candidate data according to the classification sub-condition; or when the time for querying the target data is greater than the preset time, the query mode is proxy query, and the target data is obtained by classifying and screening the candidate data according to the classification sub-condition after receiving the candidate data filtered by the database according to the composite index; The target data is returned to the synchronization client, so that the synchronization client writes the target data into the database of the synchronization client to realize data synchronization.

2. The method according to claim 1, characterized in that The data screening condition includes a first composite index and a second composite index connected by an OR operation, the first composite index includes the time field, and is used to screen data generated after the generation time of the last data synchronized last time; the second composite index includes the time field and the sequence number field, and is used to screen data generated at the same time as the last data synchronized last time but sorted after the last data synchronized last time.

3. The method according to claim 1, characterized in that When the data screening condition does not include a classification sub-condition, the query method adopts direct query; The querying the target data for data synchronization from the database according to the first query request includes: sending the data screening condition to the database; The target data filtered out by the database according to the data filtering condition is received.

4. The method according to claim 1, wherein When the data screening condition includes a classification sub-condition and the time for querying the target data is greater than a preset time, the query mode is a proxy query, and querying the target data for data synchronization from the database according to the first query request includes: Splitting the data screening condition into the composite index and the classification sub-condition; sending the composite index to the database; receiving candidate data screened by the database according to the composite index; The candidate data are classified and screened according to the classification sub-conditions to obtain the target data that meets the classification sub-conditions.

5. The method according to claim 4, characterized in that The composite index further includes a read data amount. When the data amount of the candidate data is greater than the read data amount, sending the composite index to the database includes: Update the composite index according to the time field and sequence number field corresponding to the last piece of candidate data currently received; The updated composite index is sent to the database.

6. The method according to claim 5, characterized in that The updating of the composite index includes: Increasing the time field corresponding to the last piece of candidate data by 1 time granularity to obtain an updated time field; Constructing a third composite index using the updated time field, and constructing a fourth composite index using an AND operation between the time field corresponding to the last piece of candidate data and the sequence number field corresponding to the last piece of candidate data; An OR operation is performed on the third composite index and the fourth composite index to obtain the updated composite index.

7. The method according to any one of claims 4 to 6, characterized in that: The returning the target data to the synchronization client includes: receiving a second query request, where the second query request is used to query a screening status of the target data; When a preset condition is met, the target data is returned to the synchronization client in response to the second query request, wherein the preset condition includes at least one of filtering out the target data and completing the screening of the candidate data.

8. A data synchronization device, characterized in that: include: a receiving module, configured to receive a first query request sent by a synchronization client, the first query request being used to query target data for data synchronization, the first query request including a data screening condition and a query method, the data screening condition being used to screen the target data to be synchronized, the data screening condition including a composite index and a classification sub-condition, the composite index including a time field and a sequence number field; a screening module configured to query the target data for data synchronization from the database according to the first query request; the classification sub-condition is used to perform secondary screening on candidate data; the candidate data is obtained by querying the database according to the composite index, and when the time for querying the target data is less than or equal to the preset time, the query mode is direct query, and the target data is obtained by the database performing secondary screening on the candidate data according to the classification sub-condition; or when the time for querying the target data is greater than the preset time, the query mode is proxy query, and the target data is obtained by classifying and screening the candidate data according to the classification sub-condition after receiving the candidate data filtered by the database according to the composite index; The feedback module is configured to return the target data to the synchronization client, so that the synchronization client writes the target data into a database of the synchronization client to achieve data synchronization.

9. The device according to claim 8, characterized in that The data screening condition includes a first composite index and a second composite index connected by an OR operation, the first composite index includes the time field, and is used to screen data generated after the generation time of the last data synchronized last time; the second composite index includes the time field and the sequence number field, and is used to screen data generated at the same time as the last data synchronized last time but sorted after the last data synchronized last time.

10. The device according to claim 8, characterized in that When the data screening condition does not include a classification sub-condition, the query method adopts direct query; The screening module is also used to: sending the data screening condition to the database; The target data filtered out by the database according to the data filtering condition is received.

11. The device according to claim 8, characterized in that When the data screening condition includes a classification sub-condition and the time for querying the target data is greater than a preset time, the query mode is a proxy query, and the screening module is further configured to: Splitting the data screening condition into the composite index and the classification sub-condition; sending the composite index to the database; receiving candidate data screened by the database according to the composite index; The candidate data are classified and screened according to the classification sub-conditions to obtain the target data that meets the classification sub-conditions.

12. The device according to claim 11, characterized in that The screening module is further used for: Update the composite index according to the time field and sequence number field corresponding to the last piece of candidate data currently received; The updated composite index is sent to the database.

13. The device according to claim 12, characterized in that The screening module is further used for: Increasing the time field corresponding to the last piece of candidate data by 1 time granularity to obtain an updated time field; Constructing a third composite index using the updated time field, and constructing a fourth composite index using an AND operation between the time field corresponding to the last piece of candidate data and the sequence number field corresponding to the last piece of candidate data; An OR operation is performed on the third composite index and the fourth composite index to obtain the updated composite index.

14. The device according to any one of claims 11 to 13, characterized in that The receiving module is further configured to: receive a second query request, wherein the second query request is configured to query a screening status of the target data; The feedback module is further configured to respond to the second query request and return the target data to the synchronization client when a preset condition is met, wherein the preset condition includes at least one of filtering out the target data and completing the screening of the candidate data.

15. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, it implements the data synchronization method according to any one of claims 1 to 7.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the data synchronization method according to any one of claims 1 to 7 is implemented.

Images