Data synchronization system and method

Abstract

The present specification provides a data synchronization system and method, wherein the data synchronization system comprises: a database and a cache server configured by an information table associated with the database; the database is used to receive an update request submitted by a client for a database transaction processing task; in response to the update request, the initial data meeting the synchronization condition is filtered from the locally stored data and synchronized to the target cache of the cache server; the cache server is used to create a data filtering task for the initial data in the target cache; by executing the data filtering task, the target data is determined and deleted from the initial data stored in the target cache; the task execution result of the database transaction processing task is determined according to the deletion result, and the client is responded.

Description

Technical Field

[0001] This specification relates to the field of database technology, and in particular to data synchronization systems and methods. Background Technology

[0002] With the development of internet technology, caching, as a technique to improve data read and write efficiency, plays a crucial role in high-concurrency read services. In practical applications, the performance of caching is generally higher than that of the database when querying data. Querying the cache can save more query time compared to querying the database; for example, if querying the cache takes 1ms, querying the database may take 10ms. If a plan execution error occurs, the query time difference can be even greater. To leverage caching to improve data query efficiency, the data in the cache needs to be consistent with the data in the database. In existing technologies, when performing consistency operations on the cache and the database, for modification and deletion operations, the cache needs to be deleted first, then the database needs to be modified, and then the cache needs to be deleted again; while for queries, the cache needs to be queried first, and if no match is found, the database needs to be queried, and then the cache needs to be updated to ensure consistency. Although this operation achieves consistency, it is costly to implement and carries certain risks. Therefore, an effective solution is urgently needed to address the above problems. Summary of the Invention

[0003] In view of this, embodiments of this specification provide a data synchronization system. This specification also relates to a data synchronization method, a computing device, and a computer-readable storage medium to address the technical deficiencies existing in the prior art.

[0004] According to a first aspect of the embodiments of this specification, a data synchronization system is provided, including a database and a cache server configured with information tables associated with the database;

[0005] The database is used to receive update requests submitted by clients for database transaction processing tasks; in response to the update requests, it filters the data stored locally and synchronizes the initial data that meets the synchronization conditions to the target cache of the cache server.

[0006] The cache server is configured to create a data filtering task for the initial data in the target cache; determine target data and delete it from the initial data stored in the target cache by executing the data filtering task; and determine the task execution result of the database transaction processing task based on the deletion result, which is then used to respond to the client.

[0007] According to a second aspect of the embodiments of this specification, a data synchronization method is provided, applied to a data synchronization system, the data synchronization system including a database and a cache server configured with information tables associated with the database;

[0008] The database receives update requests submitted by the client for database transaction processing tasks; in response to the update requests, it filters the initial data that meets the synchronization conditions from the data stored locally and synchronizes it to the target cache of the cache server;

[0009] The cache server creates a data filtering task for the initial data in the target cache; by executing the data filtering task, it identifies and deletes target data from the initial data stored in the target cache; and determines the task execution result of the database transaction processing task based on the deletion result, which is then used to respond to the client.

[0010] According to a third aspect of the embodiments of this specification, a computing device is provided, comprising:

[0011] Memory and processor;

[0012] The memory is used to store computer-executable instructions, and the processor is used to implement the steps of the data synchronization method when executing the computer-executable instructions.

[0013] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions that, when executed by a processor, implement the steps of the data synchronization method.

[0014] The data synchronization system provided in this embodiment includes a database and a cache server configured with information tables associated with the database. The database is used to receive update requests submitted by clients for database transaction processing tasks. In response to the update request, it filters initial data that meets synchronization conditions from locally stored data and synchronizes it to a target cache on the cache server. The cache server is used to create a data filtering task for the initial data in the target cache. By executing the data filtering task, it identifies target data in the initial data stored in the target cache and deletes it. Based on the deletion result, it determines the task execution result of the database transaction processing task and responds to the client.

[0015] In summary, to ensure data consistency between the database and cache at all times and improve data query efficiency, after the database receives an update request submitted for a database transaction processing task, it responds by filtering out initial data that meets the synchronization conditions from the locally stored data and writing it into the target cache. This performs an initial screening of all data stored in the database, filtering out data that does not need to be synchronized. At this point, the target cache already contains some data, and the database and cache have performed consistency operations. However, considering that the initial data currently stored in the cache may still contain some redundant data that is not needed in the data query phase, the cache server can create a data filtering task for the initial data in the cache. This task filters out target data from the initial data and deletes it. The result of the database transaction processing task is then determined based on the deletion result, and the client is responded to. Implementing consistency management through a server-side driver effectively reduces the cost of consistency operations and the risk of data loss, thereby ensuring strong consistency between the database and cache. This allows the cache to respond to query requests at any time, improving data access efficiency. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of a data synchronization system provided in one embodiment of this specification;

[0017] Figure 2 This is a schematic diagram of the structure of a data synchronization system provided in one embodiment of this specification;

[0018] Figure 3 This is a flowchart illustrating the processing of a data synchronization system according to an embodiment of this specification;

[0019] Figure 4 This is a flowchart of a data synchronization method provided in one embodiment of this specification;

[0020] Figure 5 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0021] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0022] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0023] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0024] This specification provides a data synchronization system, a data synchronization method, a computing device, and a computer-readable storage medium, which are described in detail in the following embodiments.

[0025] In practical applications, while achieving eventual consistency in databases and caches can meet the expected requirements, it is complex and costly. That is, for a single application or service, achieving consistency between the database and cache incurs a certain cost. In a microservice architecture, there are many different microservices written in different languages ​​that simultaneously access the same database. For example, operations management, transaction, and risk control clients all access the database. To ensure data consistency within the cache, these clients all access the same Redis instance. Since developers of these servers typically belong to different teams with varying technical preferences, backgrounds, skill levels, and understandings, and each language has its own drivers for cache and database read / write operations (e.g., JDBC, ODBC, native SDKCOCI, libpq, mysqlclient, etc.), it becomes extremely difficult to rigorously guarantee the consistency of cache and database updates. Because consistency cannot be automatically guaranteed through mechanisms, the implementation cost is very high and carries significant risks.

[0026] In view of this, see Figure 1The schematic diagram shows that the data synchronization system includes a database and a cache server configured with information tables associated with the database. The database receives update requests submitted by clients for database transaction processing tasks. In response to the update request, it filters initial data that meets synchronization conditions from locally stored data and synchronizes it to the target cache of the cache server. The cache server creates a data filtering task for the initial data in the target cache. By executing the data filtering task, it identifies and deletes target data from the initial data stored in the target cache. Based on the deletion result, it determines the task execution result of the database transaction processing task and responds to the client.

[0027] In summary, to ensure data consistency between the database and cache at all times and improve data query efficiency, after the database receives an update request submitted for a database transaction processing task, it responds by filtering out initial data that meets the synchronization conditions from the locally stored data and writing it into the target cache. This performs an initial screening of all data stored in the database, filtering out data that does not need to be synchronized. At this point, the target cache already contains some data, and the database and cache have performed consistency operations. However, considering that the initial data currently stored in the cache may still contain some redundant data that is not needed in the data query phase, the cache server can create a data filtering task for the initial data in the cache. This task filters out target data from the initial data and deletes it. The result of the database transaction processing task is then determined based on the deletion result, and the client is responded to. Implementing consistency management through a server-side driver effectively reduces the cost of consistency operations and the risk of data loss, thereby ensuring strong consistency between the database and cache. This allows the cache to respond to query requests at any time, improving data access efficiency.

[0028] Figure 2 A schematic diagram of a data synchronization system according to an embodiment of this specification is shown. The data synchronization system 200 includes a database 210 and a cache server 220 configured with information tables associated with the database.

[0029] The database 210 is used to receive update requests submitted by clients for database transaction processing tasks; in response to the update request, it filters the data stored locally that meets the synchronization conditions and synchronizes the initial data to the target cache of the cache server.

[0030] The cache server 220 is configured to create a data filtering task for the initial data in the target cache; determine target data and delete it from the initial data stored in the target cache by executing the data filtering task; and determine the task execution result of the database transaction processing task based on the deletion result, in order to respond to the client.

[0031] Specifically, "database" refers to a database used for storing data, including but not limited to business data, multimedia data, and user data; this embodiment does not impose any limitations. Correspondingly, "client" refers to a client with data access needs, such as an operations management client, a transaction client, or a risk control client; the client may vary depending on the scenario. Correspondingly, "caching server" refers to a server used to manage the target cache, which has a data synchronization relationship with the database to improve data access efficiency. It includes, but is not limited to, servers corresponding to distributed caches such as Redis, Memcached, ZooKeeper, and etcd; this embodiment does not impose any limitations.

[0032] Accordingly, database transaction processing tasks specifically refer to tasks that require consistency processing of data between the cache and the database, including but not limited to modifying, updating, and adding data. Correspondingly, update requests specifically refer to requests submitted for consistency processing operations; these requests originate from the client and are submitted by the client according to its needs. Correspondingly, locally stored data specifically refers to data stored in the database; this data type includes, but is not limited to, business data, multimedia data, user data, etc., and this embodiment does not impose any limitations. Correspondingly, synchronization conditions specifically refer to filtering data that can be synchronized to the cache. Correspondingly, the target cache specifically refers to the cache used to store data; responding to data query requests through the target cache can be faster than reading data from the database.

[0033] Accordingly, the data filtering task specifically refers to the task used to filter the initial data in the target cache. This involves deleting data that has been modified or updated to ensure that there is no redundant data in the target cache, thereby improving resource utilization. The target data specifically refers to the data that needs to be deleted from the initial data; this data will not be used when reading data from the cache.

[0034] Therefore, to synchronize data from the database to the cache while avoiding redundant data that consumes excessive storage resources, the database can first filter initial data that meets synchronization conditions from its local storage after receiving an update request from a client for a database transaction processing task. This allows for the local filtering of data that meets synchronization requirements, which is then synchronized to the target cache on the cache server. Subsequently, after the cache server managing the target cache determines the initial data to be written, a data filtering task can be created to remove redundant data. By executing this task, target data—data that can be deleted from the initial data stored in the target cache—is identified and deleted. This removes data that consumes storage resources, achieving strong consistency between the database and the cache. The result of this deletion can then be used to determine the execution result of the database transaction processing task, which is then used to respond to the client.

[0035] Furthermore, to synchronize data from the database to Redis, the cache server needs to be configured first. This involves configuring its attributes using external mapping tables and `create table` or `alter table` statements, specifying `WITH(SERVER=redis_server, user=user, fdw_table=myredishash)`. After configuring the cache server, its configuration information can be found at runtime via `cache_SERVER`, `cache_user_mapping`, and `cache_fdw_table`. This facilitates subsequent data synchronization operations.

[0036] Furthermore, considering that database and cache operations involving inserts, deletes, and updates do not automatically preload, a separate target interface can be provided to implement the preloading operation in order to load data into the cache. Since a corresponding external mapping table has already been pre-defined for the cache, the target function can be directly executed to load data from the database into the cache; that is, executing `insert into user_cache_hash_fdw(key,val)select *from user` loads the data from the `user` key into the cache.

[0037] In summary, to ensure data consistency between the database and cache at all times and improve data query efficiency, after the database receives an update request submitted for a database transaction processing task, it responds by filtering out initial data that meets the synchronization conditions from the locally stored data and writing it into the target cache. This performs an initial screening of all data stored in the database, filtering out data that does not need to be synchronized. At this point, the target cache already contains some data, and the database and cache have performed consistency operations. However, considering that the initial data currently stored in the cache may still contain some redundant data that is not needed in the data query phase, the cache server can create a data filtering task for the initial data in the cache. This task filters out target data from the initial data and deletes it. The result of the database transaction processing task is then determined based on the deletion result, and the client is responded to. Implementing consistency management through a server-side driver effectively reduces the cost of consistency operations and the risk of data loss, thereby ensuring strong consistency between the database and cache. This allows the cache to respond to query requests at any time, improving data access efficiency.

[0038] Furthermore, when synchronizing data from the database to the target cache, considering that the database may use different cache servers, and therefore the corresponding target caches will also be different, it is necessary to create different data synchronization tasks for different cache servers during data synchronization. This is so that when executing the data synchronization task, the data can be synchronized to the target cache corresponding to the cache server. In this embodiment, the specific implementation is as follows:

[0039] The database is also used to parse the update request, create a data synchronization task based on the parsing result, determine the statement type corresponding to the update request by executing the data synchronization task, and filter the initial data that meets the synchronization conditions in the locally stored data according to the statement type to synchronize to the target cache of the cache server.

[0040] Specifically, a data synchronization task refers to the task of determining the statement type of an update request. This ensures that, once the statement type is determined, initial data can be selected from locally stored data according to the statement type for synchronization processing. Correspondingly, the statement type refers to the type of statement corresponding to the update request, such as an SQL statement.

[0041] Based on this, when the database receives an update request, in order to ensure that the data to be synchronized meets the conditions, the update request can be parsed first, and a data synchronization task can be created based on the parsing results. Then, by executing the data synchronization task, the statement type corresponding to the update request can be determined. Considering that different databases correspond to different cache servers, the initial data that meets the synchronization conditions can be selected from the locally stored data according to the statement type and synchronized to the target cache of the cache server to achieve initial consistency between the two.

[0042] In other words, when performing consistency operations between the database and the cache, the client first sends the update request SQL to the database. The database can then parse the update request SQL and optimize it to generate a data synchronization task, i.e., an execution plan, which can then be executed to perform data synchronization processing.

[0043] In summary, by combining data synchronization tasks with the method of determining statement types for data synchronization processing, it can be ensured that the current data synchronization task corresponds to the cache server, so that data synchronization can be performed according to the rules of the cache server with configured information, thus ensuring the initial consistency of the data between the two.

[0044] Based on this, in order to initially filter out the data that needs to be synchronized when synchronizing data from the database to the cache, a target operator can be invoked according to the type of the request statement. This allows the storage engine to filter the initial data that meets the synchronization conditions for synchronization through the target operator. In this embodiment, the specific implementation is as follows:

[0045] The database is also used to determine the target operator corresponding to the database transaction processing task based on the statement type; and to call the storage engine associated with the cache server through the target operator, and to filter the initial data that meets the synchronization conditions from the data stored locally according to the set reading strategy and synchronize it to the target cache of the cache server.

[0046] Specifically, the target operator refers to the algorithm used to call the storage engine to filter data. Correspondingly, the preset read strategy refers to the strategy for reading data from local storage. For example, the loop read strategy can filter all data to select the initial data that meets the usage requirements.

[0047] Based on this, when the database synchronizes data according to statement type, in order to implement the rules of the cache server, the target operator corresponding to the database transaction processing task can be determined according to the statement type. At this time, the storage engine of the associated cache server is called through the target operator, so that the initial data that meets the synchronization conditions is selected from the data stored locally by the storage engine according to the set reading strategy and synchronized to the target cache of the cache server.

[0048] In other words, after determining the update request SQL, the SQL executor can be invoked to determine the statement type of the update request, and the corresponding implementation can be entered based on the determination result. During this process, for DML (Data Manipulation Language), the implementation of the operator ModifyTable, nodeModifyTable.c, can be called, making it convenient to use this operator to call the storage engine for update operations later.

[0049] In summary, by calling the storage engine to perform data synchronization processing according to the set read strategy, it can be ensured that the initial data filtered by the database meets the synchronization conditions. On this basis, it can be ensured that the target cache will not miss data, so as to facilitate the processing of redundant data in the target cache.

[0050] Furthermore, when the database performs filtering before synchronization, it actually selects data that meets certain conditions as initial data to be synchronized to the target cache. In this process, in order to ensure that all data that needs to be synchronized is synchronized to the target cache, filtering and checking can be performed. In this embodiment, the specific implementation is as follows:

[0051] The database is also used to read sub-synchronization conditions and sub-detection conditions from the synchronization conditions; to filter data that meets the sub-synchronization conditions from the locally stored data as data to be synchronized, and to detect the data to be synchronized; to select the data to be synchronized that meets the sub-detection conditions as initial data based on the detection results, and to synchronize it to the target cache of the cache server.

[0052] Specifically, the sub-synchronization condition refers to filtering data that meets the synchronization requirements, i.e., the data to be synchronized; correspondingly, the sub-detection condition refers to further filtering the data that meets the synchronization requirements and passing the detection, so as to ensure that the data finally written to the cache is data that does not exist in the cache.

[0053] Based on this, when judging whether data meets the synchronization conditions, in order to avoid missing data, the process can be performed twice. First, read the sub-synchronization conditions and sub-detection conditions in the synchronization conditions; then, filter the data that meets the sub-synchronization conditions from the data stored locally in the database as the data to be synchronized; then, detect the data to be synchronized; and finally, select the data to be synchronized that meets the sub-detection conditions as the initial data based on the detection results and synchronize it to the target cache of the cache server.

[0054] In other words, after determining the target operator, the corresponding storage engine can be called through the implementation of the target operator to read all tuples that meet the conditions from the tuples stored in the database. After obtaining the tuples, all tuples that meet the conditions are checked. If the check passes, the tuples are updated to the target cache. Since update operations usually include a WHERE clause, or may not, when writing tuples to the cache, all tuples that have actually been modified in the database can be selected. Then, tuples that satisfy MVCC (Multiversion Concurrency Control) are selected for checking. If the check passes and ExecModifyTable's junkfilter == true, the ctid can be extracted for UPDATE / DELETE, and the ExecUpdate function is called based on the extracted ctid. This calls table_tuple_update->heapam_tuple_update->heap_update to update the tuples that meet the conditions to the target cache.

[0055] In summary, by combining sub-synchronization conditions and sub-detection conditions to filter the data that needs to be synchronized to the cache, it can be ensured that the synchronized data meets the synchronization requirements, thus avoiding the waste of storage resources.

[0056] After synchronizing the initial data from the database to the target cache, considering that the data in the target cache at this time not only contains normal data but may also contain redundant data, and that redundant data can largely waste storage resources, three filtering tasks can be created to delete different types of redundant data. Each task is executed sequentially to filter out the target data that needs to be deleted. In this embodiment, the specific implementation is as follows:

[0057] The cache server is further configured to create a first filtering task, a second filtering task, and a third filtering task for the initial data in the target cache, wherein the first filtering task, the second filtering task, and the third filtering task constitute the data filtering task; by executing the first filtering task, a first target data is determined from the initial data stored in the target cache; by executing the second filtering task, a second target data is determined from the initial data stored in the target cache; and by executing the third filtering task, a third target data is determined from the initial data stored in the target cache; the first target data, the second target data, and the third target data are then used as the target data and deleted.

[0058] Specifically, the first filtering task refers to filtering data that has changed due to modification operations; correspondingly, the first target data is the data that has changed due to modification. Similarly, the second filtering task refers to filtering data that has changed due to truncation operations; correspondingly, the second target data is the data in the cache that has changed due to truncation operations. Likewise, the third filtering task refers to filtering data that has changed due to table structure modification operations; correspondingly, the third target data is the data in the cache that has changed due to table structure modification operations.

[0059] Based on this, after the database completes the synchronization of the initial data to the target cache, in order to avoid redundant data occupying the storage resources in the cache, three filtering tasks can be created for the initial data in the target cache to filter different types of redundant data. These are the first filtering task, the second filtering task, and the third filtering task, which together constitute the data filtering task.

[0060] Afterwards, the first target data can be determined from the initial data stored in the target cache by executing the first filtering task, the second target data can be determined from the initial data stored in the target cache by executing the second filtering task, and the third target data can be determined from the initial data stored in the target cache by executing the third filtering task. It should be noted that the first target data, the second target data, and the third target data are three different types of data. Then, the first target data, the second target data, and the third target data can be used as target data and deleted.

[0061] In summary, by creating multiple filtering tasks for the initial data in the target cache, each filtering task can delete different redundant data, thereby reducing the impact of redundant data from multiple dimensions and avoiding the waste of cache storage resources.

[0062] When performing the first filtering task, the actual process is to delete the modified data. In this embodiment, the specific implementation is as follows:

[0063] The cache server is further configured to determine the primary key storage information corresponding to the initial data stored in the target cache by executing the first filtering task; and if the primary key storage information is not empty, call the target encapsulation library to determine the first modified data in the initial data stored in the target cache as the first target data.

[0064] Specifically, primary key storage information refers to the data structure used to store primary key information. Correspondingly, the target encapsulation library refers to the library that encapsulates functions that can perform data deletion processing. It is a custom encapsulation library based on consistency operations.

[0065] Based on this, when executing the first filtering task, the data that needs to be modified is actually deleted from the initial data stored in the target cache. At this time, the primary key storage information corresponding to the initial data stored in the target cache can be determined by executing the first filtering task. And if the primary key storage information is not empty, the target encapsulation library is called to determine the first modified data in the initial data stored in the target cache, which is then used as the first target data.

[0066] In other words, after the database synchronizes data to the cache as described above, considering that the synchronized data may contain some redundant data, which will occupy some storage resources and is useless, in order to avoid wasting storage resources, three filtering tasks can be created for the data just synchronized to the target cache: the first filtering task, the second filtering task, and the third filtering task. These tasks are used to execute different filtering tasks to delete different redundant data, thereby ensuring strong consistency between the database and the cache. Based on this, when executing the first filtering task, considering that all tuples have already been processed, and the MessageContext stores the primary keys of all tuples, which are redundant relative to the cache; therefore, if it is confirmed that the MessageContext is not empty, the encapsulated cache_provider_wrapper library can be called to delete the modified records from the cache, that is, to delete the tuples that have been modified in the database. These tuples, after being updated to the cache, will not be requested and hit in subsequent uses, so they need to be deleted.

[0067] Furthermore, to ensure that no data that needs to be deleted is missed, a double confirmation can be performed. In this embodiment, the specific implementation is as follows:

[0068] The cache server is also configured to, upon detecting the execution of a caching policy, call the target encapsulation library to determine second modified data from the initial data stored in the target cache, as the first target data.

[0069] Specifically, the running caching strategy refers to the strategy enabled by the database. When this strategy is enabled, the database will generate data that records the corresponding table names.

[0070] Therefore, in order to avoid omissions, the modified data can be filtered and determined again through the cache server. That is, when the caching policy is detected, the target encapsulation library can be called to determine the second modified data from the initial data stored in the target cache, and use it as the first target data.

[0071] In other words, considering that transaction commit points are always managed and executed by the transaction engine, a second check can be performed to avoid omissions. That is, whether it's an implicit or explicit transaction, the `finish_xact_command` function in `postgres.c` will be called to execute the commit. Therefore, at this stage, it can be determined whether `autocache_enable` is enabled. If it is enabled, it means there are still tuples that need to be deleted. At this point, the encapsulated `cache_provider_wrapper` library can be called to delete the modified records from the cache, i.e., delete the tuples that have been modified in the database. These tuples, after being updated to the cache, will not be requested and hit in subsequent uses, so they need to be deleted; at the same time, the memory allocated in `MessageContext` also needs to be deleted.

[0072] When performing the second filtering task, the data that has changed due to the truncation operation is actually deleted. In this embodiment, the specific implementation is as follows:

[0073] The cache server is also configured to invoke a truncation function by executing the second filtering task; if it is detected that the truncation function is running a caching strategy, it calls the target encapsulation library to read the truncation table name; and determines the second target data in the initial data stored in the target cache based on the truncation table name.

[0074] Specifically, the truncation function refers to the function called during the truncation operation to perform the truncation processing. Correspondingly, the truncation table name refers to the name of the table generated during the truncation operation.

[0075] Based on this, when executing the second filtering task, the actual process is to delete the data that has changed due to the truncation operation. At this time, the cache server can call the truncation function to execute the second filtering task. If it is detected that the truncation function is running the caching strategy, it means that there is data that needs to be deleted. At this time, the target encapsulation library can be called to read the truncation table name. The second target data can be determined from the initial data stored in the target cache based on the truncation table name.

[0076] In other words, truncation operations and table structure changes in the database generate redundant data. Therefore, after updating the tuple to the cache, this redundant data also needs to be deleted. Specifically, since the cache typically stores data in the format db:schema:table:key:value, the cache level is equivalent to all key / value pairs under a table. Because DDL (Data Definition Language) is executed, all key / value pairs under the table naturally lack information; therefore, to ensure correctness, they need to be cleared.

[0077] Based on this, for truncation operations, the ExecuteTruncate function can be called. Inside ExecuteTruncate, it can be checked whether autocache_enable is enabled. If it is enabled, it means that there are still tuples that need to be deleted. Then, the encapsulated cache_provider_wrapper library can be called, passing the name of the table to be deleted, so as to determine the namespace corresponding to the table to be deleted in the cache, and directly delete the namespace corresponding to the table.

[0078] When performing the third filtering task, the actual process involves deleting data that has changed due to table structure modification operations. In this embodiment, the specific implementation is as follows:

[0079] The cache server is also used to call the data update function by executing the third filtering task; when it is detected that the data update function is running a caching strategy, it calls the target encapsulation library to read the update table name; and determines the third target data in the initial data stored in the target cache according to the update table name.

[0080] Specifically, the data update function refers to the function used when processing changes to the table structure, and correspondingly, updating the table name refers to the name information of the table before the update.

[0081] Based on this, when executing the third filtering task, the actual process is to delete the data that has changed due to the table structure modification operation. At this time, the cache server can call the data update function by executing the third filtering task. If the cache policy is detected to be running by the data update function, it means that there is data that needs to be deleted. At this time, the target encapsulation library can be called to read the updated table name. The third target data can then be determined from the initial data stored in the target cache based on the updated table name.

[0082] In other words, for table structure modification operations, the AlterTable function can be called. Inside AlterTable, it is determined whether autocache_enable is enabled. If it is enabled, it means that there are still tuples that need to be deleted. Then, the encapsulated cache_provider_wrapper library can be called, passing the name of the table to be deleted, so that the namespace corresponding to the deleted table can be determined in the cache, and the namespace corresponding to the table can be deleted directly.

[0083] In other words, the name of the table to be deleted will be passed to the cache_provider_wrapper library by exectetruncate, and the cache_provider_wrapper library will be responsible for deleting the namespace.

[0084] It should be noted that during dirty data deletion, the system checks whether autocache_enable is enabled at different stages. In fact, the system checks the value of the expanded planstate data structure to determine whether it is enabled based on the current value.

[0085] After ensuring consistency between the database and the target cache, the client can directly read data from the cache when performing data access, thereby improving data access efficiency. In this embodiment, the specific implementation is as follows:

[0086] The data synchronization system also includes the client; the client is used to send a target query request to the cache server; the cache server is used to respond to the target query request, read the target data in the target cache and feed it back to the client.

[0087] Based on this, once the database and the target cache are consistent, the client can directly read data from the target cache when reading data. That is, the client can send a target query request to the cache server; at this time, the cache server will respond to the target query request, read the target data from the target cache and send it back to the client.

[0088] At this point, all redundant data in the cache has been cleared. This confirms the completion of the entire database transaction, and the cache has also successfully processed the data in the database, thus ready to respond to the client. From this point onward, whether it's this client or another client, no expired data will be found when accessing the cache, enabling fast data retrieval and improving data access efficiency. In other words, after a client sends a target query request to the cache server, the server can directly retrieve the target data from the target cache based on the query request and return it to the client.

[0089] Furthermore, on the database side, in addition to extending kernel support, proactive pushing is also achieved through triggers. Due to the inherent low performance and business intrusion of triggers, the kernel implementation specifies the triggers through flags during `create table / atter table`, which is completely non-business intrusive, supports any operation and data type, and allows for on-demand configuration per table.

[0090] Furthermore, through the external table interface, it supports automatically loading data into Redis at specified time points, without requiring application intervention. This mechanism not only significantly reduces the risk of cache inconsistency but also greatly improves efficiency and out-of-the-box convenience during initial loading.

[0091] The following is in conjunction with the appendix Figure 3 Taking the data synchronization system provided in this manual as an example in a Redis synchronization scenario, the data synchronization system will be further explained. Figure 3 This specification shows a flowchart of a data synchronization system provided in one embodiment, which specifically includes a client, a database, and a cache server configured with information tables associated with the database.

[0092] In step S302, the database parses the update request and creates a data synchronization task based on the parsing result.

[0093] This embodiment uses the example of synchronizing data from a database to Redis to illustrate the specific implementation process of a data synchronization system. Descriptions of other application scenarios can refer to the same or similar descriptions in this embodiment, and no limitations are made here.

[0094] It's important to note that to synchronize data from the database to Redis, the cache server needs to be configured first. This involves using an external mapping table and configuring its attributes using `create table` or `alter table`, specifying `WITH(SERVER=redis_server, user=user, fdw_table=myredishash)`. After configuring the cache server, its configuration information can be found at runtime via `cache_SERVER`, `cache_user_mapping`, and `cache_fdw_table`. This facilitates subsequent data synchronization operations.

[0095] Furthermore, considering that database and cache operations involving inserts, deletes, and updates do not automatically preload, a separate target interface can be provided to implement the preloading operation in order to load data into the cache. Since a corresponding external mapping table has already been pre-defined for the cache, the target function can be directly executed to load data from the database into the cache; that is, executing `insert into user_cache_hash_fdw(key,val)select *from user` loads the data from the `user` key into the cache.

[0096] Building upon this, when performing consistency operations between the database and the cache, the client first sends the update request SQL to the database. The database can then parse the update request SQL and optimize it to generate a data synchronization task, i.e., an execution plan, which can then be executed to perform subsequent data synchronization operations.

[0097] In step S304, the database determines the statement type corresponding to the update request by executing a data synchronization task, and determines the target operator corresponding to the database transaction processing task based on the statement type.

[0098] Specifically, after determining the update request SQL, the SQL executor can be invoked to determine the statement type of the update request, and the corresponding implementation can be entered based on the determination result. During this process, for DML (Data Manipulation Language), the implementation of the operator ModifyTable, nodeModifyTable.c, can be called, facilitating subsequent use of this operator to call the storage engine for update operations.

[0099] In step S306, the database calls the storage engine of the associated cache server through the target operator, selects data that meets the sub-synchronization conditions from the data stored locally according to the set reading strategy, and checks the data to be synchronized.

[0100] Step S308: Select the data to be synchronized that meets the sub-detection conditions as the initial data based on the detection results, and synchronize it to the target cache of the cache server.

[0101] Specifically, after determining the target operator, the corresponding storage engine can be called through the implementation of the target operator to read all tuples that meet the conditions from the tuples stored in the database. After obtaining the tuples, all tuples that meet the conditions will be checked, and the tuples will be updated to the target cache after the check is passed.

[0102] In other words, since update usually includes a WHERE clause or not, when writing tuples to the cache, you can select all tuples in the database that have actually been modified. Then, select tuples that meet MVCC (Multiversion Concurrency Control) for checking. If the check passes and ExecModifyTable's junkfilter == true, you can extract the ctid for UPDATE / DELETE and call the ExecUpdate function based on the extracted ctid. This will call table_tuple_update->heapam_tuple_update->heap_update to update the target cache with the tuples that meet the above conditions.

[0103] In step S310, the cache server creates a first filtering task, a second filtering task, and a third filtering task for the initial data in the target cache.

[0104] In step S312, the cache server determines the primary key storage information corresponding to the initial data stored in the target cache by performing the first filtering task.

[0105] In step S314, if the primary key storage information is not empty, the cache server calls the target encapsulation library to determine the first modified data in the initial data stored in the target cache, and uses it as the first target data.

[0106] Specifically, after the database synchronizes data to the cache as described above, considering that the synchronized data may contain some redundant data, which will occupy some storage resources and is useless, in order to avoid wasting storage resources, three filtering tasks can be created for the data just synchronized to the target cache: the first filtering task, the second filtering task, and the third filtering task. These tasks are used to execute different filtering tasks to delete different types of redundant data, thereby ensuring strong consistency between the database and the cache.

[0107] Based on this, when performing the first filtering task, considering that all tuples have already been processed beforehand, and that MessageContext stores the primary keys of all tuples, which is redundant relative to the cache; therefore, if it is confirmed that MessageContext is not empty, the encapsulated cache_provider_wrapper library can be called to delete the changed records from the cache, that is, to delete the tuples that have been modified in the database. These tuples will not be requested and hit in subsequent use after being updated to the cache, so they need to be deleted.

[0108] In step S316, when the cache server detects that a caching policy is in operation, it calls the target encapsulation library to determine the second modified data from the initial data stored in the target cache, and uses it as the first target data.

[0109] Based on the above detection and processing, considering that transaction commit points are always executed by the transaction engine, a second detection can be performed to avoid omissions. That is, whether it's an implicit or explicit transaction, the `finish_xact_command` function in `postgres.c` will be called to execute the commit. Therefore, at this stage, it can be determined whether `autocache_enable` is enabled. If it is enabled, it means there are still tuples that need to be deleted. At this point, the encapsulated `cache_provider_wrapper` library can be called to delete the modified records from the cache, i.e., delete the tuples that have been modified in the database. These tuples, after being updated to the cache, will not be requested and hit in subsequent uses, so they need to be deleted; at the same time, the memory allocated in `MessageContext` also needs to be deleted.

[0110] In step S318, the cache server calls the truncation function by executing the second filtering task.

[0111] In step S320, when the cache server detects that the truncation function is executed and the caching strategy is running, it calls the target encapsulation library to read the truncation table name and determines the second target data in the initial data stored in the target cache based on the truncation table name.

[0112] In step S322, the cache server calls the data update function by executing the third filtering task.

[0113] In step S324, when the cache server detects that the data update function is running the caching strategy, it calls the target encapsulation library to read the update table name and determines the third target data in the initial data stored in the target cache based on the update table name.

[0114] Furthermore, truncation operations and table structure changes in the database generate redundant data. Therefore, after updating the tuple to the cache, this redundant data also needs to be deleted. In other words, since the cache typically stores data in the format db:schema:table:key:value, the cache level is equivalent to all key / value pairs under a table. Because DDL (Data Definition Language) is executed, all key / value pairs under the table naturally lack information; therefore, to ensure correctness, a clearing process is required.

[0115] Based on this, for truncation operations, the ExecuteTruncate function can be called. Inside ExecuteTruncate, it is determined whether autocache_enable is enabled. If it is enabled, it means that there are still tuples that need to be deleted. Then, the encapsulated cache_provider_wrapper library can be called, passing the name of the table to be deleted, so as to determine the namespace corresponding to the table to be deleted in the cache, and directly delete the namespace corresponding to the table.

[0116] For table structure modification operations, the AlterTable function can be called. Inside AlterTable, it is determined whether autocache_enable is enabled. If it is enabled, it means that there are still tuples that need to be deleted. Then, the encapsulated cache_provider_wrapper library can be called, passing the name of the table to be deleted, so that the namespace corresponding to the deleted table can be determined in the cache, and the namespace corresponding to the table can be deleted directly.

[0117] In other words, the name of the table to be deleted will be passed to the cache_provider_wrapper library by exectetruncate, and the cache_provider_wrapper library will be responsible for deleting the namespace.

[0118] It should be noted that during dirty data deletion, the system checks whether autocache_enable is enabled at different stages. In fact, the system checks the value of the expanded planstate data structure to determine whether it is enabled based on the current value.

[0119] In step S326, the cache server deletes the first target data, the second target data, and the third target data from the target cache, and determines the task execution result of the database transaction processing task based on the deletion result, which is then used to respond to the client.

[0120] Specifically, after the first, second, and third filtering tasks are completed, the first, second, and third target data can be determined. At this point, the three sets of data are deleted from the target cache, and the execution result of the database transaction processing task can be determined based on the deletion result, which is then used to respond to the client.

[0121] In step S328, the client sends a target query request to the cache server.

[0122] In step S330, the cache server responds to the target query request by reading the target data from the target cache and sending it back to the client.

[0123] Specifically, at this point, all redundant data in the cache has been cleared. This confirms the completion of the entire database transaction, and the cache has also successfully processed the data in the database, thus enabling it to respond to the client. From this point onward, whether it's this client or another client, no expired data will be found when accessing the cache, allowing for fast data retrieval and improved data access efficiency. In other words, after a client sends a target query request to the cache server, the server can directly retrieve the target data from the target cache based on the query request and return it to the client.

[0124] In summary, to ensure data consistency between the database and cache at all times and improve data query efficiency, after the database receives an update request submitted for a database transaction processing task, it responds by filtering out initial data that meets the synchronization conditions from the locally stored data and writing it into the target cache. This performs an initial screening of all data stored in the database, filtering out data that does not need to be synchronized. At this point, the target cache already contains some data, and the database and cache have performed consistency operations. However, considering that the initial data currently stored in the cache may still contain some redundant data that is not needed in the data query phase, the cache server can create a data filtering task for the initial data in the cache. This task filters out target data from the initial data and deletes it. The result of the database transaction processing task is then determined based on the deletion result, and the client is responded to. Implementing consistency management through a server-side driver effectively reduces the cost of consistency operations and the risk of data loss, thereby ensuring strong consistency between the database and cache. This allows the cache to respond to query requests at any time, improving data access efficiency.

[0125] Corresponding to the above method embodiments, this specification also provides data synchronization method embodiments. Figure 4 A flowchart illustrating a data synchronization method provided in one embodiment of this specification is shown. Figure 4 As shown, the method is applied to a data synchronization system, which includes a database and a cache server configured with information tables associated with the database.

[0126] Step S402: The database receives an update request submitted by the client for a database transaction processing task; in response to the update request, it filters the data stored locally and synchronizes the initial data that meets the synchronization conditions to the target cache of the cache server.

[0127] In step S404, the cache server creates a data filtering task for the initial data in the target cache; by executing the data filtering task, target data is identified and deleted from the initial data stored in the target cache; the task execution result of the database transaction processing task is determined based on the deletion result, and is used to respond to the client.

[0128] In one optional embodiment, the database parses the update request and creates a data synchronization task based on the parsing result; by executing the data synchronization task, the database determines the statement type corresponding to the update request, and filters the initial data that meets the synchronization conditions from the locally stored data according to the statement type, and synchronizes it to the target cache of the cache server.

[0129] In one optional embodiment, the database determines the target operator corresponding to the database transaction processing task based on the statement type; and calls the storage engine associated with the cache server through the target operator, and filters the initial data that meets the synchronization conditions from the data stored locally according to the set reading strategy and synchronizes it to the target cache of the cache server.

[0130] In one optional embodiment, the database reads sub-synchronization conditions and sub-detection conditions from the synchronization conditions; filters data that meets the sub-synchronization conditions from the locally stored data as data to be synchronized, and detects the data to be synchronized; selects the data to be synchronized that meets the sub-detection conditions as initial data based on the detection results, and synchronizes it to the target cache of the cache server.

[0131] In an optional embodiment, the caching server creates a first filtering task, a second filtering task, and a third filtering task for the initial data in the target cache, wherein the first filtering task, the second filtering task, and the third filtering task constitute the data filtering task; by executing the first filtering task, a first target data is determined from the initial data stored in the target cache; by executing the second filtering task, a second target data is determined from the initial data stored in the target cache; and by executing the third filtering task, a third target data is determined from the initial data stored in the target cache; the first target data, the second target data, and the third target data are then used as the target data and deleted.

[0132] In an optional embodiment, the cache server determines the primary key storage information corresponding to the initial data stored in the target cache by executing the first filtering task; if the primary key storage information is not empty, the target encapsulation library is called to determine the first modified data in the initial data stored in the target cache, which is then used as the first target data.

[0133] In an optional embodiment, when the cache server detects that a caching policy is being implemented, it calls the target encapsulation library to determine second modified data from the initial data stored in the target cache, and uses this as the first target data.

[0134] In an optional embodiment, the cache server invokes a truncation function by executing the second filtering task; upon detecting that the truncation function is running a caching strategy, it invokes the target encapsulation library to read the truncation table name; and determines the second target data from the initial data stored in the target cache based on the truncation table name.

[0135] In an optional embodiment, the cache server invokes a data update function by executing the third filtering task; upon detecting that the data update function is running a caching strategy, it invokes the target encapsulation library to read the update table name; and determines the third target data from the initial data stored in the target cache based on the update table name.

[0136] In an optional embodiment, the data synchronization system further includes the client; the client sends a target query request to the cache server; the cache server responds to the target query request by reading target data from the target cache and feeding it back to the client.

[0137] In summary, to ensure data consistency between the database and cache at all times and improve data query efficiency, after the database receives an update request submitted for a database transaction processing task, it responds by filtering out initial data that meets the synchronization conditions from the locally stored data and writing it into the target cache. This performs an initial screening of all data stored in the database, filtering out data that does not need to be synchronized. At this point, the target cache already contains some data, and the database and cache have performed consistency operations. However, considering that the initial data currently stored in the cache may still contain some redundant data that is not needed in the data query phase, the cache server can create a data filtering task for the initial data in the cache. This task filters out target data from the initial data and deletes it. The result of the database transaction processing task is then determined based on the deletion result, and the client is responded to. Implementing consistency management through a server-side driver effectively reduces the cost of consistency operations and the risk of data loss, thereby ensuring strong consistency between the database and cache. This allows the cache to respond to query requests at any time, improving data access efficiency.

[0138] The above is an illustrative scheme of a data synchronization method according to this embodiment. It should be noted that the technical solution of this data synchronization method and the technical solution of the data synchronization system described above belong to the same concept. For details not described in detail in the technical solution of the data synchronization method, please refer to the description of the technical solution of the data synchronization system described above.

[0139] Figure 5 A structural block diagram of a computing device 500 according to an embodiment of this specification is shown. The components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. The processor 520 is connected to the memory 510 via a bus 530, and a database 550 is used to store data.

[0140] The computing device 500 also includes an access device 540, which enables the computing device 500 to communicate via one or more networks 560. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 540 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0141] In one embodiment of this application, the aforementioned components of the computing device 500 and Figure 5 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 5 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art can add or replace other components as needed.

[0142] Computing device 500 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). Computing device 500 can also be a mobile or stationary server.

[0143] The processor 520 is used to execute the following computer-executable instructions:

[0144] The database receives update requests submitted by the client for database transaction processing tasks; in response to the update requests, it filters the data stored locally and synchronizes the initial data that meets the synchronization conditions to the target cache of the cache server.

[0145] The cache server creates a data filtering task for the initial data in the target cache; by executing the data filtering task, it identifies and deletes target data from the initial data stored in the target cache; and determines the task execution result of the database transaction processing task based on the deletion result, which is then used to respond to the client.

[0146] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the data synchronization method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the data synchronization method described above.

[0147] One embodiment of this specification also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, are used for:

[0148] The database receives update requests submitted by the client for database transaction processing tasks; in response to the update requests, it filters the data stored locally and synchronizes the initial data that meets the synchronization conditions to the target cache of the cache server.

[0149] The cache server creates a data filtering task for the initial data in the target cache; by executing the data filtering task, it identifies and deletes target data from the initial data stored in the target cache; and determines the task execution result of the database transaction processing task based on the deletion result, which is then used to respond to the client.

[0150] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the data synchronization method described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the data synchronization method described above.

[0151] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0152] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0153] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this specification is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this specification. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this specification.

[0154] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0155] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. These embodiments have been selected and specifically described in this specification to better explain the principles and practical applications of this specification, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data synchronization system, characterized in that, This includes a database and a cache server configured with information tables associated with the database; The database is used to receive update requests submitted by clients for database transaction processing tasks; in response to the update request, it filters the data stored locally and synchronizes the initial data that meets the synchronization conditions to the target cache of the cache server. The cache server is configured to create a data filtering task for the initial data in the target cache; determine target data and delete it from the initial data stored in the target cache by executing the data filtering task; determine the task execution result of the database transaction processing task based on the deletion result, and respond to the client, wherein the data filtering task is used to filter the data that has changed.

2. The system according to claim 1, characterized in that, The database is also used to parse the update request, create a data synchronization task based on the parsing result, determine the statement type corresponding to the update request by executing the data synchronization task, and filter the initial data that meets the synchronization conditions in the locally stored data according to the statement type to synchronize to the target cache of the cache server.

3. The system according to claim 2, characterized in that, The database is further configured to determine the target operator corresponding to the database transaction processing task based on the statement type; and to call the storage engine associated with the cache server through the target operator, and to filter the initial data that meets the synchronization conditions from the data stored locally according to the set reading strategy and synchronize it to the target cache of the cache server.

4. The system according to claim 3, characterized in that, The database is also used to read sub-synchronization conditions and sub-detection conditions from the synchronization conditions; to filter data that meets the sub-synchronization conditions from the locally stored data as data to be synchronized, and to detect the data to be synchronized; to select the data to be synchronized that meets the sub-detection conditions as initial data based on the detection results, and to synchronize it to the target cache of the cache server.

5. The system according to claim 1, characterized in that, The cache server is configured to create a first filtering task, a second filtering task, and a third filtering task for the initial data in the target cache, wherein the first filtering task, the second filtering task, and the third filtering task constitute the data filtering task; by executing the first filtering task, a first target data is determined from the initial data stored in the target cache; by executing the second filtering task, a second target data is determined from the initial data stored in the target cache; and by executing the third filtering task, a third target data is determined from the initial data stored in the target cache; the first target data, the second target data, and the third target data are then used as the target data and deleted.

6. The system according to claim 5, characterized in that, The cache server is further configured to determine the primary key storage information corresponding to the initial data stored in the target cache by executing the first filtering task; and if the primary key storage information is not empty, call the target encapsulation library to determine the first modified data in the initial data stored in the target cache as the first target data.

7. The system according to claim 6, characterized in that, The cache server is further configured to, upon detecting the execution of a caching policy, call the target encapsulation library to determine second modified data from the initial data stored in the target cache, and use this modified data as the first target data.

8. The system according to claim 5, characterized in that, The cache server is further configured to invoke a truncation function by executing the second filtering task; upon detecting that the truncation function is running a caching strategy, invoke the target encapsulation library to read the truncation table name; and determine the second target data in the initial data stored in the target cache based on the truncation table name.

9. The system according to claim 5, characterized in that, The cache server is also used to call the data update function by executing the third filtering task; and when it is detected that the data update function is running a caching strategy, it calls the target encapsulation library to read the update table name. The third target data is determined from the initial data stored in the target cache based on the updated table name.

10. The system according to any one of claims 1-9, characterized in that, The data synchronization system also includes the client; the client is used to send a target query request to the cache server; The cache server is used to respond to the target query request by reading the target data from the target cache and sending it back to the client.

11. A data synchronization method, characterized in that, It is applied to a data synchronization system, which includes a database and a cache server configured with information tables associated with the database; The database receives update requests submitted by the client for database transaction processing tasks; in response to the update requests, it filters the data stored locally and synchronizes the initial data that meets the synchronization conditions to the target cache of the cache server. The cache server creates a data filtering task for the initial data in the target cache; by executing the data filtering task, target data is identified and deleted from the initial data stored in the target cache; the task execution result of the database transaction processing task is determined based on the deletion result, and is used to respond to the client, wherein the data filtering task is used to filter the data that has changed.

12. A computing device, characterized in that, It includes a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the steps of the method of claim 11.

13. A computer-readable storage medium storing computer instructions, characterized in that, When executed by the processor, this instruction implements the steps of the method of claim 11.

Images