Data synchronization method and device

By asynchronously serializing the update and read operations of database and cache in high concurrency scenarios, the problem of inconsistency between cache and database data is solved and the user experience is improved.

CN113986981BActive Publication Date: 2025-02-07HUNAN HAPPLY SUNSHINE INTERACTIVE ENTERTAINMENT MEDIA CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202111334163.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-11
Publication Date
2025-02-07
Estimated Expiration
2041-11-11

AI Technical Summary

Technical Problem

In high concurrency scenarios, the inconsistency between cache and database data leads to poor user experience, and the existing technology is difficult to effectively solve this problem.

Method used

Through asynchronous serialization of database and cache update and read operations, we ensure that data requests are stored and processed synchronously in the request routing of multi-service instances deployed by the application server to avoid inconsistencies between cache and database data.

Benefits of technology

In high concurrency scenarios, the inconsistency between cache and database data is effectively avoided, the user experience is improved, and the problem of poor user experience is solved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113986981B_ABST
    Figure CN113986981B_ABST
Patent Text Reader

Abstract

The present application provides a data synchronization method and device, which requires the deployment of request routing for multiple service instances on an application server, receiving data requests (read data requests or update data requests) sent by users; if it is a read data request, a first queue is generated (the task of refreshing the cache is routed to the queue for refreshing the cache); the corresponding data is first obtained from the cache according to the first preset rule, and when the data in the cache expires or is cleared, the corresponding data is obtained from the database and asynchronously stored in the cache; if it is a request to update data, a second queue is generated (the request to update the database is routed to the queue for updating the database); the data in the cache and the database are synchronized according to the second preset rule. Without introducing new middleware, the present application solves the problem of poor user experience caused by the inconsistency between cache and database data in high concurrency scenarios by asynchronously serializing database and cache update and read operations.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of Internet data processing technology, and in particular, to a data synchronization method and device. Background Art

[0002] As Internet users continue to grow, the performance bottleneck of relational databases has become more prominent. Therefore, cache has become an important middleware to overcome this problem. However, cache is mostly based on memory, which is expensive. Therefore, it is impossible to store the entire database data in the cache. When the database changes and is synchronized to the cache, the database is modified first and then the cache is updated. If the cache update fails, the database will contain new data while the cache will contain old data. This can easily cause inconsistency between the cache and database data, affecting user experience and even causing financial losses to the company.

[0003] In order to avoid inconsistencies between cache and database data, the current common processing method is to delete the cache first and then modify the database. If the cache is deleted successfully and the database modification fails, the old data is still stored in the database, but the cache is empty. Therefore, when reading data, if there is no data in the cache, the old data in the database is read and then updated to the cache, thus avoiding inconsistencies between cache and database data. However, in high-concurrency scenarios, database data changes are synchronized to the cache, which still cannot avoid the poor user experience caused by inconsistencies between cache and database data. Summary of the invention

[0004] In view of the problems in the above content, the present application provides a data synchronization method and device, which solves the problem of poor user experience caused by inconsistency between cache and database data in high concurrency scenarios by asynchronously serializing database and cache update and read operations.

[0005] In order to achieve the above objectives, this application provides the following technical solutions:

[0006] A data synchronization method deploys request routing of multiple service instances on an application server, the method comprising:

[0007] Receiving a data request sent by a user, wherein the data request is a request to read data or a request to update data;

[0008] If the data request is a request to read data, a first queue is generated, wherein the first queue is a queue to which the task of refreshing the cache is routed to refresh the cache;

[0009] Firstly, the corresponding data is obtained from the cache according to a first preset rule, and when the data in the cache is expired or cleared, the corresponding data is obtained from the database and asynchronously stored in the cache;

[0010] If the data request is a request to update data, a second queue is generated, the second queue being a queue for routing requests for updating the database to update the database;

[0011] The data in the cache and the data in the database are synchronized according to a second preset rule.

[0012] Furthermore, after receiving the data request sent by the user, the method further includes:

[0013] According to the unique identifier of the data, the data request is sent to a preset queue of the Java virtual machine, each preset queue corresponds to a working thread, and each working thread processes the corresponding operation in series.

[0014] Furthermore, it also includes:

[0015] The data requests in the preset queue are deduplicated to ensure that the preset queue contains only one data request for updating the cache.

[0016] Furthermore, it also includes:

[0017] Determine whether the data request is within a preset waiting time range, and if so, poll to obtain the corresponding data;

[0018] If not, the current value in the database is read.

[0019] Furthermore, synchronizing the data in the cache and the data in the database according to the second preset rule includes:

[0020] If the task in the preset queue is a task to update the database, the update status is set to pending update, and a new cache refresh task in the preset queue will refresh the cache, delete the data in the cache accordingly, and update the database;

[0021] If the task in the preset queue is a task for refreshing the cache, then if the update state is set, it is initialized to updated, and the database is queried accordingly, and the data in the cache is updated;

[0022] If the update status is pending update, the status is set to updated, and the database is queried accordingly, and the data in the cache is updated;

[0023] If the update status is updated, it means that the cache has been updated or a task is being executed, and the synchronization of the data in the cache and the database is ended.

[0024] A data synchronization device deploys request routing of multiple service instances on an application server, the device comprising:

[0025] A first processing unit, configured to receive a data request sent by a user, wherein the data request is a request to read data or a request to update data;

[0026] A second processing unit is configured to generate a first queue if the data request is a request to read data, wherein the first queue is a queue to which a task to refresh the cache is routed to refresh the cache;

[0027] A third processing unit is used to first obtain the corresponding data from the cache according to a first preset rule, and when the data in the cache expires or is cleared, obtain the corresponding data from the database and asynchronously store it in the cache;

[0028] a fourth processing unit, configured to generate a second queue if the data request is a request to update the data, the second queue being a queue for routing requests for updating the database to update the database;

[0029] The fifth processing unit is used to synchronize the data in the cache and the data in the database according to a second preset rule.

[0030] Furthermore, the first processing unit is further configured to:

[0031] According to the unique identifier of the data, the data request is sent to a preset queue of the Java virtual machine, each preset queue corresponds to a working thread, and each working thread processes the corresponding operation in series;

[0032] Deduplication of the data requests in the preset queue is performed to ensure that the preset queue contains only one data request for updating the cache;

[0033] Determine whether the data request is within a preset waiting time range, and if so, poll to obtain the corresponding data;

[0034] If not, the current value in the database is read.

[0035] Furthermore, the fifth processing unit is specifically configured to:

[0036] If the task in the preset queue is a task to update the database, the update status is set to pending update, and a new cache refresh task in the preset queue will refresh the cache, delete the data in the cache accordingly, and update the database;

[0037] If the task in the preset queue is a task for refreshing the cache, then if the update state is set, it is initialized to updated, and the database is queried accordingly, and the data in the cache is updated;

[0038] If the update status is pending update, the status is set to updated, and the database is queried accordingly, and the data in the cache is updated;

[0039] If the update status is updated, it means that the cache has been updated or a task is being executed, and the synchronization of the data in the cache and the database is ended.

[0040] A storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the data synchronization method as described above.

[0041] An electronic device, comprising at least one processor, and at least one memory and a bus connected to the processor; wherein the processor and the memory communicate with each other via the bus; and the processor is used to call program instructions in the memory to execute the data synchronization method as described above.

[0042] The data synchronization method and device described in the present application require the deployment of request routing for multiple service instances on the application server, and receive data requests sent by users, wherein the data request is a request to read data or a request to update data; if the data request is a request to read data, a first queue is generated, and the first queue is a queue for refreshing cache tasks and is routed to the queue for refreshing cache; the corresponding data is first obtained from the cache according to a first preset rule, and when the data in the cache expires or is cleared, the corresponding data is obtained from the database and asynchronously stored in the cache; if the data request is a request to update data, a second queue is generated, and the second queue is a queue for updating the database and is routed to the queue for updating the database; data synchronization is performed on the cache and the data in the database according to the second preset rule. Without introducing new middleware, the present application solves the problem of poor user experience caused by inconsistency between cache and database data in high concurrency scenarios by asynchronously serializing database and cache update and read operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 A flowchart of a data synchronization method disclosed in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of the structure of a data synchronization device disclosed in an embodiment of the present application;

[0046] Figure 3A schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0048] Please see attached Figure 1 , is a flow chart of a data synchronization method provided in an embodiment of the present application. Figure 1 As shown, an embodiment of the present application provides a data synchronization method, which deploys request routing of multiple service instances on an application server, and the method includes the following steps:

[0049] S101: receiving a data request sent by a user, where the data request is a request to read data or a request to update data;

[0050] S102: If the data request is a request to read data, a first queue is generated, where the first queue is a queue to which a task to refresh the cache is routed;

[0051] S103: first obtaining corresponding data from the cache according to a first preset rule, and when the data in the cache expires or is cleared, obtaining corresponding data from a database and asynchronously storing it in the cache;

[0052] S104: If the data request is a request for updating data, a second queue is generated, where the second queue is a queue for routing requests for updating the database to update the database;

[0053] S105: Synchronize the data in the cache and the data in the database according to a second preset rule.

[0054] Furthermore, after receiving the data request sent by the user, the method further includes:

[0055] According to the unique identifier of the data, the data request is sent to a preset queue of the Java virtual machine, each preset queue corresponds to a working thread, and each working thread processes the corresponding operation in series.

[0056] It should be noted that one queue corresponds to one working thread, and each working thread gets the corresponding operations serially and then executes them one by one.

[0057] In the embodiment of the present application, when updating data, the operation is routed according to the unique identifier of the data and sent to a queue of a Java virtual machine. When reading data, if the data is found not in the cache, the operation of re-reading the data and updating the cache is routed according to the unique identifier and sent to the queue of the same Java virtual machine.

[0058] The specific implementation effect of the embodiment of the present application is as follows: for a data change operation, the cache is deleted first, and then the database is updated, but the update has not been completed. At this time, if a read request comes and reads an empty cache, the cache update request can be sent to the queue first. At this time, the read request will be backlogged in the queue, and then synchronously wait for the cache update to be completed.

[0059] It should be further explained that it is meaningless to string multiple cache update requests together in one queue. Therefore, when filtering, if it is found that there is already a cache update request in the queue, then there is no need to put another update request operation in, and just wait for the previous update operation request to be completed. After the worker thread corresponding to the queue completes the database modification of the previous operation, it will execute the next operation, that is, the cache update operation. At this time, the latest value will be read from the database and then written into the cache. Specifically, the method also includes:

[0060] The data requests in the preset queue are deduplicated to ensure that the preset queue contains only one data request for updating the cache.

[0061] It should be further explained that if the request is still within the waiting time range and the value can be obtained through continuous polling, it will be returned directly; if the waiting time of the request exceeds a certain period of time, the current old value will be directly read from the database this time. Specifically, the method also includes:

[0062] Determine whether the data request is within a preset waiting time range, and if so, poll to obtain the corresponding data;

[0063] If not, the current value in the database is read.

[0064] It should be further explained that the above-mentioned data synchronization of the cache and the data in the database according to the second preset rule includes: if the task in the preset queue is a task to update the database, the update status is set to pending update, and a new cache refresh task in the preset queue will refresh the cache, delete the data in the cache accordingly, and update the database; if the task in the preset queue is a task to refresh the cache, then if the update status is set, it is initialized to updated, and the database is queried accordingly, and the data in the cache is updated; if the update status is pending update, the status is set to updated, and the database is queried accordingly, and the data in the cache is updated; if the update status is updated, it means that the cache has been updated or a task is being executed, and the synchronization of the data in the cache and the database is ended.

[0065] In order to facilitate the understanding of the technical solution, the embodiment of the present application takes the commodity inventory as an example, and the user can browse the commodity (query inventory) or purchase the commodity (update inventory). Specifically, for a write operation (update inventory), the commodity inventory status is set to not updated, and a write database request is constructed and placed in an asynchronous queue. The request will first delete the cache and then update the database; for a read operation (query inventory), an update cache request is constructed and placed in an asynchronous queue. The request will asynchronously decide whether to refresh the cache based on the updated status of the commodity inventory, read the inventory from the cache and return it.

[0066] The preset columns in the embodiment of the present application realize the serial execution of read and write requests, ensuring the consistency of cache values ​​with database data. It should be noted that the read request type: route to the specified queue according to the product id; obtain the update status of the product inventory from the queue; if the update status is not initialized, initialize it to pending update, and perform a cache refresh operation; if the update status is pending update, perform a cache refresh operation; if the update status is updated, it means that a refresh request has been queued before, and it returns directly without any operation. Write request type: delete the cache first, then update the database.

[0067] In the embodiment of the present application, data consistency is ensured from the following points. First, the read operation is abnormal: when an exception occurs in the update cache request of the asynchronous queue: the product inventory update status will be reset to not updated, and an attempt will be made to read the cache. If the cache is read, it means that the new read request has re-triggered the read request queue and updated the cache. The cache at this time is the latest value; secondly, the write operation is abnormal. When an exception occurs in the write database request of the asynchronous queue, the update status will be reset to not updated, and the read request behind the queue will be required to re-read the database and update it to the cache.

[0068] It should be noted that in extreme cases, corresponding processing strategies need to be determined:

[0069] First, avoid long-term blocking of read requests. What scenarios will cause read request blocking: a large number of write requests: write requests themselves are relatively time-consuming; uneven distribution of write requests: in cluster mode, if all write requests are concentrated on one machine, then the read requests of this machine will be blocked due to the large number of write requests. Corresponding solution: This solution is not used in business scenarios with more writes and fewer reads. Each read request must return within the timeout range when a timeout is set. The cluster mode distributes the pressure of write requests.

[0070] Secondly, the concurrent read request volume is too high. When the read request volume increases suddenly during the business peak period, too many write requests will affect the response time of the interface. The corresponding solution is to prevent the cache from expiring at the same time and causing cache avalanche.

[0071] Again, when write requests are concentrated, read performance is affected. The corresponding solution is: request routing for multi-service instance deployment: ensure that requests for data update operations and cache update operations are routed to the same service instance through the nginx server, so that write requests are distributed as evenly as possible.

[0072] Finally, hot data causes request skew. When the read and write requests for a certain data are very high, all of them are sent to the same queue of the same machine, which may cause excessive pressure on a certain machine. The corresponding solution is to clear the cache only when the data is updated, which will cause read and write concurrency. Therefore, if the update frequency is not high, the impact of this problem is not significant.

[0073] The embodiment of the present application provides a data synchronization method, which requires the deployment of request routing for multiple service instances on an application server, and receives data requests sent by users, wherein the data request is a request to read data or a request to update data; if the data request is a request to read data, a first queue is generated, and the first queue is a queue for refreshing cache tasks and is routed to the queue for refreshing cache; the corresponding data is first obtained from the cache according to a first preset rule, and when the data in the cache expires or is cleared, the corresponding data is obtained from the database and asynchronously stored in the cache; if the data request is a request to update data, a second queue is generated, and the second queue is a queue for updating the database and is routed to the queue for updating the database; data synchronization is performed on the cache and the data in the database according to the second preset rule. Without introducing new middleware, the embodiment of the present application solves the problem of poor user experience caused by the inconsistency between cache and database data in high concurrency scenarios by asynchronously serializing database and cache update and read operations.

[0074] See also Figure 2 Based on a data synchronization method disclosed in the above embodiment, this embodiment correspondingly discloses a data synchronization device, which deploys request routing for multiple service instances on an application server, and the device includes:

[0075] The first processing unit 201 is used to receive a data request sent by a user, where the data request is a read data request or an update data request;

[0076] The second processing unit 202 is configured to generate a first queue if the data request is a request to read data, wherein the first queue is a queue to which the task of refreshing the cache is routed to refresh the cache;

[0077] The third processing unit 203 is used to first obtain the corresponding data from the cache according to the first preset rule, and when the data in the cache expires or is cleared, obtain the corresponding data from the database and asynchronously store it in the cache;

[0078] The fourth processing unit 204 is configured to generate a second queue if the data request is a request for updating data, wherein the second queue is a queue for routing requests for updating the database to update the database;

[0079] The fifth processing unit 205 is used to synchronize the data in the cache and the data in the database according to a second preset rule.

[0080] Furthermore, the first processing unit 201 is further configured to:

[0081] According to the unique identifier of the data, the data request is sent to a preset queue of the Java virtual machine, each preset queue corresponds to a working thread, and each working thread processes the corresponding operation in series;

[0082] Deduplication of the data requests in the preset queue is performed to ensure that the preset queue contains only one data request for updating the cache;

[0083] Determine whether the data request is within a preset waiting time range, and if so, poll to obtain the corresponding data;

[0084] If not, the current value in the database is read.

[0085] Furthermore, the fifth processing unit 205 is specifically configured to:

[0086] If the task in the preset queue is a task to update the database, the update status is set to pending update, and a new cache refresh task in the preset queue will refresh the cache, delete the data in the cache accordingly, and update the database;

[0087] If the task in the preset queue is a task for refreshing the cache, then if the update state is set, it is initialized to updated, and the database is queried accordingly, and the data in the cache is updated;

[0088] If the update status is pending update, the status is set to updated, and the database is queried accordingly, and the data in the cache is updated;

[0089] If the update status is updated, it means that the cache has been updated or a task is being executed, and the synchronization of the data in the cache and the database is ended.

[0090] The data synchronization device includes a processor and a memory. The first processing unit, the second processing unit and the third processing unit are all stored in the memory as program units. The processor executes the program units stored in the memory to implement corresponding functions.

[0091] The processor contains a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and the kernel parameters can be adjusted to solve the problem of poor user experience caused by inconsistency between cache and database data in high concurrency scenarios.

[0092] An embodiment of the present application provides a storage medium on which a program is stored, and when the program is executed by a processor, the data synchronization method is implemented.

[0093] An embodiment of the present application provides a processor, which is used to run a program, wherein the data synchronization method is executed when the program is running.

[0094] The present application embodiment provides an electronic device, such as Figure 3 As shown, the electronic device 30 includes at least one processor 301, and at least one memory 302 and a bus 303 connected to the processor; wherein the processor 301 and the memory 302 communicate with each other through the bus 303; the processor 301 is used to call the program instructions in the memory 302 to execute the above-mentioned data synchronization method.

[0095] The electronic devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0096] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing the following method steps:

[0097] Receiving a data request sent by a user, wherein the data request is a request to read data or a request to update data;

[0098] If the data request is a request to read data, a first queue is generated, wherein the first queue is a queue to which the task of refreshing the cache is routed to refresh the cache;

[0099] Firstly, the corresponding data is obtained from the cache according to a first preset rule, and when the data in the cache is expired or cleared, the corresponding data is obtained from the database and asynchronously stored in the cache;

[0100] If the data request is a request to update data, a second queue is generated, the second queue being a queue for routing requests for updating the database to update the database;

[0101] The data in the cache and the data in the database are synchronized according to a second preset rule.

[0102] Furthermore, after receiving the data request sent by the user, the method further includes:

[0103] According to the unique identifier of the data, the data request is sent to a preset queue of the Java virtual machine, each preset queue corresponds to a working thread, and each working thread processes the corresponding operation in series.

[0104] Furthermore, it also includes:

[0105] The data requests in the preset queue are deduplicated to ensure that the preset queue contains only one data request for updating the cache.

[0106] Furthermore, it also includes:

[0107] Determine whether the data request is within a preset waiting time range, and if so, poll to obtain the corresponding data;

[0108] If not, the current value in the database is read.

[0109] Furthermore, synchronizing the data in the cache and the data in the database according to the second preset rule includes:

[0110] If the task in the preset queue is a task to update the database, the update status is set to pending update, and a new cache refresh task in the preset queue will refresh the cache, delete the data in the cache accordingly, and update the database;

[0111] If the task in the preset queue is a task for refreshing the cache, then if the update state is set, it is initialized to updated, and the database is queried accordingly, and the data in the cache is updated;

[0112] If the update status is pending update, the status is set to updated, and the database is queried accordingly, and the data in the cache is updated;

[0113] If the update status is updated, it means that the cache has been updated or a task is being executed, and the synchronization of the data in the cache and the database is ended.

[0114] The present application is described by flowcharts and / or block diagrams of the methods, devices (systems), and computer program products of the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0115] In a typical configuration, the device includes one or more processors (CPU), memory and bus. The device may also include input / output interface, network interface and the like.

[0116] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip. The memory is an example of a computer-readable medium.

[0117] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0118] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0119] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0120] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A data synchronization method, characterized in that: Deploy request routing for multiple service instances on an application server, the method comprising: Receiving a data request sent by a user, wherein the data request includes a read data request and an update data request; If the data request is a request to read data, a first queue is generated, wherein the first queue is a queue to which the task of refreshing the cache is routed to refresh the cache; Firstly, the corresponding data is obtained from the cache according to a first preset rule, and when the data in the cache is expired or cleared, the corresponding data is obtained from the database and asynchronously stored in the cache; If the data request is a request to update data, a second queue is generated, the second queue being a queue for routing requests for updating the database to update the database; Synchronizing the data in the cache and the data in the database according to a second preset rule; Wherein, after receiving the data request sent by the user, the method further includes: According to the unique identifier of the data, the data request is sent to a preset queue of the Java virtual machine, each preset queue corresponds to a working thread, and each working thread processes the corresponding operation in series; When updating data, the operation is routed according to the unique identifier of the data and sent to the queue of the Java virtual machine; when reading data, if the data is found not in the cache, the data is re-read and the cache is updated, and the operation is routed according to the unique identifier and sent to the queue of the Java virtual machine; The step of synchronizing the data in the cache and the data in the database according to the second preset rule includes: If the task in the preset queue is a task to update the database, the update status is set to pending update, and a new cache refresh task in the preset queue will refresh the cache, delete the data in the cache accordingly, and update the database; If the task in the preset queue is a task for refreshing the cache, then if the update state is set, it is initialized to updated, and the database is queried accordingly, and the data in the cache is updated; If the update status is pending update, the status is set to updated, and the database is queried accordingly, and the data in the cache is updated; If the update status is updated, it means that the cache has been updated or a task is being executed, and the synchronization of the data in the cache and the database is ended.

2. The method according to claim 1, characterized in that: Also includes: The data requests in the preset queue are deduplicated to ensure that the preset queue contains only one data request for updating the cache.

3. The method according to claim 1, characterized in that Also includes: Determine whether the data request is within a preset waiting time range, and if so, poll to obtain the corresponding data; If not, the current value in the database is read.

4. A data synchronization device, characterized in that: Deploy request routing for multiple service instances on an application server, the device comprising: A first processing unit, configured to receive a data request sent by a user, wherein the data request includes a read data request and an update data request; A second processing unit is configured to generate a first queue if the data request is a request to read data, wherein the first queue is a queue to which a task to refresh the cache is routed to refresh the cache; A third processing unit is used to first obtain the corresponding data from the cache according to a first preset rule, and when the data in the cache expires or is cleared, obtain the corresponding data from the database and asynchronously store it in the cache; a fourth processing unit, configured to generate a second queue if the data request is a request to update the data, the second queue being a queue for routing requests for updating the database to update the database; A fifth processing unit, configured to synchronize the data in the cache and the data in the database according to a second preset rule; Wherein, the first processing unit is further used for: According to the unique identifier of the data, the data request is sent to a preset queue of the Java virtual machine, each preset queue corresponds to a working thread, and each working thread processes the corresponding operation in series; When updating data, the operation is routed according to the unique identifier of the data and sent to the queue of the Java virtual machine; when reading data, if the data is found not in the cache, the data is re-read and the cache is updated, and the operation is routed according to the unique identifier and sent to the queue of the Java virtual machine; The fifth processing unit is specifically configured to: If the task in the preset queue is a task to update the database, the update status is set to pending update, and a new cache refresh task in the preset queue will refresh the cache, delete the data in the cache accordingly, and update the database; If the task in the preset queue is a task for refreshing the cache, then if the update state is set, it is initialized to updated, and the database is queried accordingly, and the data in the cache is updated; If the update status is pending update, the status is set to updated, and the database is queried accordingly, and the data in the cache is updated; If the update status is updated, it means that the cache has been updated or a task is being executed, and the synchronization of the data in the cache and the database is ended.

5. The device according to claim 4, characterized in that The first processing unit is further configured to: Deduplication of the data requests in the preset queue is performed to ensure that the preset queue contains only one data request for updating the cache; Determine whether the data request is within a preset waiting time range, and if so, poll to obtain the corresponding data; If not, the current value in the database is read.

6. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the data synchronization method according to any one of claims 1 to 3.

7. An electronic device, characterized in that: The electronic device includes at least one processor, and at least one memory and a bus connected to the processor; wherein the processor and the memory communicate with each other via the bus; and the processor is used to call program instructions in the memory to execute the data synchronization method as described in any one of claims 1 to 3.

Citation Information

Patent Citations

  • Data processing method and device, electronic equipment and storage medium

    CN111414389A

  • Cache database and database synchronization method, device and terminal

    CN113420088A