Data synchronization method and device

Abstract

The present invention discloses a data synchronization method and device, and relates to the field of computer technology. The specific implementation of the method includes: receiving one or more data synchronization requests, the data synchronization request indicates a target server and data to be synchronized; according to the target server indicated by the data synchronization request and the performance parameters of the target server, one or more corresponding synchronization threads are allocated to the data synchronization request; using the synchronization thread, the data to be synchronized is sent to the target server, so as to synchronize data according to the data to be synchronized. This implementation can improve the efficiency of data synchronization, reasonably allocate synchronization thread resources, and ensure the consistency of online and offline data.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data synchronization method and device. Background Art

[0002] Online consumption refers to using the Internet as a tool to enjoy various services, usually through the assistance of terminal APPs, etc. With the development of computer technology, online consumption has become a part of daily life.

[0003] Terminal APP can usually provide a list of services from multiple brands. Users can enjoy brand-exclusive services by registering as brand members. The online APP service provider and the offline brand service provider manage member data through their respective CRM systems and perform interactive data synchronization; among them, the CRM (Customer Relationship Management) system is a customer relationship management system.

[0004] Existing online CRM and offline CRM data synchronization is usually transmitted through the gateway. Due to limited resources at the gateway layer, when the performance of one brand service is poor or unavailable, it will occupy the gateway resources for a long time, causing other synchronization services to be unable to proceed, or even the gateway layer to crash, and the consistency of online and offline data cannot be guaranteed. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a data synchronization method and device, which can determine the synchronization thread of the corresponding data processing request according to the performance of the brand service, and then isolate the resources of the online synchronization request, improve the efficiency of data synchronization, reasonably allocate synchronization thread resources, and ensure the consistency of online and offline data.

[0006] The technical solution of the present invention can achieve isolation and dynamic adjustment of request resources through the judgment and scheduling of data synchronization requests by the client without relying on the gateway layer and offline server, and further achieve flow control and flow shaping in conjunction with current limiting measures. It can maximize the consistency of online system data and offline service data, improve synchronization performance, and ensure timely response of offline services.

[0007] To achieve the above object, according to one aspect of an embodiment of the present invention, a data synchronization method is provided, comprising:

[0008] Receiving one or more data synchronization requests, wherein the data synchronization request indicates a target server and data to be synchronized;

[0009] Allocate one or more corresponding synchronization threads to the data synchronization request according to the target server indicated by the data synchronization request and the performance parameters of the target server;

[0010] The synchronization thread is used to send the data to be synchronized to the target server, so as to perform data synchronization according to the data to be synchronized.

[0011] Optionally, it also includes:

[0012] grouping the one or more data synchronization requests according to a performance parameter of the target server;

[0013] According to the grouping result of the data synchronization request, the execution priority of the data synchronization request is determined, and the synchronization thread is allocated to the data synchronization request according to the execution priority.

[0014] Optionally, determining the execution priority of the data synchronization request according to the grouping result of the data synchronization request, and allocating the synchronization thread to the data synchronization request according to the execution priority includes:

[0015] Determining, according to the grouping result, a first number of the data synchronization requests corresponding to each of the execution priorities;

[0016] The synchronization threads corresponding to the data synchronization requests corresponding to different execution priorities are determined according to the first number, the second number of the one or more synchronization threads, and the status of the synchronization threads.

[0017] Optionally, for the first data synchronization request whose execution priority is higher than a preset first threshold, determining the execution priority of the data synchronization request according to the grouping result of the data synchronization request, and allocating the synchronization thread to the data synchronization request according to the execution priority includes:

[0018] Allocate an idle synchronization thread among the one or more synchronization threads to the first data synchronization request.

[0019] Optionally, for the second data synchronization request whose execution priority is lower than a preset second threshold, determining the execution priority of the data synchronization request according to the grouping result of the data synchronization request, and allocating the synchronization thread to the data synchronization request according to the execution priority includes:

[0020] Determine whether there is an unexecuted first data synchronization request, and if not, assign an idle synchronization thread among the one or more synchronization threads to the second data synchronization request.

[0021] Optionally, the performance parameters include: response time and / or response result,

[0022] Determine that the response result is an abnormal server and / or the response duration is longer than a preset response duration threshold, divide the data synchronization requests corresponding to the server into a first group, and determine that the execution priority of the first group is lower than a preset second threshold;

[0023] and / or,

[0024] The server determines that the response result is normal and the response duration is not greater than a preset response duration threshold, divides the data synchronization request corresponding to the server into a second group, and determines that the execution priority of the second group is higher than the preset first threshold.

[0025] Optionally, using the synchronization thread to send the data to be synchronized to the target server includes:

[0026] According to the type of the data to be synchronized, the data to be synchronized is sent in real time or asynchronously to a target server corresponding to the data synchronization request by using the synchronization thread.

[0027] Optionally, after sending the data to be synchronized to the target server, the method further includes:

[0028] Determine whether data synchronization is successful based on the response of the target server to the data to be synchronized, and if not, re-execute the data synchronization request corresponding to the data to be synchronized.

[0029] Optionally, when the number of the data synchronization requests received within a preset time period is greater than a preset number threshold, the method further includes:

[0030] According to the number of the data synchronization requests received within the preset time period and the number of the one or more synchronization threads, some of the multiple data synchronization requests are intercepted, and the synchronization threads are allocated to the data synchronization requests that are not intercepted.

[0031] According to another aspect of an embodiment of the present invention, there is provided a data synchronization device, comprising:

[0032] A receiving module, configured to receive one or more data synchronization requests, wherein the data synchronization request indicates a target server and data to be synchronized;

[0033] A data processing module, configured to allocate one or more corresponding synchronization threads to the data synchronization request according to the target server indicated by the data synchronization request and the performance parameters of the target server;

[0034] The synchronization module is used to send the data to be synchronized to the target server by using the synchronization thread, so as to perform data synchronization according to the data to be synchronized.

[0035] According to another aspect of an embodiment of the present invention, there is provided a data synchronization electronic device, comprising:

[0036] one or more processors;

[0037] a storage device for storing one or more programs,

[0038] When the one or more programs are executed by the one or more processors, the one or more processors implement the data synchronization method provided by the present invention.

[0039] According to another aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the data synchronization method provided by the present invention is implemented.

[0040] An embodiment of the above invention has the following advantages or beneficial effects: because it adopts a technical means to judge the data synchronization request on the client and allocate request resources to it according to the performance parameters of the server, it overcomes the technical problem that the existing online and offline data synchronization services are poor and the consistency of online and offline data cannot be guaranteed, thereby achieving the technical effect of being able to determine the synchronization thread of the corresponding data processing request according to the performance of the brand service, and then isolating resources for the online synchronization request, improving the efficiency of data synchronization, reasonably allocating synchronization thread resources, and ensuring the consistency of online and offline data.

[0041] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0043] Figure 1 An exemplary system architecture diagram of a data synchronization method or a data synchronization device suitable for application in an embodiment of the present invention is shown;

[0044] Figure 2 is a schematic diagram of the main process of the data synchronization method according to an embodiment of the present invention;

[0045] Figure 3 is a schematic diagram of the main process of a method for determining a synchronization thread of a data synchronization request according to an embodiment of the present invention;

[0046] Figure 4 is a schematic diagram of the main process of a method for determining a synchronization thread of a data synchronization request according to yet another embodiment of the present invention;

[0047] Figure 5is a schematic diagram of main modules of a data synchronization device according to an embodiment of the present invention;

[0048] Figure 6 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0050] Figure 1 An exemplary system architecture diagram of a data synchronization method or a data synchronization device suitable for use in an embodiment of the present invention is shown. Figure 1 As shown, an exemplary system architecture of a data synchronization method or a data synchronization device according to an embodiment of the present invention includes:

[0051] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0052] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to synchronize data, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0053] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0054] The server 105 may be a server that provides various services, such as a background management server that provides support for shopping websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze the received member points data synchronization request and feedback the processing result (e.g., synchronization success) to the terminal devices 101, 102, and 103.

[0055] It should be noted that the data synchronization method provided in the embodiment of the present invention is generally executed by the terminal devices 101 , 102 , 103 , and accordingly, the data synchronization apparatus is generally disposed in the terminal devices 101 , 102 , 103 .

[0056] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0057] Figure 2 is a schematic diagram of the main process of the data synchronization method according to an embodiment of the present invention, such as Figure 2 As shown, the data synchronization method of the present invention includes:

[0058] Step S201: receiving one or more data synchronization requests, wherein the data synchronization request indicates a target server and data to be synchronized.

[0059] When users enjoy brand services through terminal APP, in order to achieve effective management of user data by the offline brand server and provide better services to users to enhance user experience, it is necessary to synchronize the user data received by the online client to the offline brand server in a timely manner.

[0060] In an embodiment of the present invention, the online client receives one or more data synchronization requests, where the data synchronization request indicates the target server and the data to be synchronized. The data synchronization request can be triggered by the user in the terminal APP.

[0061] In an embodiment of the present invention, the data to be synchronized may be member data, including member card information, points information, etc. For example, member card information includes user ID, brand ID, contact number, birthday, gender, etc.; points information includes points value, points increase, points decrease, points exchange, points modification, etc. The terminal APP is provided with interfaces such as card opening, input, and modification. The user triggers the corresponding interface to operate, and the client sends a data synchronization request to the server in response to the user's trigger.

[0062] In the embodiment of the present invention, the data synchronization request may also indicate a synchronization time limit for the data to be synchronized, indicating that the synchronization of the data to be synchronized must be completed before the time limit.

[0063] Step S202: allocate one or more corresponding synchronization threads to the data synchronization request according to the target server indicated by the data synchronization request and the performance parameters of the target server.

[0064] In an embodiment of the present invention, the client interacts with the server through a gateway, and sends a data synchronization request to the server through the gateway. The gateway is provided with multiple synchronization threads for data transmission. According to the performance parameters of the target server indicated by the data synchronization request, the number of data synchronization requests, the number of synchronization threads, the state of the synchronization threads (for example, whether they are idle), etc., one or more corresponding synchronization threads are allocated to the data synchronization request.

[0065] In an embodiment of the present invention, the performance parameters of the server include the response time of the server, and / or the response result. Among them, the performance parameters of the server can be determined according to the historical response data of the server, and / or the real-time response data. For example, according to the time taken by the server to respond to the client's single point increase data synchronization request, it is determined that the response time of the server to the client's single point increase data synchronization request is 100ms. For another example, it is determined whether the server responds to the client's single point increase data synchronization request. If so, it is determined that the response result of the server to the client's single point increase data synchronization request is normal; if not, it is determined that the response result is abnormal. For another example, it is determined whether the server responds to the client's single point increase data synchronization request within a preset time (for example, 10s). If the server responds to the client within a predetermined response time threshold, it is determined that the response result of the server to the client's single point increase data synchronization request is normal; if the server does not respond to the client within the predetermined response time threshold, it is determined that the response result is abnormal.

[0066] In the embodiment of the present invention, when determining the performance parameters of the server according to the historical response data of the server, the performance of the server interface can be monitored in real time, thereby achieving real-time adjustment of the server performance parameters.

[0067] like Figure 3 As shown, an embodiment of the present invention discloses a method for determining a synchronization thread of a data synchronization request, comprising:

[0068] According to the performance parameters of the target server indicated by the data synchronization request, one or more corresponding synchronization threads are allocated to the data synchronization request in proportion. For example, according to the performance of the server represented by the performance parameters: excellent, medium, and poor, synchronization threads are allocated to the data synchronization request in a ratio of 0.5:0.3:0.2.

[0069] Step S301, grouping one or more data synchronization requests according to the performance parameters of the target server.

[0070] In the embodiment of the present invention, data synchronization requests corresponding to the target server whose response result is abnormal and / or whose response time is longer than a preset response time threshold are divided into a first group;

[0071] and / or,

[0072] The data synchronization requests corresponding to the target server whose response result is normal and whose response time is not greater than a preset response time threshold are divided into a second group.

[0073] In the embodiment of the present invention, the grouping of the data synchronization request can also be determined according to the type of the data to be processed. For example, if the data to be processed is real-time synchronization data, the corresponding data synchronization request is divided into the second group; if the data to be processed is non-real-time synchronization data, the data synchronization request is grouped according to the performance parameters of the target server within the synchronization time limit indicated by the corresponding data synchronization request.

[0074] In the embodiment of the present invention, the grouping is not limited to the first grouping and the second grouping, and the granularity of the grouping can be set as needed, for example, divided into fast grouping, slow grouping and retry grouping. For another example, divided into fast grouping, general grouping, slow grouping and retry grouping, etc. For another example, divided into the first grouping, the second grouping, ..., the Nth grouping and the retry grouping; wherein the specific value of N can be determined according to different intervals of the response time.

[0075] Step S302: determining the execution priority of the data synchronization request according to the grouping result of the data synchronization request.

[0076] In the embodiment of the present invention, according to the grouping result of the data synchronization request, the execution priority of the data synchronization request corresponding to the first group is determined to be lower than a preset second threshold;

[0077] and / or,

[0078] The execution priority of the data synchronization request corresponding to the second group is determined to be higher than a preset first threshold.

[0079] In the embodiment of the present invention, the first threshold may be greater than or equal to the second threshold, and accordingly, the execution priority of the data synchronization request corresponding to the second group is the first priority, and the execution priority of the data synchronization request corresponding to the first group is the second priority.

[0080] Step S303: determining a first number of data synchronization requests corresponding to each execution priority.

[0081] In an embodiment of the present invention, a first number of data synchronization requests corresponding to a first priority is determined, and / or a first number of data synchronization requests corresponding to a second priority is determined; wherein the first number of data synchronization requests corresponding to each execution priority may be the same or different.

[0082] Step S304, determining synchronization threads corresponding to data synchronization requests corresponding to different execution priorities according to the first number of data synchronization requests corresponding to each execution priority, the second number of one or more synchronization threads, and the state of the synchronization threads.

[0083] In the embodiment of the present invention, synchronization threads can be allocated to data synchronization requests according to the execution priority, and the higher the execution priority, the greater the ratio of the number of synchronization threads allocated to it. For example, the ratio of the number of synchronization threads allocated to the data synchronization requests corresponding to the first priority and the second priority is 0.6:0.2.

[0084] At present, online and offline data synchronization is directly transmitted through the gateway, without resource isolation for different data synchronization requests. If one or more brand servers have poor performance or are unavailable, their data transmission will always occupy gateway resources. However, gateway resources are limited, that is, the data synchronization requests that can be processed are limited, which limits the number of requests processed by the gateway, resulting in the inability to allocate synchronization threads for other data synchronization requests, and data synchronization is blocked. The offline server is unable to receive data processing requests and respond in time for a long time, which greatly affects the performance of the brand server and the efficiency of data synchronization.

[0085] By judging the data synchronization request on the client side, a larger proportion of synchronization threads are allocated to data synchronization requests with good performance, thereby isolating resources for data synchronization requests. This provides sufficient threads for requests with good performance, controls the threads allocated to requests with poor performance, and maximizes the synchronization of requests with poor performance while ensuring that good offline services are not affected. This implementation method can improve the efficiency of data synchronization, reasonably allocate synchronization thread resources, and ensure the consistency of online and offline data.

[0086] By identifying data synchronization requests on the client side and allocating a larger proportion of synchronization threads to data synchronization requests with good performance, we can control requests with poor performance and directly return request failures on the client side, thereby leaving more gateway thread resources for servers with normal performance, so that the server can respond to data synchronization requests in a timely manner and improve data synchronization efficiency.

[0087] In an embodiment of the present invention, for data synchronization requests corresponding to a server with poor performance, after rejecting some of its requests, a real-time alarm can be issued to notify the offline server to upgrade, and its performance parameters can be re-determined based on the upgraded server, and the data synchronization requests corresponding to the server can be regrouped.

[0088] In an embodiment of the present invention, the number of synchronization threads allocated to different execution priorities can be dynamically adjusted according to the first number of data synchronization requests corresponding to each execution priority, the second number of one or more synchronization threads, and the state of the synchronization threads. For example, the first number of data synchronization requests corresponding to the first priority and the second priority is 1:100, the second number of synchronization threads is 20, 16 of which are in an idle state, then 1 synchronization thread is allocated to the data synchronization request corresponding to the first priority, and the remaining 15 synchronization threads are allocated to the data synchronization request corresponding to the second priority. For another example, the first number of data synchronization requests corresponding to the first priority and the second priority is 100:80, the second number of synchronization threads is 20, 10 of which are in an idle state, then the synchronization thread is allocated to the first priority first, and 6 synchronization threads can be allocated to the data synchronization request corresponding to the first priority, and the remaining 4 synchronization threads are allocated to the data synchronization request corresponding to the second priority. For another example, the first number of data synchronization requests corresponding to the first priority and the second priority is 100:0, the second number of synchronization threads is 20, 15 of which are in an idle state, then 15 synchronization threads are allocated to the data synchronization request corresponding to the first priority. For another example, the first number of data synchronization requests corresponding to the first priority and the second priority is 0:80, the second number of synchronization threads is 20, 20 of which are idle, and 20 synchronization threads are allocated to the data synchronization requests corresponding to the second priority.

[0089] By grouping data synchronization requests according to the performance of different servers on the client side and dynamically adjusting the resources allocated for data synchronization requests, resource isolation of different brand servers can be achieved. By controlling the request resources on the client side to limit the flow of brands with poor performance, precise control of the request resources for offline brand services can be achieved, and the pressure on the gateway and offline brand services caused by the instantaneous excessive number of requests can be reduced.

[0090] In the embodiment of the present invention, for the corresponding data synchronization request with high execution priority, the gateway interface is requested in a normal request mode, and as many requests as there are are passed, and the request speed is the fastest, which is usually used to allocate request resources for normal online brand services. Correspondingly, the consumption rate of data synchronization requests is faster, and generally there will be no data backlog, and all requests can be requested at the first time.

[0091] For the corresponding data synchronization requests with low execution priority, the consumption rate is controlled through asynchronous MQ. After sending, the consumer uses the pull mode for consumption. The pull rate can be controlled. The consumption speed is adjusted through dynamic configuration. It is usually used for offline brand services with poor performance, so the consumption speed should be controlled as much as possible to ensure that too many resources are not occupied when requesting offline services.

[0092] Furthermore, for data synchronization requests with high execution priority, the consumption rate can also be controlled through asynchronous MQ. For example, membership card information of users opening a card and similar data to be synchronized that does not require high real-time performance can be processed asynchronously.

[0093] In the embodiment of the present invention, after the grouping result of the data synchronization request is determined, the consumption rate is controlled through asynchronous MQ, thereby realizing the allocation of synchronization threads.

[0094] like Figure 4 As shown, an embodiment of the present invention discloses a method for determining a synchronization thread of a data synchronization request, comprising:

[0095] According to the performance parameters of the target server indicated by the data synchronization request, the synchronization threads are allocated to the corresponding data synchronization requests with good performance first, and after the corresponding data synchronization requests with good performance are processed, the synchronization threads are allocated to the corresponding data synchronization requests with poor performance. For example, according to the performance of the server represented by the performance parameters: excellent, medium, and poor, the synchronization threads are allocated to the corresponding data synchronization requests with excellent performance first, and after the corresponding data synchronization requests with excellent performance are processed, the synchronization threads are allocated to the corresponding data synchronization requests with medium performance, and after the corresponding data synchronization requests with medium performance are processed, the synchronization threads are allocated to the corresponding data synchronization requests with poor performance.

[0096] Step S401: grouping one or more data synchronization requests according to the performance parameters of the target server.

[0097] In the embodiment of the present invention, the method for grouping data synchronization requests is the same as step S301.

[0098] Step S402: determining the execution priority of the data synchronization request according to the grouping result of the data synchronization request.

[0099] In the embodiment of the present invention, the method for determining the execution priority of the data synchronization request is the same as step S302.

[0100] Step S403: for a first data synchronization request having an execution priority higher than a preset first threshold, allocate an idle synchronization thread among the one or more synchronization threads to the first data synchronization request.

[0101] Step S404: for a second data synchronization request whose execution priority is lower than a preset second threshold, determine whether there is an unexecuted first data synchronization request; if not, assign an idle synchronization thread among one or more synchronization threads to the second data synchronization request.

[0102] In an embodiment of the present invention, if there is an unexecuted first data synchronization request, the idle synchronization thread will continue to be assigned to the first data synchronization request until the first data synchronization request is executed. After it is determined that there is no unexecuted first data synchronization request, the idle synchronization thread among one or more synchronization threads will be assigned to the second data synchronization request.

[0103] Step S203: using the synchronization thread, sending the data to be synchronized to the target server, so as to perform data synchronization according to the data to be synchronized.

[0104] In the embodiment of the present invention, after determining the synchronization thread corresponding to the data synchronization request, the synchronization thread is used to send the data to be synchronized to the target server, so as to perform data synchronization according to the data to be synchronized, and receive the synchronization result returned by the target server.

[0105] In an embodiment of the present invention, whether the data synchronization is successful is determined based on the response of the target server to the data to be synchronized (including: response time, and / or response result). If so, the performance parameters corresponding to the synchronization result are saved to facilitate real-time monitoring of the server; if not, the data synchronization request corresponding to the server that failed in synchronization is re-determined to retry the synchronization thread and re-execute the data synchronization request corresponding to the data to be synchronized.

[0106] Furthermore, when one or more synchronization threads are allocated to the data synchronization request according to the target server indicated by the data synchronization request and the performance parameters of the target server, one or more synchronization threads are reserved in advance as synchronization retry threads for retrying the data synchronization request corresponding to the server that failed to synchronize. Retry refers to setting a mechanism for re-requesting the data synchronization request request that failed to synchronize, so as to ensure the maximum synchronization success rate.

[0107] When an existing data synchronization request times out in offline response, the failed records will be retried. When a data synchronization request that failed to synchronize is retried, it usually occupies resources together with other new data synchronization requests. If the retry mechanism is incomplete and synchronization cannot be carried out in time, data backlog will be caused, resulting in long-term occupation of resources, making the synchronization performance worse and the backlog more serious.

[0108] By reserving one or more synchronization threads in advance for retried data synchronization requests and retrying the push of backlogged failed data, data consistency can be further guaranteed online and offline without affecting normal synchronization threads.

[0109] In the embodiment of the present invention, the data to be synchronized can be sent in real time or asynchronously to the target server corresponding to the data synchronization request using a synchronization thread according to the type of the data to be synchronized. For example, if the data to be processed is real-time synchronization data, the data to be synchronized is sent in real time to the target server corresponding to the data synchronization request using a synchronization thread; if the data to be processed is non-real-time synchronization data, the data to be synchronized is sent asynchronously to the target server corresponding to the data synchronization request using a synchronization thread.

[0110] In an embodiment of the present invention, when the number of data synchronization requests received within a preset time period is greater than a preset number threshold, some of the multiple data synchronization requests can be intercepted according to the number of data synchronization requests received within the preset time period and the number of one or more synchronization threads, and synchronization threads are allocated to the data synchronization requests that are not intercepted. By partially intercepting a large number of data synchronization requests in a short period of time, the pressure on the gateway and offline service end may be reduced, the synchronization efficiency may be further improved, and the consistency of online and offline data may be ensured.

[0111] In an embodiment of the present invention, by receiving one or more data synchronization requests, the data synchronization request indicates a target server and data to be synchronized; according to the target server indicated by the data synchronization request and the performance parameters of the target server, one or more corresponding synchronization threads are allocated to the data synchronization request; using the synchronization thread, the data to be synchronized is sent to the target server, so as to perform data synchronization according to the data to be synchronized, etc., the efficiency of data synchronization can be improved, the synchronization thread resources can be reasonably allocated, and the consistency of online and offline data can be ensured.

[0112] The online and offline synchronization system isolates resources through grouping, and then uses current limiting to achieve flow control; the number of requests is controlled by different consumption rates between groups; and retry groups are used to allocate resources for retry requests. This reduces the resource usage of offline brand services when there are performance issues, and does not rely on offline brand services and gateways, and can achieve resource control and scheduling of data synchronization requests through the client.

[0113] Figure 5 is a schematic diagram of main modules of a data synchronization device according to an embodiment of the present invention, such as Figure 5 As shown, the data synchronization device 500 of the present invention includes:

[0114] The receiving module 501 is used to receive one or more data synchronization requests, where the data synchronization request indicates a target server and data to be synchronized.

[0115] In the embodiment of the present invention, the receiving module 501 of the online client receives one or more data synchronization requests, where the data synchronization request indicates the target server and the data to be synchronized. The data synchronization request can be triggered by the user in the terminal APP.

[0116] In the embodiment of the present invention, the data to be synchronized may be member data, including member card information, points information, etc. For example, member card information includes user ID, brand ID, contact number, birthday, gender, etc.; points information includes points value, points increase, points decrease, points exchange, points modification, etc.

[0117] The data processing module 502 is used to allocate one or more corresponding synchronization threads to the data synchronization request according to the target server indicated by the data synchronization request and the performance parameters of the target server.

[0118] In an embodiment of the present invention, after the receiving module 501 receives a data synchronization request, the data processing module 502 allocates one or more corresponding synchronization threads to the data synchronization request based on the performance parameters of the target server indicated by the data synchronization request, the number of data synchronization requests, the number of synchronization threads, the status of the synchronization threads (for example, whether they are idle), etc.

[0119] In the embodiment of the present invention, the performance parameters of the server include the response time of the server and / or the response result. The performance parameters of the server can be determined based on the historical response data of the server and / or the real-time response data.

[0120] In the embodiment of the present invention, when determining the performance parameters of the server according to the historical response data of the server, the performance of the server interface can be monitored in real time, thereby achieving real-time adjustment of the server performance parameters.

[0121] The synchronization module 503 is used to send the data to be synchronized to the target server by using the synchronization thread, so as to perform data synchronization according to the data to be synchronized.

[0122] In an embodiment of the present invention, after the data processing module 502 determines the synchronization thread corresponding to the data synchronization request, the synchronization module 503 uses the synchronization thread to send the data to be synchronized to the target server, performs data synchronization based on the data to be synchronized, and receives the synchronization result returned by the target server.

[0123] In an embodiment of the present invention, a retry module 504 is also included, which is used to determine whether the data synchronization is successful based on the response of the target server to the data to be synchronized (including: response time, and / or response result). If not, the data synchronization request corresponding to the server that failed the synchronization determines the synchronization thread to retry again, and re-executes the data synchronization request corresponding to the data to be synchronized.

[0124] In an embodiment of the present invention, a current limiting module 505 is also included, which is used to intercept some data synchronization requests from multiple data synchronization requests based on the number of data synchronization requests received within the preset time period and the number of one or more synchronization threads when the number of data synchronization requests received within the preset time period is greater than a preset number threshold, and allocate synchronization threads to the data synchronization requests that are not intercepted through the data processing module 502.

[0125] In an embodiment of the present invention, through modules such as a receiving module, a data processing module, a synchronization module, a retry module and a current limiting module, the efficiency of data synchronization can be improved, the synchronization thread resources can be reasonably allocated, and the consistency of online and offline data can be ensured.

[0126] Figure 6 is a schematic diagram of the structure of a computer system of a terminal device suitable for implementing an embodiment of the present invention, such as Figure 6 As shown, the computer system 600 of the terminal device of the embodiment of the present invention includes:

[0127] A central processing unit (CPU) 601 can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage section 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the system 600 are also stored in the RAM 603. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0128] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0129] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-mentioned functions defined in the system of the present invention are executed.

[0130] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0131] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0132] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor. For example, they may be described as: a processor includes a receiving module, a data processing module, and a synchronization module. The names of these modules do not, in some cases, constitute limitations on the modules themselves. For example, the data processing module may also be described as a "module for determining the synchronization thread of the data to be synchronized according to the target server corresponding to the data synchronization request."

[0133] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: receiving one or more data synchronization requests, the data synchronization request indicating the target server and the data to be synchronized; allocating one or more corresponding synchronization threads to the data synchronization request according to the target server indicated by the data synchronization request and the performance parameters of the target server; using the synchronization thread, sending the data to be synchronized to the target server, so as to perform data synchronization according to the data to be synchronized.

[0134] According to the technical solution of the embodiment of the present invention, the synchronization thread of the corresponding data processing request can be determined according to the performance of the brand service, and then the resources of the online synchronization request can be isolated, the efficiency of data synchronization can be improved, the synchronization thread resources can be reasonably allocated, and the consistency of online and offline data can be ensured.

[0135] Furthermore, by judging the data synchronization request on the client side and then allocating the synchronization thread reasonably, there is no need to develop / couple business logic at the gateway layer for resource isolation, making the gateway layer function pure.

[0136] Furthermore, by limiting the flow of data synchronization requests, combined with judging the data synchronization requests on the client side, and then rationally allocating synchronization threads, dynamic adjustment of gateway resources can be achieved, which greatly optimizes the efficiency of data synchronization and promotes data consistency assurance online and offline.

[0137] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data synchronization method, characterized in that: include: Receiving one or more data synchronization requests, wherein the data synchronization request indicates a target server and data to be synchronized; Allocate one or more corresponding synchronization threads to the data synchronization request according to the target server indicated by the data synchronization request and the performance parameters of the target server; Using the synchronization thread, the data to be synchronized is sent to the target server, so as to perform data synchronization according to the data to be synchronized; Also includes: The one or more data synchronization requests are grouped according to performance parameters of the target server; the performance parameters include: response time and / or response result; According to the grouping result of the data synchronization request, the execution priority of the data synchronization request is determined, and the synchronization thread is allocated to the data synchronization request according to the execution priority.

2. The method according to claim 1, characterized in that The step of determining the execution priority of the data synchronization request according to the grouping result of the data synchronization request, and allocating the synchronization thread to the data synchronization request according to the execution priority includes: Determining, according to the grouping result, a first number of the data synchronization requests corresponding to each of the execution priorities; The synchronization threads corresponding to the data synchronization requests corresponding to different execution priorities are determined according to the first number, the second number of the one or more synchronization threads, and the status of the synchronization threads.

3. The method according to claim 1, characterized in that For the first data synchronization request whose execution priority is higher than a preset first threshold, determining the execution priority of the data synchronization request according to the grouping result of the data synchronization request, and allocating the synchronization thread to the data synchronization request according to the execution priority, includes: Allocate an idle synchronization thread among the one or more synchronization threads to the first data synchronization request.

4. The method according to claim 3, characterized in that For the second data synchronization request whose execution priority is lower than a preset second threshold, determining the execution priority of the data synchronization request according to the grouping result of the data synchronization request, and allocating the synchronization thread to the data synchronization request according to the execution priority, includes: Determine whether there is an unexecuted first data synchronization request, and if not, assign an idle synchronization thread among the one or more synchronization threads to the second data synchronization request.

5. The method according to claim 4, characterized in that Determine that the response result is an abnormal server and / or the response duration is longer than a preset response duration threshold, divide the data synchronization requests corresponding to the server into a first group, and determine that the execution priority of the first group is lower than a preset second threshold; and / or, The server determines that the response result is normal and the response duration is not greater than a preset response duration threshold, divides the data synchronization request corresponding to the server into a second group, and determines that the execution priority of the second group is higher than the preset first threshold.

6. The method according to claim 1, characterized in that The using the synchronization thread to send the data to be synchronized to the target server includes: According to the type of the data to be synchronized, the data to be synchronized is sent in real time or asynchronously to a target server corresponding to the data synchronization request by using the synchronization thread.

7. The method according to claim 1, characterized in that After sending the data to be synchronized to the target server, the method further includes: Determine whether data synchronization is successful based on the response of the target server to the data to be synchronized, and if not, re-execute the data synchronization request corresponding to the data to be synchronized.

8. The method according to claim 1, characterized in that: When the number of the data synchronization requests received within a preset time period is greater than a preset number threshold, the method further includes: According to the number of the data synchronization requests received within the preset time period and the number of the one or more synchronization threads, some of the multiple data synchronization requests are intercepted, and the synchronization threads are allocated to the data synchronization requests that are not intercepted.

9. A data synchronization device, characterized in that: include: A receiving module, configured to receive one or more data synchronization requests, wherein the data synchronization request indicates a target server and data to be synchronized; A data processing module, configured to allocate one or more corresponding synchronization threads to the data synchronization request according to the target server indicated by the data synchronization request and the performance parameters of the target server; A synchronization module, used to send the data to be synchronized to the target server by using the synchronization thread, so as to perform data synchronization according to the data to be synchronized; The data processing module is also used for: The one or more data synchronization requests are grouped according to performance parameters of the target server; the performance parameters include: response time and / or response result; According to the grouping result of the data synchronization request, the execution priority of the data synchronization request is determined, and the synchronization thread is allocated to the data synchronization request according to the execution priority.

10. A data synchronization electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

11. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images