Data synchronization method and system, server and client

Abstract

The invention discloses a data synchronization method and system, a server and a client, and the method comprises the steps: based on a category total data request of the client, sending the total data of a corresponding category to a corresponding request client in an IP unicast mode, so that the request client stores the total data; and when the incremental data event is monitored, acquiring an IP multicast address of a multicast client corresponding to the incremental data event, and sending incremental data corresponding to the incremental data event to the multicast client in a multicast mode according to the IP multicast address, so that the multicast client synchronously updates a local database according to the incremental data. The method only relates to the server and the client, does not need to depend on an extra central node, avoids complexity of system architecture and extra node resource overhead, the incremental data is actively pushed by the server, and the IP multicast mode is adopted to ensure rapid transmission of the incremental data and realize data real-time performance at the same time.

Description

Technical Field

[0001] The present application belongs to the field of database technology, and in particular, relates to a data synchronization method, system, server, and client. Background Art

[0002] Currently, the industry generally adopts the following four technical solutions for data synchronization of unified authentication systems:

[0003] A. Based on file server: As shown in the system architecture of Figure 1(a), the unified authentication system (also called the unified authentication server) periodically generates files containing full or incremental data and uploads the files to a distributed file server. Each subsystem periodically synchronizes data files and processes them into the local database.

[0004] B. Based on message middleware: As shown in the system architecture of Figure 1(b), the unified authentication system publishes messages through message middleware (such as Kafka, RabbitMQ, etc.), and each subsystem consumes these messages through the subscriber model and processes them into the local database.

[0005] C. Based on database replication: As shown in the system architecture of Figure 1(c), the database of the unified authentication system is synchronized to the local database of each subsystem directly through database replication or database synchronization tools (such as GoldenGate, Canal, Otter, etc.).

[0006] D. Based on API calls: As shown in the system architecture of Figure 1(d), the unified authentication system exposes the API interface, and each subsystem directly obtains the data of the unified authentication system through API calls and processes it into the local database.

[0007] However, these technical solutions have the following defects: Solutions A, B, and C have the risk of central node dependence and are strongly dependent on central nodes such as file servers / message middleware / database synchronization tools. If the central nodes fail, data synchronization cannot be performed. The introduction of central nodes such as file servers / message middleware / database synchronization tools also increases the complexity of the system and requires additional development and maintenance costs. Solutions A and D use the method of client-side periodic data pulling, which has poor real-time performance.

[0008] Therefore, how to design a technical solution that does not rely on a central node while ensuring real-time data synchronization between the unified authentication server and the client is an urgent problem to be solved. Summary of the invention

[0009] The present application aims to solve at least one of the technical problems existing in the related art. To this end, the present application proposes a data synchronization method, system, server and client, which neither rely on a central node nor ensure the real-time performance of data synchronization.

[0010] In a first aspect, the present application provides a data synchronization method, the method comprising:

[0011] Based on the client's request for full data of a category, the full data of the corresponding category is sent to the corresponding requesting client in an IP unicast manner, so that the requesting client stores the full data;

[0012] When an incremental data event is monitored, the IP multicast address of the multicast client corresponding to the incremental data event is obtained, and the incremental data corresponding to the incremental data event is sent to the multicast client in a multicast manner according to the IP multicast address, so that the multicast client synchronously updates the local database according to the incremental data.

[0013] In a second aspect, the present application provides a data synchronization method, the method comprising:

[0014] Sending a category full data request to a server, and receiving and storing full data of a corresponding category sent by the server in an IP unicast manner based on the category full data request;

[0015] When the existence of incremental data of the corresponding category is monitored, the incremental data is received and the local database is synchronously updated according to the incremental data, wherein the incremental data is the data of changes in the corresponding category sent by the server in a multicast manner when the incremental data event of the corresponding category is monitored.

[0016] In a third aspect, the present application provides a server, including:

[0017] A first full data module is used to send the full data of the corresponding category to the corresponding requesting client in an IP unicast manner based on the full data request of the category by the client, so that the requesting client stores the full data;

[0018] The first incremental data module is used to obtain the IP multicast address of the multicast client corresponding to the incremental data event when monitoring the occurrence of an incremental data event, and send the incremental data corresponding to the incremental data event to the multicast client in a multicast manner according to the IP multicast address, so that the multicast client updates the local database according to the incremental data.

[0019] In a fourth aspect, the present application provides a client, including:

[0020] A second full data module is used to send a category full data request to the server, and receive and store the full data of the corresponding category sent by the server in an IP unicast manner based on the category full data request;

[0021] The second incremental data module is used to receive the incremental data and update the local database according to the incremental data when it is monitored that there is incremental data of the corresponding category, wherein the incremental data is sent by the server in a multicast manner when it is monitored that an incremental data event occurs in the corresponding category.

[0022] In a fifth aspect, the present application provides a data synchronization system, including a server and a client;

[0023] The client is used to send a category full data request to the server;

[0024] The server is used to send the full data of the corresponding category to the corresponding client in an IP unicast manner based on the full data request of the category by the client;

[0025] The client is used to receive and store the full amount of data of the corresponding category;

[0026] The server is further configured to obtain the IP multicast address of the multicast client corresponding to the incremental data event when monitoring the occurrence of the incremental data event, and send the incremental data corresponding to the incremental data event in a multicast manner according to the IP multicast address;

[0027] The client is also used to receive the incremental data and update the local database according to the incremental data when it is monitored that there is incremental data of a corresponding category in the incremental data sent by the server.

[0028] In a sixth aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the data synchronization method as described in the first or second aspect above is implemented.

[0029] In a seventh aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the data synchronization method as described in the first aspect above is implemented.

[0030] In an eighth aspect, the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the data synchronization method as described in the first aspect.

[0031] In a ninth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the data synchronization method as described in the first aspect above.

[0032] The above one or more technical solutions in the embodiments of the present application have at least the following technical effects:

[0033] The data synchronization method, system, server and client provided in the embodiments of the present application only involve the server and client in the solution, and IP unicast and IP multicast are simple and efficient network communication methods, which do not require complex configuration and do not rely on additional central nodes, avoiding the complexity of the system architecture and additional node resource overhead. In addition, IP unicast directly transmits data through a clear target address, and the configuration and maintenance operations are simple and intuitive. IP multicast realizes multi-point transmission of data through multicast addresses, and management, maintenance and expansion are more convenient. At the same time, the incremental data in the present application is actively pushed by the server. The use of IP multicast can ensure the rapid transmission of incremental data while realizing real-time data, ensuring a rapid system response and a smooth user experience.

[0034] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0036] FIG. 1( a ) is a schematic diagram of a data synchronization system architecture based on a file server provided in the prior art;

[0037] FIG. 1( b ) is a schematic diagram of a data synchronization system architecture based on a message middleware provided in the prior art;

[0038] FIG. 1( c ) is a schematic diagram of a data synchronization system architecture based on database replication provided in the prior art;

[0039] FIG1( d ) is a schematic diagram of a data synchronization system architecture based on API calls provided in the prior art;

[0040] Figure 2 It is a flowchart of a data synchronization method on a server provided in an embodiment of the present application;

[0041] Figure 3 It is a schematic diagram of the system architecture when the full data is updated provided in an embodiment of the present application;

[0042] Figure 4 is a schematic diagram of the system architecture for incremental data update provided in an embodiment of the present application;

[0043] Figure 5 It is a flowchart of a method for synchronizing data on a client provided in an embodiment of the present application;

[0044] Figure 6 It is a schematic diagram of the structure of the server provided in the embodiment of the present application;

[0045] Figure 7 It is a schematic diagram of the structure of the client provided in the embodiment of the present application;

[0046] Figure 8 It is a schematic diagram of the architecture of the data synchronization system provided in an embodiment of the present application;

[0047] Fig. 9 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0049] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0050] The data synchronization method, server, client, electronic device and readable storage medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0051] The data synchronization method may be applied to a terminal, and may be specifically executed by hardware or software in the terminal.

[0052] The data synchronization method provided in the embodiment of the present application may be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the data synchronization method. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, and wearable devices. The data synchronization method provided in the embodiment of the present application is described below using an electronic device as an example of the execution subject.

[0053] like Figure 2 As shown, a data synchronization method provided in an embodiment of the present application is applied to a server, and the method comprises the following steps:

[0054] S110, based on the client's request for full data of a category, sending full data of a corresponding category to a corresponding requesting client in an IP unicast manner, so that the requesting client stores the full data.

[0055] S120, when an incremental data event is monitored, obtain the IP multicast address of the multicast client corresponding to the incremental data event, and send the incremental data corresponding to the incremental data event to the multicast client in a multicast manner according to the IP multicast address, so that the multicast client synchronously updates the local database according to the incremental data.

[0056] It should be noted that the data in the server in the present application is stored in a classified storage manner. In some embodiments, the data categories stored in the server include organization categories, user categories, and role categories. It can be understood that the above data categories are only examples and do not mean that the server only contains data of these categories. In some embodiments, incremental data events include addition events, deletion events, and modification events. These incremental data events can be events for the aforementioned data categories. In this way, incremental data events can include addition events, deletion events, or modification events for organization categories, addition events, deletion events, or modification events for user categories, and addition events, deletion events, or modification events for role categories.

[0057] It is understandable that before data synchronization is performed between the server and the client, due to different requirements of each client, the client may need to obtain the full amount of data in a certain category, some categories, or all categories on the server, and this application does not limit this. Figure 3 , for example, client A (corresponding to Figure 3 Application A) in the organization only needs to obtain all the data in the organization category, and client B (corresponding to Figure 3 Application B) in the client only needs to obtain the full amount of data in the user category, and client C (corresponding to Figure 3 Application C) in the server only needs to obtain the full data in the role category. Of course, there are also clients that need to obtain the full data of all categories in the server. For example, client D needs to obtain the full data of all categories such as organization category, user category and role category. For another example, there are clients that need to obtain the full data of some categories in the server. Client E needs to obtain the full data in the organization category and user category, and client F needs to obtain the full data in the user category and role category.

[0058] When the server receives a request for full data of a category sent by a client, it sends the full data of the corresponding category to the corresponding requesting client by IP unicast based on the request for full data of the category, so that the corresponding requesting client only uses persistent data storage. It should be noted that IP unicast, i.e. point-to-point, is used here to push full data to the client because each client has different full data requirements. Therefore, IP unicast can not only meet the specific needs of the client, but also avoid the data pressure on the server and client caused by unified push of full data.

[0059] It should be noted that when the client sends a request for the full data of a category, the request carries the client's IP address. Therefore, when the server sends the full data of the corresponding category to the corresponding requesting client using IP unicast, the server sends the full data of the corresponding category to the corresponding requesting client based on the IP address in the request.

[0060] On the basis that the full data has been updated, if the server monitors the occurrence of incremental data events, that is, an addition event, deletion event, or modification event occurs in one, part, or all of the organization categories, user categories, and role categories, the IP multicast address of the multicast client corresponding to the incremental data event is obtained, and the incremental data corresponding to the incremental data event is sent to the multicast client in multicast mode according to the IP multicast address, so that the multicast client corresponding to this incremental data event performs persistent incremental data updates in the local database. For example, see Figure 4 For example, if this incremental data event is an addition event, deletion event or modification event of the organization category, if we follow the example in the aforementioned embodiment, we can know that the multicast clients corresponding to this incremental data event are clients A, D, and E, obtain the IP multicast addresses of clients A, D, and E, and use the data added, modified, or deleted in the organization category in this incremental data event as incremental data, and send the incremental data to multicast clients A, D, and E based on the IP multicast address, so that multicast clients A, D, and E update the incremental data in the local database. For another example: if the local incremental data event is an addition event, deletion event or modification event of the organization category and role category, then the multicast clients corresponding to the local incremental data event are determined to be clients A, D, E and clients C and F, and the IP multicast addresses of clients A, D, E, C and F are obtained. The data added, modified or deleted in the organization category in this incremental data event is used as the first incremental data, and the data added, modified or deleted in the role category is used as the second incremental data. Based on the IP multicast address, the first incremental data is sent to multicast clients A, D, E, and the second incremental data is sent to multicast clients C and F, so that multicast clients A, D, E, C, F update the incremental data in the local database.

[0061] In actual execution, step S200 includes: when an incremental data event is monitored, obtaining the corresponding event category in the incremental data event, determining the IP multicast address of the multicast client corresponding to each event category and the incremental data of each event category based on the event category, and sending the incremental data of each event category to the multicast client corresponding to each event category in a multicast manner according to the IP multicast address corresponding to each event category.

[0062] According to the data synchronization method provided in the embodiment of the present application, the full amount of data of the corresponding category is sent to the corresponding requesting client by IP unicast, so that the requesting client performs persistent storage of the full amount of data in the corresponding category, and the push of the full amount of data is realized. At the same time, when the incremental data event is monitored, the incremental data corresponding to the incremental data event is sent to the corresponding multicast client by multicast according to the IP multicast address of the multicast client corresponding to the incremental data event, so that the corresponding multicast client performs local data persistent update on the corresponding incremental data, and the real-time push of the incremental data is realized. In the embodiment of the present application, since only the server and the client are involved, and IP unicast and IP multicast are simple and efficient network communication methods, no complex configuration is required, and no additional central node is relied on, the complexity of the system architecture and the additional node resource overhead are avoided, and IP unicast directly transmits data through a clear target address, and the configuration and maintenance operations are simple and intuitive, and IP multicast realizes multi-point transmission of data through the multicast address, and management, maintenance and expansion are more convenient; at the same time, the incremental data in the present application is actively pushed by the server, and the use of IP multicast can ensure the rapid transmission of incremental data while realizing the real-time nature of data, ensuring rapid system response and smooth user experience.

[0063] It is worth mentioning that in the prior art (such as solutions A and D in the background technology), in order to avoid excessive pressure on the server or file server caused by frequent access, the traditional data pulling mode usually synchronizes at intervals of a certain period of time, and is generally set to synchronize once every 15 minutes or more. This results in the client taking some time to see the latest incremental data after the administrator modifies the data (i.e., a data incremental event occurs), thereby affecting the system's response speed and user experience. In contrast, the incremental data is actively pushed by the server, and the use of IP multicast can ensure the rapid transmission of incremental data while achieving real-time data, ensuring a fast system response and a smooth user experience.

[0064] In some embodiments, the method further comprises:

[0065] In a specific time period, the timed full data of a specific category is distributed to the specific multicast client corresponding to the specific category in a multicast manner, so that the specific multicast client updates the local database according to the timed full data. The specific category is a data category for which an incremental data event occurred before the specific time period but the server did not push the incremental data in time, or a data category requested by the client but the server did not push the full data in time.

[0066] Due to the unreliability of the network, there may be an extreme situation where data synchronization fails due to network disconnection. To avoid this situation, at a specific time point, such as the time period from 0:00 to 3:00 in the morning, the timed full data of a specific category where an incremental data event occurs or the timed full data of a specific category requested by the client will be distributed in a multicast manner to the specific multicast client corresponding to the specific category, so that the specific multicast client updates the local database according to the timed full data, thereby ensuring the ultimate consistency of the client and server data.

[0067] For example, continuing with the previous example, if the storage data of the organization category has an incremental data event of adding events, deleting events or modifying events at 4 o'clock in the day, or the client requests the full data of the organization category, but the network is busy or disconnected at that time, the incremental data or full data of the organization category cannot be multicasted and pushed to the corresponding multicast clients A, D, and E in time, then when the waiting time reaches 0 o'clock in the morning, the server will distribute the specific full data corresponding to the organization category to the specific multicast clients A, D, and E in a multicast manner. It can be understood that the specific full data here can be all the incremental data of the specific category before a specific time period, or it can be the full data of the specific category, so as to ensure the final consistency of the client and server data.

[0068] In some embodiments, the server includes a first database and a distributed cache space. Step S100 specifically includes:

[0069] Based on the client's request for full data of a category, the full data of the corresponding category is read from the distributed cache space, and the full data is sent to the corresponding requesting client in an IP unicast manner.

[0070] If reading from the distributed cache space fails, the first database is queried, and all data of the corresponding category is read from the first database and cached into the distributed cache space.

[0071] In this embodiment, the server preferentially reads the full data from its distributed cache interval. When reading from the distributed cache interval fails, it reads the full data of the corresponding category from the server's first database and caches it into the distributed cache space, thereby avoiding excessive access to the server's source data (first database).

[0072] In some embodiments, whether the data in the distributed cache space is invalid depends on the type of user operation event. For example, if the user adds, deletes or modifies the stored data in a certain category, the cache data corresponding to the category in the distributed cache space will be invalidated, thereby ensuring the real-time performance of the cache data in the distributed cache space.

[0073] like Figure 5 As shown, the embodiment of the present application also provides a data synchronization method, which is applied to a client, and the method includes the following steps:

[0074] S210, sending a category full data request to a server, and receiving and storing full data of a corresponding category sent by the server in an IP unicast manner based on the category full data request.

[0075] S220, when it is monitored that there is incremental data of the corresponding category, the incremental data is received and the local database is synchronously updated according to the incremental data, wherein the incremental data is the data of changes in the corresponding category sent by the server in a multicast manner when it is monitored that an incremental data event occurs in the corresponding category.

[0076] According to the data synchronization method provided in the embodiment of the present application, a request for full data of a category is sent to the server, so that the server sends the full data of the corresponding category to the client in an IP unicast manner according to the request for full data of the category, and the client receives and persistently stores the full data. At the same time, when it is detected that there is incremental data of the corresponding category, the incremental data sent by the server in a multicast manner when it is detected that an incremental data event occurs in the corresponding category is received, and the local database is persistently updated according to the incremental data. In the embodiment of the present application, since only the server and the client are involved, and both IP unicast and IP multicast are simple and efficient network communication methods, no complex configuration is required, and no additional central nodes are relied on, thus avoiding the complexity of the system architecture and additional node resource overhead. At the same time, the incremental data in the present application is actively pushed by the server, and the use of IP multicast can ensure the rapid transmission of incremental data while achieving data real-time, ensuring rapid system response and smooth user experience.

[0077] Among them, the data categories stored on the server include organization categories, user categories and role categories, and the incremental data events include addition events, deletion events and modification events.

[0078] It is worth noting that when the client performs step S210, i.e., initializing the full data, it cannot simply perform a TRUNCATE operation and then insert the full data, as such an operation is likely to affect the existing business data. Therefore, when storing the full data, it is necessary to consider automatically converting to an update operation if there is prior business data. Specifically, the INSERT INTO...ON DUPLICATE KEY UPDATE... statement can be used to automatically convert to an update operation when there is existing business data.

[0079] It can be understood that the client in the embodiment of the present application can be any client in the data synchronization method in the aforementioned server-side embodiment. This embodiment can achieve various effects of the data synchronization method in the aforementioned server-side embodiment. To avoid repetition, it will not be repeated here.

[0080] In some embodiments, the method further comprises:

[0081] Receive the timed full data of a specific category sent by the server in a multicast manner in a specific time period, and update the local data according to the timed full data. The specific category is a data category for which an incremental data event occurred before the specific time period but the server did not push the incremental data in time, or a data category requested by the client but the server did not push the full data in time.

[0082] This step can avoid the problem of data synchronization failure caused by unreliable network, thereby ensuring the final consistency of client and server data. For specific explanations, please refer to the description in the server side embodiment.

[0083] In some embodiments, when storing the full amount of data or updating the local database according to the incremental data, the method further includes:

[0084] Determine whether the target update time of the full data or the incremental data is greater than the latest update time of the data to be updated in the local database; when the target update time is greater than the latest update time, update the local database; otherwise, do not update.

[0085] In extreme cases, full data synchronization and incremental data synchronization may be executed concurrently, which may cause the latest data to be overwritten by old data. To avoid this situation, when updating full data or incremental data, the client determines whether the target update time of the full data or incremental data is greater than the latest update time of the data to be updated in the local database. If the target update time is greater than the latest update time, the local database is updated; otherwise, it is not updated, so as to ensure that the updated coverage data is the latest data.

[0086] Specifically, in some embodiments, data errors can be updated by adding a LAST_UPDATE_TIME<#{LAST_UPDATE_TIME} statement at the same time, where LAST_UPDATE_TIME is the latest update time of the data to be updated, and #{LAST_UPDATE_TIME} is the target update time of the incremental data.

[0087] The data synchronization method provided in the embodiment of the present application can be executed by a server. In the embodiment of the present application, the server provided in the embodiment of the present application is described by taking the server executing the data synchronization method as an example.

[0088] The present application also provides a server, such as Figure 6 As shown, the server includes: a first full data module 110, which is used to send the full data of the corresponding category to the corresponding requesting client by IP unicast based on the client's category full data request, so that the requesting client stores the full data. A first incremental data module 120 is used to obtain the IP multicast address of the multicast client corresponding to the incremental data event when monitoring the occurrence of an incremental data event, and send the incremental data corresponding to the incremental data event to the multicast client in a multicast manner according to the IP multicast address, so that the multicast client updates the local database according to the incremental data.

[0089] According to the server provided in the embodiment of the present application, the full data of the corresponding category is sent to the corresponding requesting client by IP unicast, so that the requesting client performs persistent storage of the full data in the corresponding category, thereby realizing the push of the full data. At the same time, when the occurrence of an incremental data event is monitored, the incremental data corresponding to the incremental data event is sent to the corresponding multicast client by multicast according to the IP multicast address of the multicast client corresponding to the incremental data event, so that the corresponding multicast client performs persistent update of the local data of the corresponding incremental data, thereby realizing the real-time push of the incremental data. In the embodiment of the present application, since only the server and the client are involved, and both IP unicast and IP multicast are simple and efficient network communication methods, no complex configuration is required, and no additional central node is relied on, thus avoiding the complexity of the system architecture and the additional node resource overhead. At the same time, the incremental data in the present application is actively pushed by the server, and the use of IP multicast can ensure the rapid transmission of incremental data while realizing the real-time nature of data, thereby ensuring the rapid response of the system and the smooth user experience.

[0090] The data synchronization method provided in the embodiment of the present application can also be executed by a client. In the embodiment of the present application, the client provided in the embodiment of the present application is described by taking the client executing the data synchronization method as an example.

[0091] The present application embodiment also provides a client, such as Figure 7 As shown, the client includes: a second full data module 210, which is used to send a category full data request to the server, and receive and store the full data of the corresponding category sent by the server in an IP unicast manner based on the category full data request. A second incremental data module 220 is used to receive the incremental data and update the local database according to the incremental data when it is monitored that there is incremental data of the corresponding category, wherein the incremental data is sent by the server in a multicast manner when it is monitored that an incremental data event occurs in the corresponding category.

[0092] According to the client provided in the embodiment of the present application, a category full data request is sent to the server, so that the server sends the full data of the corresponding category to the client in an IP unicast manner according to the category full data request. The client receives and persistently stores the full data. At the same time, when it is monitored that there is incremental data of the corresponding category, the client receives the incremental data sent by the server in a multicast manner when it is monitored that an incremental data event occurs in the corresponding category, and persistently updates the local database according to the incremental data. Since only the server and the client are involved, and IP unicast and IP multicast are simple and efficient network communication methods, no complex configuration is required, and no additional central nodes are relied on, thereby avoiding the complexity of the system architecture and additional node resource overhead. At the same time, the incremental data in the present application is actively pushed by the server, and the use of IP multicast can ensure the rapid transmission of incremental data while achieving real-time data, thereby ensuring a fast system response and a smooth user experience.

[0093] The present application also provides a data synchronization system. Figure 8 As shown, it includes the server and the client.

[0094] The client is used to send a category full data request to the server.

[0095] The server is used to send the full data of the corresponding category to the corresponding client in an IP unicast manner based on the full data request of the category by the client.

[0096] The client is used to receive and store the full amount of data of the corresponding category.

[0097] The server is also used to obtain the IP multicast address of the multicast client corresponding to the incremental data event when monitoring the occurrence of the incremental data event, and send the incremental data corresponding to the incremental data event in a multicast manner according to the IP multicast address.

[0098] The client is also used to receive the incremental data and update the local database according to the incremental data when it is monitored that there is incremental data of a corresponding category in the incremental data sent by the server.

[0099] According to the data synchronization system provided in the embodiment of the present application, since it only involves the server and the client, and IP unicast and IP multicast are simple and efficient network communication methods, it does not require complex configuration and does not rely on additional central nodes, thus avoiding the complexity of the system architecture and additional node resource overhead. At the same time, the incremental data in this application is actively pushed by the server, and the use of IP multicast can ensure the rapid transmission of incremental data while realizing real-time data, ensuring a quick system response and a smooth user experience.

[0100] The following combination Figure 8 The data synchronization system architecture diagram shown in FIG. Figure 8 It can be seen that there are two stages between the server and the client, one is the full data synchronization stage, and the other is the incremental data synchronization stage. In the full data synchronization stage, the client's synchronization component initializes the full data and sends a category full data request to the server's synchronization component. The request carries the client's IP address. When the server-side full data synchronization request listener thread listens to the category full data request, it sends the full data of the corresponding category to the client in IP unicast mode. The client receives and stores the full data of the corresponding category. In this process, the client implements the final persistent storage or update through the INSERT INTO...ON DUPLICATE KEY UPDATE...WHERE...AND LAST_UPDATE_TIME<#{LAST_UPDATE_TIME} syntax format.

[0101] It is worth noting that in this process, the client's synchronization logic is encapsulated through springboot-starter-sync. The client directly introduces project dependencies through Maven or Gradle to automatically synchronize data with the server in real time, reducing the complexity of manual configuration, simplifying configuration, reducing boilerplate code, and speeding up development.

[0102] The Maven example is as follows:

[0103] <dependency>

[0104] <groupid> org.springframework.boot< / groupid>

[0105] <artifactid> spring-boot-starter-sync< / artifactid>

[0106] < / dependency>

[0107] The Gradle example is as follows:

[0108] implementation'org.springframework.boot:spring-boot-starter-sync'

[0109] At the same time, in order to avoid the problem that data may not be synchronized in time due to unreliable network, the server will also distribute the timed full data of a specific category to the specific multicast client corresponding to the specific category in a multicast manner during a specific time period (such as the early morning time period), so that the specific multicast client can update the local database according to the timed full data. Among them, the specific category is a data category for which an incremental data event occurred before the specific time period but the server did not push the incremental data in time, or a data category requested by the client but the server did not push the full data in time. It can be understood that only pushing timed full data of a specific category during a specific time period can avoid pushing data of data categories that have not changed or are not in demand, thereby minimizing resource usage.

[0110] In addition, the server includes a first database (ie Figure 8 The server-side database) and distributed cache space (i.e. Figure 8 The server will first read the corresponding full data from the distributed cache space. If the data reading from the distributed cache space fails, it will query the first database and read the full data of the corresponding category from the first database to minimize the access to the first database. Whether the distributed cache space is cached successfully depends on the user's operation event type. For example, when an add, delete or modify event occurs, the cache is invalid.

[0111] In the incremental data synchronization stage, the server uses an incremental data event listener to monitor whether an incremental data event occurs. Incremental data events are divided into user events, organization events, and role events according to data categories. When an incremental data event is monitored, the IP multicast address of the multicast client corresponding to the incremental data event is obtained, and the incremental data corresponding to the incremental data event is sent in a multicast manner according to the IP multicast address. For example, if an organization event is monitored, the corresponding multicast client and its IP multicast address are obtained from the organization event, and the incremental data of the organization event is sent to the multicast client according to the IP multicast address. When the client corresponding to the IP multicast address receives the incremental data, it updates the local data (i.e. Figure 8 in the client's database).

[0112] In the above data synchronization process, only the server, client, IP unicast and IP multicast are involved, and IP unicast and IP multicast are simple and efficient network communication methods, which do not require complex configuration and do not rely on additional central nodes, avoiding the complexity of the system architecture and additional node resource overhead. At the same time, the incremental data in this application is actively pushed by the server, and the use of IP multicast can ensure the rapid transmission of incremental data while realizing real-time data, ensuring rapid system response and smooth user experience.

[0113] The server, client, and data synchronization system in the embodiments of the present application may be electronic devices, or components in electronic devices, such as integrated circuits or chips. The electronic device may be a terminal, or other devices other than a terminal. Exemplarily, the electronic device may be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook, or a personal digital assistant (personal digital assistant, PDA), etc. It may also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine, or a self-service machine, etc., which is not specifically limited in the embodiments of the present application.

[0114] The server, client, and data synchronization system in the embodiments of the present application may be devices with an operating system. The operating system may be a Microsoft (Windows) operating system, an Android (Android) operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiments of the present application.

[0115] The server, client, and data synchronization system provided in the embodiments of the present application can achieve Figures 2 to 5 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0116] In some embodiments, Fig. 9 As shown, an embodiment of the present application also provides an electronic device 800, including a processor 801, a memory 802, and a computer program stored in the memory 802 and executable on the processor 801. When the program is executed by the processor 801, each process of the above-mentioned data synchronization method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0117] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0118] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned data synchronization method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0119] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0120] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned data synchronization method when executed by a processor.

[0121] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0122] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned data synchronization method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0123] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0124] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0125] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0126] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0127] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0128] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A data synchronization method, characterized in that: Applied to the server, the method includes: Based on the client's request for full data of a category, the full data of the corresponding category is sent to the corresponding requesting client in an IP unicast manner, so that the requesting client stores the full data; When an incremental data event is monitored, the IP multicast address of the multicast client corresponding to the incremental data event is obtained, and the incremental data corresponding to the incremental data event is sent to the multicast client in a multicast manner according to the IP multicast address, so that the multicast client synchronously updates the local database according to the incremental data.

2. The method according to claim 1, characterized in that The method further comprises: Distributing the timed full data of a specific category to a specific multicast client corresponding to the specific category in a multicast manner in a specific time period, so that the specific multicast client updates the local database according to the timed full data; Among them, the specific category is a data category for which an incremental data event occurred before the specific time period but the server did not push the incremental data in time, or a data category for which the client requested but the server did not push the full data in time.

3. The method according to claim 1, characterized in that The server includes a first database and a distributed cache space; The method of sending the full data of the corresponding category to the corresponding requesting client in an IP unicast manner based on the full data request of the client category, so that the requesting client stores the full data, includes: Based on the client's request for full data of a category, read the full data of the corresponding category from the distributed cache space, and send the full data to the corresponding requesting client in an IP unicast manner; If reading from the distributed cache space fails, the first database is queried, and all data of the corresponding category is read from the first database and cached into the distributed cache space.

4. The method according to any one of claims 1 to 3, characterized in that: The categories include organization categories, user categories and role categories; the incremental data events include addition events, deletion events and modification events.

5. A data synchronization method, characterized in that: Applied to a client, the method comprises: Sending a category full data request to a server, and receiving and storing full data of a corresponding category sent by the server in an IP unicast manner based on the category full data request; When the existence of incremental data of the corresponding category is monitored, the incremental data is received and the local database is synchronously updated according to the incremental data, wherein the incremental data is the data of changes in the corresponding category sent by the server in a multicast manner when the incremental data event of the corresponding category is monitored.

6. The method according to claim 5, characterized in that The method further comprises: Receiving a specific category of timed full data sent by the server in a multicast manner in a specific time period, and updating the local data according to the timed full data; Among them, the specific category is a data category for which an incremental data event occurred before the specific time period but the server did not push the incremental data in time, or a data category for which the client requested but the server did not push the full data in time.

7. The method according to claim 5, characterized in that When storing the full amount of data or updating the local database according to the incremental data, the method further includes: Determine whether the target update time of the full data or the incremental data is greater than the latest update time of the data to be updated in the local database; when the target update time is greater than the latest update time, determine to update the local database; otherwise, do not update.

8. A server, characterized in that: include: A first full data module is used to send the full data of the corresponding category to the corresponding requesting client in an IP unicast manner based on the full data request of the category by the client, so that the requesting client stores the full data; The first incremental data module is used to obtain the IP multicast address of the multicast client corresponding to the incremental data event when monitoring the occurrence of an incremental data event, and send the incremental data corresponding to the incremental data event to the multicast client in a multicast manner according to the IP multicast address, so that the multicast client updates the local database according to the incremental data.

9. A client, characterized in that: include: A second full data module is used to send a category full data request to the server, and receive and store the full data of the corresponding category sent by the server in an IP unicast manner based on the category full data request; The second incremental data module is used to receive the incremental data and update the local database according to the incremental data when it is monitored that there is incremental data of the corresponding category, wherein the incremental data is sent by the server in a multicast manner when it is monitored that an incremental data event occurs in the corresponding category.

10. A data synchronization system, characterized in that: Including server and client; The client is used to send a category full data request to the server; The server is used to send the full data of the corresponding category to the corresponding client in an IP unicast manner based on the full data request of the category by the client; The client is used to receive and store the full amount of data of the corresponding category; The server is further configured to obtain the IP multicast address of the multicast client corresponding to the incremental data event when monitoring the occurrence of the incremental data event, and send the incremental data corresponding to the incremental data event in a multicast manner according to the IP multicast address; The client is also used to receive the incremental data and update the local database according to the incremental data when it is monitored that there is incremental data of a corresponding category in the incremental data sent by the server.

Images