A method and system for data change awareness and subscription synchronization

Abstract

The present invention relates to a data change awareness and subscription synchronization method and system. The method includes: Step S1: Archive log monitoring; Step S2: Access a message queue based on the subscription configuration of the client to obtain subscription data; Step S3: Send the subscription data to the client for data change awareness and subscription synchronization. The present invention reduces the management difficulty of data change and synchronization by making full and effective use of the database and its interface access capabilities, and ultimately improves the synchronization efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field of data synchronization, and particularly relates to a method and system for data change perception and subscription synchronization.

Background Art

[0002] With the construction of the data middle platform, master data is centrally managed and stored, and the shared service platform based on master data has a great impact on various related services such as data query, data synchronization, and change notification in the original business systems. For example: With the construction of the employee center, both data and service interfaces have changed greatly. The system needs to synchronously obtain personnel information data, and the original mechanism can no longer meet the requirements; due to its own business needs, key employee position change information needs to store all change histories (zipper tables) in order to trace historical information; in the metering business, it is necessary to associate files and collection information for cross-database query operations, but the amount of data is too large to be realized. The commonly used methods for data change perception, such as stored procedures or interface hard coding, all have problems such as intrusion into the original system code, affecting system performance, difficult later maintenance, and low generality.

[0003] The present invention realizes actively pushing data changes to the message queue by listening to the database archival logs, and then, according to the subscription configuration information, the subsequent data processing program processes it, reducing the management difficulty of data change and synchronization and improving the efficiency of subscription synchronization.

Summary of the Invention

[0004] In order to solve the above problems in the prior art, the present invention proposes a method and system for data change perception and subscription synchronization. The system includes:

[0005] A subscription buffer module, a message queue, a server and the master database configured thereon, and a business database configured on the client machine;

[0006] The subscription buffer module is used to store subscription configuration information for change information, obtain and store change information;

[0007] The subscription buffer module includes a first buffer and a second buffer; wherein the first buffer is used to store subscription configuration information; read the subscription configuration information in the first buffer to determine whether the data change involved in the change notification in the message queue is subscribed. If so, process the message change notification to obtain the corresponding change message, and store the change information in the second buffer; the second buffer is used to save the change information obtained according to the subscription configuration information; it is used to store the data change information of the subscribed master database obtained from the server based on the subscription configuration information in the first buffer; the second buffer adopts a multi-partition dynamic management method;

[0008] The message queue is used to obtain and store data change notifications by processing log files; the messages in the message queue are arranged in the order of log processing, with the log messages processed first located at the head of the message queue and the log messages processed later located at the rear of the message queue;

[0009] The client subscribes and configures through the subscription interface provided by the buffer module, stores the obtained subscription configuration information in the first buffer, and obtains the change information in the second buffer for data synchronization.

[0010] Furthermore, the data change notification includes the database, data table, and primary key identifier targeted by the change.

[0011] Furthermore, when the first buffer receives new subscription configuration information, it merges the subscription configuration information in the first buffer based on the inclusion relationship of the subscription configuration information.

[0012] Furthermore, the client is one or more.

[0013] Furthermore, there is one or more business databases.

[0014] A data change perception and subscription synchronization method based on the above data change perception and subscription synchronization system, including:

[0015] Step S1: Archive log listening;

[0016] Step S2: Access the message queue based on the client's subscription configuration to obtain subscription data;

[0017] Step S3: Send the subscription data to the client for data change perception and subscription synchronization.

[0018] Furthermore, the subscription data is sent to the client in the form of data packets for subscription synchronization.

[0019] A processor, which is used to run a program, wherein when the program runs, it executes the above data change perception and subscription synchronization method.

[0020] A computer-readable storage medium, including a program, which when running on a computer causes the computer to execute the above data change perception and subscription synchronization method.

[0021] A cloud server, characterized in that the cloud server is configured to execute the above data change perception and subscription synchronization method.

[0022] The beneficial effects of the present invention include:

[0023] (1) By merging the subscribed configuration information, the access frequency of server data and the size of the required second buffer space are reduced. Setting up the second buffer space for partition management can transform out-of-order information into in-order information during the reading process, reducing the management difficulty of data change and synchronization and improving the efficiency of subscription synchronization.

[0024] (2) By using the gapless cooperation between the second buffer and the data packet buffer space, the serial data access to a log and a client database is converted into a data packet that can fully utilize the database access capability at one time. While ensuring the data relationship and the log processing order, the database and its interface access efficiency are maximized.

[0025] (3) Based on the size of the data packet sent at one time, a dynamically sized data packet is formed adaptively during the synchronization process, which can most effectively utilize the database and its interface access capability and ultimately improve the synchronization efficiency.

[0026] (4) By taking advantage of the non-correlation characteristics of the data to be synchronized, a trade-off is made between the access overhead of a large data block at one time and the access overhead of multiple small-sized data blocks based on the fragmentation degree and size of the synchronized data. The data packets are reorganized to improve the synchronization efficiency on the basis of reducing the possible waste of synchronization time.

BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The drawings described herein are used to provide a further understanding of the present invention, form a part of this application, but do not constitute an improper limitation to the present invention. In the drawings:

[0028] Figure 1 It is a schematic diagram of the data change perception and subscription synchronization system provided by the present invention;

[0029] Figure 2 It is a schematic diagram of the data change perception and subscription synchronization method provided by the present invention;

[0030] Figure 3 It is a schematic diagram of an implementation manner of the data change perception and subscription synchronization method provided by the present invention;

[0031] Figure 4 It is a schematic diagram of an implementation manner of obtaining the complete data of a data block containing changed data provided by the present invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

[0032] The present invention will be described in detail below in conjunction with the drawings and specific embodiments. The illustrative embodiments and descriptions are only used to explain the present invention, but do not limit the present invention;

[0033] As shown in the attached Figure 1As shown in the figure, a data change awareness and subscription synchronization system provided by the present invention includes: a subscription buffer module, a message queue, a server and a main database configured thereon, and a business database configured on a client machine;

[0034] The subscription buffer module is used to store subscription configuration information for change information, obtain and store change information; the subscription buffer module includes a first buffer and a second buffer; wherein the first buffer is used to store subscription configuration information; when the first buffer receives new subscription configuration information, the subscription configuration information in the first buffer is merged based on the inclusion relationship of the subscription configuration information;

[0035] The second buffer is used to save the change information obtained according to the subscription configuration information; it is used to store the data change information subscribed from the server based on the subscription configuration information in the first buffer; the second buffer adopts a multi-partition dynamic management method;

[0036] The message queue is used to obtain and store data change notifications by processing log files; the data change notifications include the database, data table, primary key identifier, etc. targeted by the change; the log file corresponding to the operation condition record of the main database of the server corresponds to one or more data tables, row records and / or databases; the log file can only be processed strictly in the generated order, therefore, the messages in the message queue are arranged in the order of the log processing sequence, the log message processed first is located at the head of the message queue, and the log message processed later is located at the rear of the message queue;

[0037] The subscription buffer module is also used to read the subscription configuration information in the first buffer to determine whether the data change involved in the change notification in the message queue is subscribed. If so, process the message change notification to obtain the corresponding change message, and store the change information in the second buffer;

[0038] Preferably: after receiving a new data change notification, read the subscription configuration information in the first buffer to determine whether the data change involved in the notification is subscribed;

[0039] The client performs subscription configuration through the subscription interface provided by the subscription buffer module, stores the obtained subscription configuration information in the first buffer, and obtains the change information in the second buffer for data synchronization;

[0040] The merging of the subscription configuration information in the first buffer is specifically as follows: based on the comparison of the data ranges involved in the subscription configuration attributes in the subscription configuration information, when the data ranges involved in two subscription configuration information are the same, the two subscription configuration information are merged; it also includes: merging in the order of the data ranges involved in the subscription configuration attributes in the subscription configuration information from large to small, determining whether the data ranges involve an inclusion relationship, and if so, merging the subscription configuration information; retaining the subscription configuration information with a larger data range and deleting the subscription configuration information with a smaller data range;

[0041] The determination of whether the data ranges involve an inclusion relationship is specifically as follows: sequentially use the data ranges involved in the subscription configuration attributes in the subscription configuration information from large to small as the current attributes for judgment. When the attribute values of the current attributes of two subscription configuration information are in an inclusion relationship and the attributes larger than the data ranges involved in the current attributes are the same, the two subscription configuration attributes are merged, and the merged subscription configuration information is used for data change subscription; among them: the subscription attributes include one or more of database, service identifier, data table, field, keyword, etc.; after the subscription configuration attributes are merged, subscribe using the subscription configuration information with a larger data range, and set the subscription flag corresponding to the merged subscription configuration information to be equal to the number of merges plus 1; use the subscription flag to record the situation where the data in the second buffer needs to be reused; the number of merges plus 1 can be used as a value fixedly associated with the subscription configuration information, and when the subscription configuration does not change, this fixedly associated value will not change either;

[0042] Read the subscription configuration information in the first buffer to determine whether the data changes involved in the change notification in the message queue are subscribed. If so, process the message change notification to obtain the corresponding change message, and store the change information in the second buffer; specifically: access the server based on the merged subscription configuration information to obtain the subscribed change data; the second buffer is provided with multiple partitions; each partition is a dynamic buffer space with a head pointer set; each partition corresponds to a log file; the dynamic buffer space is managed in the form of a dynamic linked list; after obtaining the subscription-related change information from the server, apply for a dynamic buffer space according to the size of the change information, store the change information in the dynamic buffer space, and then connect the dynamic buffer space to the tail of the dynamic linked list of the partition corresponding to the change information; the notifications in the message queue arrive in disorder because the refresh frequency is not fixed either. Here, the merging of the subscription configuration information reduces the access frequency of the server data and the size of the second buffer space required, and setting the second buffer space managed by partitions can transform the disordered information into sequential information during the reading process, reducing the management difficulty of data change and synchronization and improving the efficiency of subscription synchronization;

[0043] The obtaining of the change information in the second buffer for data synchronization is specifically as follows: data is packed according to the client corresponding to the subscription configuration information corresponding to the change information, it is judged whether the data packing is completed, and after the packing is completed, the formed data packet is sent to each client, and the client writes the data packet into the service database for data synchronization;

[0044] The obtaining of the change information in the second buffer for data synchronization specifically includes the following steps:

[0045] Step SA1: Real-time detect the synchronization timing parameter to determine whether the synchronization timing for the client has arrived. If the synchronization timing for the client has arrived, use the client as the current client;

[0046] The synchronization timing settings for each client are different. Therefore, the arrival of their synchronization timing is asynchronous and uncorrelated;

[0047] Preferably: The synchronization timing is to judge whether a specific time interval has been reached, and the corresponding synchronization timing parameter is the elapsed time; if so, the synchronization timing has arrived;

[0048] Step SA2: Obtain an unprocessed partition of the second buffer as the current partition; if the dynamic linked list of the current partition is empty, obtain another unprocessed partition as the current partition; if all partitions have been processed, enter step SA7;

[0049] One partition corresponds to one log information, and the sequential processing of the processing of one log information can ensure the correctness of the data; the present invention uses the gapless cooperation between the second buffer and the data packet buffer space to convert the serial data access of one log and one client database into a data packet that can fully utilize the database access capability at one time. While ensuring the data relationship and the log processing order, the access efficiency of the database and its interface is maximized;

[0050] Step SA3: Initialize the sliding pointer; set the initial value of the sliding pointer to the head pointer of the current partition;

[0051] Step SA4: Obtain the change information pointed to by the sliding pointer; judge whether the change information is subscribed by the current client. If so, put the change information into the data packet buffer space; subtract 1 from the value of the subscription flag; when the subscription flag is 0, delete the dynamic buffer space corresponding to the change information, and reconnect the dynamic linked list corresponding to the partition; if not, update the sliding pointer to point to the next dynamic buffer space in the dynamic linked list;

[0052] Preferably, when the change information is subscribed by the current client and the remaining data packet buffer space cannot accommodate the change information, set the current partition as an unprocessed partition and return to step SA2 to attempt to obtain the change information from the next unprocessed branch. Of course, corresponding flags need to be set to prevent the partition from being processed repeatedly and falling into an infinite loop. Through such a local optimization method of data packet patching based on the processing order, the utilization rate of data packets is improved with minimal computational overhead. Of course, a less efficient processing method is to split the change information for multiple transmissions, or to apply for a temporary buffer to store all change information. Such a method can also be adopted when a complete data packet buffer space cannot store a piece of change information.

[0053] Step SA5: Determine whether the sliding pointer is at the end of the dynamic linked list. If not, return to step SA4; if so, enter step SA6.

[0054] Preferably, an invalid value is stored at the end of the linked list to indicate the end of the dynamic linked list.

[0055] Step SA6: Determine whether to continue filling the data packet. If not, enter step SA7; if so, return to step SA2.

[0056] The determination of whether to continue filling the data packet is specifically as follows: Determine whether the occupancy ratio of the data packet buffer space exceeds a preset occupancy ratio, or determine whether the occupied size of the data packet buffer space exceeds a preset size. The setting size of the preset size is related to the greediness.

[0057] Preferably, the preset occupancy ratio is 90%; the occupied size is equal to the buffer space size * preset occupancy ratio.

[0058] Preferably, the size of the data packet buffer space is equal to the size of the data packet sent at one time. Among them, the size of the data packet sent at one time is the size of the cache space allowed by the database interface. Different databases provide different interfaces, and the amount of data transmission that can be allowed to be completed by the latter each time accessing the database interface is different. Based on the size of the data packet sent at one time, a dynamically sized data packet is adaptively formed during the synchronization process, which can most effectively utilize the database and its interface access efficiency, and ultimately improve the synchronization efficiency.

[0059] Step SA7: Synchronize the data in the data packet buffer space to the business database of the current client through the database access interface. At this time, the data synchronization of the current client ends.

[0060] Preferably, after the data synchronization of the current client ends, clear the data packet buffer space.

[0061] When synchronizing data, there is a relationship between the synchronization overhead of the data and the data size, but such a relationship is not a linear relationship, but rather a stage growth relationship. The time length required for data synchronization for the data size of the same data block is the same or very similar. Whether the data is in the same data block is related to the actual physical storage method of the data. The present invention utilizes the non-correlation characteristics of the data to be synchronized, makes a trade-off between the access overhead of a large data block at one time and the access overhead of multiple small-sized data blocks based on the fragmentation degree and size of the synchronized data, re-organizes the data packets, and improves the synchronization efficiency on the basis of reducing the possible waste of synchronization time;

[0062] Synchronizing the data in the data packet buffer space to the business database of the client through the database access interface specifically includes:

[0063] Step SB1: Initialize the data block size;

[0064] For example: Initialize the data block size to the minimum data block size LBS*X; where: X is the memory bit width, for example, 32bit;

[0065] Preferably: Initialize the data block size to the maximum data size that can be synchronized within a fixed time length for one memory access;

[0066] Step SB2: Analyze the synchronization address of the data to be synchronized in the data packet to obtain the total synchronized data size for the same data block in the data packet buffer space. When the ratio of the total synchronized data size to the data block size exceeds the preset ratio, and the fragmentation degree of the synchronized data in the same data block is greater than the preset fragmentation value, go to step SB3; otherwise, determine whether the data block size is the minimum data block size LBS (for example, 8*64Bit). If so, go to step SB4; otherwise, reduce the data block size and return to step SB2;

[0067] Preferably: Set the fragmentation degree Ds equal to the number Cn of synchronized data with consecutive addresses in the same data block; and the discontinuous part constitutes the fragments in the data block;

[0068] Preferably: Set the fragmentation degree Ds = Cn×LBS / BS; for example: the same data block involves addresses add0000 - add1111, and the synchronization addresses are add0000 and add0001. Since the addresses add0000 and add0001 are adjacent, although there are two synchronized data, they are continuous and form a continuous synchronized data, so no fragments will be generated. However, since the addresses add0011 and add0001 are not adjacent, they are two continuous synchronized data and will increase the fragmentation degree; where add is the omitted address value;

[0069] Preferably, reducing the data block size specifically means setting the data block size BS = data block size BS / 2;

[0070] Step SB3: Obtain the complete data of the data block containing the changed data and synchronize the complete data to the business database of the client; after completing this step, all the data to be synchronized for the same data block has been synchronized;

[0071] As shown in the appendix Figure 4 Specifically, obtaining the complete data of the data block containing the changed data means: obtaining the logical identifier of the data in the data block based on the physical address of the data block, and querying the server database based on the logical identifier to obtain the complete data of the data block; of course, we can also obtain the complete data by splicing the changed data with the unchanged data part read from the business database, and its efficiency depends on the fragmentation degree of the changed data. When the fragmentation degree is not high, the splicing method obviously has higher efficiency; this is the part that can be weighed;

[0072] Step SB4: Determine whether all the data to be synchronized in the data packet buffer space has been synchronized. If not, synchronize the unsynchronized data to be synchronized in the data packet buffer space to the business database of the client one by one; if so, the data synchronization ends;

[0073] As shown in the appendix Figures 2-3 As shown, a data change perception and subscription synchronization method provided by the present invention includes the following steps:

[0074] Step S1: Archive log listening; specifically: scan the archive logs of the server main database and organize the scanned data into the format of a data table and store it in the message queue;

[0075] Preferably, the database is a multi-source database; when the database is an Oracle database, listen to the Oracle archive logs, use the Ogg component to scan the archive logs, send them to the OggForBigData component, and then press them into the Kafka queue with the data table as the key of the Kafka message; when the database is a MySQL database, the listening service is simulated as a sub-station of MySQL, send instructions that need to be synchronized to the MySQL database, so as to obtain the BinLog log data, and thus obtain the data change information of the main database. The service presses the data into the Kafka queue with the data table as the key of the Kafka message; for the DMHS-provided log CPT component, the CPT module parses the transaction logs of the source database, obtains the incremental changes of the data, and encapsulates the incremental changes into DMHS messages. The data messages are filtered in the extract end in the same mechanism as Ogg using the schema method;

[0076] Preferably: The scanning is parallel scanning. An Ogg component is configured separately for each Oracle database. The Ogg component directly scans the archived log files on the storage medium. Multiple Replicate processes are deployed for one Ogg component. An rmttrail directory is set in the Ogg component to distinguish different data sources in the database. Each Replicate process corresponds to an Ogg data source by corresponding to an exttrail directory.

[0077] Preferably: The storage medium is a disk.

[0078] Preferably: Data is filtered through subscriptions, and only the subscribed data is pushed into Kafka, thereby reducing the pressure on Kafka.

[0079] Preferably: Each time the service successfully sends a data table to Kafka, it records the log position in Redis. When a failure occurs, or when the listening service restarts or is transferred due to load balancing, the log transmission can be restored from the previous CheckPoint. Only one listener needs to be started for each database. Therefore, the CheckPoint in Redis is recorded by database, and the RedisKey is designed as: xgg_data source name:binlog_position.

[0080] The log listening service is designed without a centralization. Therefore, through the publish / subscribe mode of RabbitMQ, its own registration information is sent to other log listening services. To ensure the timeliness and effectiveness of the registration information, the registration information cache in the service needs to expire regularly. The service updates the cache information by sending the registration information regularly to ensure the cache is valid. The validity period of the cache is set to 3 times the registration information sending period.

[0081] Preferably: The registration information includes the IP and port of the service.

[0082] Step S2: Access the message queue based on the subscription configuration of the client to obtain the subscribed data. As shown in the attached Figure 3 figure, for the requirement of pushing some key fields of certain data tables to the business database, the key field pushing method is adopted. Through the setting of the subscription configuration, some key field contents in the subscribed table are directly written into the business database.

[0083] Preferably: Regularly load the subscription configuration to update the first cache. The first buffer is used to store the subscription configuration information.

[0084] Preferably: The subscription configuration includes the name of the subscription, the database number where the subscribed table is located, the schema name where it is located, the subscription mode, the name of the timestamp field, etc.

[0085] Preferably: The client is a client that needs to perform data change awareness and subscription synchronization; Only registered clients can call the shared service interface for subscription;

[0086] Preferably: There is one or more clients; A business database is set in the client;

[0087] Step S3: Send the subscription data to the client for data change awareness and subscription synchronization; The client database performs database synchronization based on the data change and subscription messages;

[0088] Preferably: Send the subscription data to the client in the form of a data packet for subscription synchronization;

[0089] The terms "server" and "client" encompass all kinds of devices, equipment, and machines used for processing data, including, by way of example, programmable processors, computers, system-on-chips, or multiple of the foregoing or combinations thereof. The device can include dedicated logic circuitry, such as FPGAs (Field Programmable Gate Arrays) or ASICs (Application Specific Integrated Circuits). In addition to hardware, the device can also include code that creates an execution environment for the computer program, e.g., code that constitutes processor firmware, protocol stacks, database management systems, operating systems, cross-platform runtime environments, virtual machines, or combinations of one or more of the foregoing. The device and the execution environment can implement various different computing model infrastructures, such as web services, distributed computing, and grid computing infrastructures.

[0090] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including assembly or interpreted languages, declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may or may not correspond to a file in a file system. The program can be stored as part of a file that holds other programs or data (e.g., one or more scripts in a markup language document), in a single file dedicated to the program, or in multiple cooperating files (e.g., files that store one or more modules, subroutines, or portions of code). A computer program can be deployed to execute on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.

[0091] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.

[0092] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0093] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0094] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications or equivalent replacements can still be made to the specific implementation manners of the present invention, and any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the protection scope of the claims of the present invention.

Claims

1. A data change perception and subscription synchronization system, characterized in that The system includes: a subscription buffer module, a message queue, a server, a main database configured thereon, and a business database configured on a client machine; the subscription buffer module is used to store subscription configuration information for change information, and obtain and store change information; the subscription buffer module includes a first buffer and a second buffer; wherein the first buffer is used to store subscription configuration information; read the subscription configuration information in the first buffer to determine whether the data change involved in the change notification in the message queue is subscribed. If so, process the message change notification to obtain a corresponding change message, and store the change information in the second buffer; the second buffer is used to save the change information obtained according to the subscription configuration information; used to store the data change information of the subscribed main database obtained from the server based on the subscription configuration information in the first buffer; the second buffer adopts a multi-partition dynamic management method; when the first buffer receives new subscription configuration information, merge the subscription configuration information in the first buffer based on the inclusion relationship of the subscription configuration information; the message queue is used to obtain and store data change notifications by processing log files; the messages in the message queue are arranged in the order of log processing sequence. The log messages processed first are located at the head of the message queue, and the log messages processed later are located at the rear of the message queue; the client performs subscription configuration through the subscription interface provided by the subscription buffer module, stores the obtained subscription configuration information in the first buffer, and obtains the change information in the second buffer for data synchronization.

2. The data change perception and subscription synchronization system according to claim 1, characterized in that The data change notification includes the database, data table, and primary key identifier targeted by the change.

3. The data change perception and subscription synchronization system according to claim 2, characterized in that The client is one or more.

4. The data change perception and subscription synchronization system according to claim 3, characterized in that, The business database is one or more.

5. A data change perception and subscription synchronization method based on any one of the data change perception and subscription synchronization systems of claims 1-4, characterized in that, including: Step S1: Archive log listening; Step S2: Access the message queue based on the client's subscription configuration to obtain subscription data; Step S3: Send the subscription data to the client for data change awareness and subscription synchronization.

6. The data change perception and subscription synchronization method according to claim 5, wherein Send the subscription data to the client in the form of a data packet for subscription synchronization.

7. A processor, characterized in that, The processor is used to run a program, wherein, when the program runs, it executes the data change awareness and subscription synchronization method according to any one of claims 5-6.

8. A computer-readable storage medium, characterized in that, including a program, when it runs on a computer, causing the computer to execute the data change awareness and subscription synchronization method according to any one of claims 5-6.

9. A cloud server, characterized in that, The cloud server is configured to execute the data change awareness and subscription synchronization method according to any one of claims 5-6.

Images