Client request processing method and data synchronization system
By using tracepoint information and a synchronization point library in the MongoDB data synchronization system to verify data integrity, the latency problem in multi-datacenter, multi-active deployments was solved, and the system reliability and response speed were improved.
Patent Information
- Application Number
- CN202411144538.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2025-11-04
- Estimated Expiration
- 2044-08-20
AI Technical Summary
In a multi-datacenter, multi-active deployment of MongoDB, there is a data inconsistency issue caused by latency, which leads to abnormal client request processing.
By using trace point information and local/global synchronization point libraries to verify data integrity at the application layer and service routers in the target data center, we ensure that the latest and consistent data is used, thereby reducing the risk of processing anomalies caused by latency.
This improves the overall reliability and response speed of the system, reduces performance issues caused by data synchronization delays, and ensures the accuracy and consistency of client request processing.
Smart Images

Figure CN119094176B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] Embodiments of the present application relate to the technical field of cloud computing, in particular to a client request processing method and a data synchronization system. BACKGROUND
[0002] In today's cloud computing environment, multi-data center active-active deployment is mainly to improve system reliability, availability, disaster recovery capability, performance and scale expansion capability, and to meet the requirements of laws on data geographical location. MongoDB is a current most popular distributed document type NoSql database. The official recommended multi-data center deployment scheme of MongoDB mainly uses the replica set and sharding functions of MongoDB.
[0003] However, the way of using replica set and sharding functions of MongoDB for active-active deployment will cause delay problem between different nodes, and cause data inconsistency due to delay, thereby causing request processing exception when processing client requests. SUMMARY
[0004] Embodiments of the present application provide a client request processing method and a data synchronization system, which confirm data integrity by using tracking point information and local synchronization point library, ensure that the latest and consistent data is used when processing client requests, reduce the risk of processing exception caused by data delay, and improve the overall reliability of the system. The technical scheme is as follows.
[0005] On the one hand, a client request processing method is provided, which is executed by an application layer of a target data center, the target data center is any one of at least two data centers included in a MongoDB-based data synchronization system, each data center is deployed with an independent MongoDB cluster, and each MongoDB cluster has a unique partition identifier PID; operation log synchronization is performed between each MongoDB cluster; the method comprises:
[0006] receiving a service access request sent by a client; the service access request contains tracking point information; the tracking point information contains a first PID of a MongoDB cluster on which the client performs the latest write operation and a first timestamp;
[0007] perform data integrity verification based on the tracking point information and a local sync point library of the target data center, to obtain a verification result; the local sync point library contains sync point information; the sync point information is used to identify a PID and a timestamp of a MongoDB cluster corresponding to a log synchronization operation;
[0008] In a case where the verification result indicates that the local MongoDB cluster of the target data center contains complete data of the client, a business service corresponding to the service access request of the client is provided.
[0009] In another aspect, a client request processing method is provided, which is performed by a service router in a MongoDB-based data synchronization system; the data synchronization system contains at least two data centers, each of which is deployed with an independent MongoDB cluster, and each of the MongoDB clusters has a unique partition identifier PID; the MongoDB clusters are synchronized in operation logs; the apparatus comprises:
[0010] receiving an address query request sent by a client; the address query request contains tracking point information; the tracking point information contains a first PID and a first timestamp of a MongoDB cluster in which the client performs a latest write operation;
[0011] performing address query based on the tracking point information and a global sync point library, to obtain a target service address; a local MongoDB cluster of a data center corresponding to the target service address contains complete data of the client;
[0012] feeding back the target service address to the client, so that the client performs service access to a corresponding data center based on the target service address.
[0013] In another aspect, a client request processing apparatus is provided, which can be applied to an application layer of a target data center, the target data center being any one of at least two data centers contained in a data synchronization system, each of the data centers being deployed with an independent MongoDB cluster, and each of the MongoDB clusters having a unique partition identifier PID; the MongoDB clusters are synchronized in operation logs; the apparatus comprises:
[0014] a first receiving module, configured to receive a service access request sent by a client; the service access request contains tracking point information; the tracking point information contains a first PID and a first timestamp of a MongoDB cluster in which the client performs a latest write operation;
[0015] a verification module, configured to perform data integrity verification based on the tracking point information and a local sync point library of the target data center, to obtain a verification result; the local sync point library contains sync point information; the sync point information is used to identify a PID and a timestamp of a MongoDB cluster corresponding to a log synchronization operation;
[0016] a service module, configured to provide a business service corresponding to the service access request for the client in a case where the verification result indicates that the local MongoDB cluster of the target data center contains complete data of the client.
[0017] In a possible implementation, the verification module is configured to determine, in a case where it is determined based on the tracking point information and the local sync point library of the target data center that an integrity condition is met, that the verification result indicates that the local MongoDB cluster contains complete data of the client.
[0018] The integrity condition includes:
[0019] the first PID is the same as a second PID of the local MongoDB cluster;
[0020] or,
[0021] in a case where the first PID is different from the second PID, a timestamp identified by first sync point information is greater than the first timestamp; the first sync point information is the latest sync point information between a MongoDB cluster corresponding to the first PID and the local MongoDB cluster, which is extracted from the local sync point library.
[0022] In a possible implementation, the apparatus further includes:
[0023] an instruction sending module, configured to send a redirection instruction to the client in a case where the verification result indicates that the local MongoDB cluster of the target data center does not contain complete data of the client; the redirection instruction is used to instruct the client to reacquire a service address.
[0024] In a possible implementation, the apparatus further includes:
[0025] an updating module, configured to update the tracking point information after completing the business service corresponding to the client.
[0026] a delivery module, configured to deliver the updated tracking point information to the client.
[0027] In a possible implementation, the data synchronization system further contains a service router.
[0028] The first receiving module is configured to receive an access service request sent by the client based on a target service address; the target service address is a service address fed back to the client by the service router based on address query in a global synchronization point library after receiving an address query request sent by the client; the global synchronization point library contains synchronization point information between at least two MongoDB clusters.
[0029] In another aspect, a client request processing apparatus is provided, which can be applied to a service router in a MongoDB-based data synchronization system; the data synchronization system contains at least two data centers, each of which is deployed with an independent MongoDB cluster, and each of the MongoDB clusters has a unique partition identifier (PID); the MongoDB clusters are synchronized in operation logs; the apparatus comprises:
[0030] The second receiving module is configured to receive an address query request sent by the client; the address query request contains tracking point information; the tracking point information contains a first PID of a MongoDB cluster where the client performs a latest write operation and a first timestamp;
[0031] The address query module is configured to perform address query based on the tracking point information and a global synchronization point library to obtain a target service address; the local MongoDB cluster of the data center corresponding to the target service address contains complete data of the client;
[0032] The address feedback module is configured to feed back the target service address to the client, so that the client performs service access to the corresponding data center based on the target service address.
[0033] In a possible implementation, the address query module comprises:
[0034] The verification submodule is configured to perform data integrity verification between each service address and the tracking point information in a priority order of the service addresses based on synchronization point information corresponding to the service addresses;
[0035] The address confirmation submodule is configured to determine a service address corresponding to the second synchronization point information as the target service address in a case where the second synchronization point information and the tracking point information satisfy an integrity condition;
[0036] The integrity condition comprises:
[0037] The first PID is the same as a third PID of a MongoDB cluster identified by the second synchronization point information;
[0038] Or,
[0039] In the case that the first PID is different from the third PID, the timestamp identified by the second synchronization point information is greater than the first timestamp.
[0040] In another aspect, a MongoDB-based data synchronization system is provided, the system comprising: at least two data centers and a service router; each of the data centers is deployed with an independent MongoDB cluster, and each of the MongoDB clusters has a unique partition identification PID;
[0041] A replicator is deployed between each pair of the MongoDB clusters; the replicator is configured to synchronize a target operation log in a source database to a target database, and save synchronization point information of the source database and the target database into a local synchronization point database of the data center where the target database is located and a global synchronization point database corresponding to the service router; the synchronization point information is used to identify a PID of a MongoDB cluster corresponding to a log synchronization operation and a timestamp; the target operation log is an operation log containing the PID of the local MongoDB cluster of the source database in a specific field;
[0042] The service router is configured to provide an address query service for a client based on the global synchronization point database;
[0043] The data center comprises an application layer, and the application layer is configured to provide a service for the client based on received tracking point information and the local synchronization point database in a case that complete data of the client in the local MongoDB cluster is determined to be contained.
[0044] In a possible implementation, the application layer comprises a write operation marker in an application driving layer corresponding to the application layer; the write operation marker is configured to add the PID of the local MongoDB cluster to a specific field of an operation log based on an operation type of a write operation.
[0045] In a possible implementation, in a case that the operation type of the write operation is an insertion operation, the write operation marker is configured to add the PID of the local MongoDB cluster to an object identifier ObjectId of an operation document, and the format of the added ObjectId is a timestamp-PID-process identification-increment counter.
[0046] In another aspect, a computer device is provided, the computer device comprising a processor and a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to implement the above-mentioned client request processing method.
[0047] In another aspect, a computer-readable storage medium is provided, and the computer-readable storage medium stores at least one computer program, the computer program is loaded and executed by a processor to implement the client request processing method described above.
[0048] In another aspect, a computer program product is provided, and the computer program product includes at least one computer program, the computer program is loaded and executed by a processor to implement the client request processing method provided in various optional implementation manners described above.
[0049] The technical solutions provided in the present application can include the following beneficial effects:
[0050] The client request processing method provided in the embodiments of the present application is executed by the application layer of each data center included in the MongoDB-based data synchronization system, each data center has an independent MongoDB cluster, each MongoDB cluster has a unique PID, and the operation logs are synchronized between each MongoDB cluster; after receiving the service access request sent by the client, the data integrity is verified based on the tracking point information included in the service access request and the synchronization point information included in the local synchronization point library of the target data center, and in the case that the verification result indicates that the local MongoDB cluster of the target data center contains complete data of the client, the corresponding business service is provided for the client; in the above scheme, the tracking point information and the local synchronization point library are used to verify the data integrity, which ensures that the latest and consistent data is used when processing the client request, reduces the risk of processing exceptions caused by data delay, and improves the overall reliability of the system; the application layer can provide services for the client based on the local MongoDB cluster, which can reduce the performance problems caused by data synchronization delay, and further improve the service response speed.
[0051] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS
[0052] The accompanying drawings, which are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application and, together with the specification, serve to explain the principles of the present application.
[0053] Figure 1 A schematic diagram of a MongoDB-based data synchronization system provided by an exemplary embodiment of the present application is shown;
[0054] Figure 2 A schematic diagram of a replicator provided by an exemplary embodiment of the present application is shown;
[0055] Figure 3A flow chart of a client request processing method provided by an example embodiment of the present application is shown;
[0056] Figure 4 A flow chart of a client request processing method provided by another example embodiment of the present application is shown;
[0057] Figure 5 A flow chart of a client request processing method provided by another example embodiment of the present application is shown;
[0058] Figure 6 A block diagram of a client request processing apparatus provided by an example embodiment of the present application is shown;
[0059] Figure 7 A block diagram of a client request processing apparatus provided by another example embodiment of the present application is shown;
[0060] Figure 8 A structural block diagram of a computer device shown by an example embodiment of the present application is shown. DETAILED DESCRIPTION
[0061] The example embodiments will be described in detail herein with reference to the drawings. When the following description refers to accompanying drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following example embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims.
[0062] The example embodiments of the present application provide a MongoDB-based data synchronization system, which does not rely on a multi-data center deployment scheme of a MongoDB database, is completely implemented by an application layer, and has the features of multi-active, off-site disaster recovery, user data consistency, and simple application configuration. Figure 1 A schematic diagram of a MongoDB-based data synchronization system provided by an example embodiment of the present application is shown, as shown in Figure 1 The system includes at least two data centers and a service router, each data center is deployed with an independent MongoDB cluster, and each MongoDB cluster has a unique PID (Partition Identifier).
[0063] The replicator is used for synchronizing the target operation log in the source library to the target library, and saving the synchronization point information of the source library and the target library to a local synchronization point library of a data center where the target library is located and a global synchronization point (Sync Point) library corresponding to a service router.
[0064] The source library is a MongoDB cluster generating the target operation log, and the target library is a MongoDB cluster to which the target operation log is to be synchronized. Illustratively, for the MongoDB cluster 1 in the data center 1 and the MongoDB cluster 2 in the data center 2, in the case of synchronizing the target operation log in the MongoDB cluster 1 to the target library, the MongoDB cluster 1 is the source library, and the MongoDB cluster 2 is the target library.
[0065] Since the source library contains operation logs synchronized from other MongoDB clusters in addition to the operation logs (oplog) generated by itself, in order to avoid redundant replication of the operation logs, a filter is included in the replicator, which is used for filtering out the operation logs generated by the source library itself and synchronizing the operation logs to the target library. Illustratively, Figure 2 A schematic diagram of the replicator provided by an example embodiment of the present application is shown as follows, Figure 2As shown, the replicator comprises a pull service component 210, a filter 220 and a replay service component 230; the replicator pulls the operation log from the source database through the pull service component 210, in order to avoid the waste of data processing resources in the subsequent processing process caused by excessive data pulling, the pull service component 210 can obtain the latest synchronization point information corresponding to the source database and the target database in the target database before pulling the operation log from the source database, and when pulling the operation log, the operation log with a timestamp greater than the timestamp of the latest synchronization point information is pulled from the source database; illustratively, if the timestamp of the latest synchronization point information corresponding to the source database and the target database in the target database is time point 1, the replicator pulls the operation log after time point 1 from the source database when pulling the operation log; the filter 220 filters the operation log based on the PID of the operation log to filter out the target operation log from the pulled operation log; in the embodiment of the present application, when generating the operation log, the PID corresponding to the MongoDB cluster can be added to a specific field of the operation log, so that the filter 220 can extract the corresponding PID from the specific field of the operation log when filtering the operation log, and by comparing the PID of the operation log with the PID of the local MongoDB cluster corresponding to the source database, the target operation log containing the PID of the local MongoDB cluster of the source database can be filtered out; the replay service component 230 writes the target operation log into the target database.
[0066] In the embodiment of the present application, the data center comprises an application layer and an application driving layer, wherein the application driving layer comprises a write operation marker; the write operation marker is used to add the PID of the local MongoDB cluster to a specific field of the operation log based on the operation type of the write operation. The write operation comprises an insert operation, an update operation and a delete operation, and further, the specific field corresponding to different write operations is different; wherein for the insert operation, the specific field corresponding to the insert operation can be the primary key field "_id"; for the update, the specific field corresponding to the update can be the document version field "doc_version".
[0067] In the embodiment of the present application, the write operation marker is used to add the PID of the local MongoDB cluster to the object identifier ObjectId of the operation document when the operation type of the write operation is the insert operation, and the format of the added ObjectId is timestamp-PID-process identifier-increment counter. The ObjectID of the MongoDB cluster is a special data type used to identify the primary key (_id) in the MongoDB cluster; the ObjectID provided in the embodiment of the present application is a hexadecimal string composed of 12 bytes, and the format of the ObjectID is as follows:
[0068] The first four bytes: the timestamp accurate to seconds, indicating the time when the ObjectID is created;
[0069] The next two bytes: this part is the globally unique PID allocated by the system for the MongoDB cluster, indicating the identity of the MongoDB cluster that creates the operation log;
[0070] The next three bytes: this part is the identity of the application process, used to identify the application process ID that generates the operation log, and the application process obtains a unique process ID within the cluster from the PID issuing module when it starts;
[0071] The last three bytes: this part is a counter that is recycled within the same application process, used to ensure the uniqueness of each ObjectID within the same process within the same second.
[0072] For example, the operation log corresponding to the insert (Insert) operation is as follows:
[0073]
[0074] Among them, op is equal to i, indicating that the write operation corresponding to the operation log is an insert operation, the value of the _id field under the o field is an ObjectId, and the ObjectId contains the PID of the MongoDB cluster that generates the operation log. In the above example, the value of the byte corresponding to the PID of the MongoDB cluster is "d624".
[0075] In the case where the operation type of the write operation is an update operation, the write operation table and the "PID+operation timestamp" string are saved by defining the doc_version (document version) field in the operation log. Therefore, by querying the doc_version field of the operation log, the PID of the MongoDB cluster can be intercepted, and the MongoDB cluster where the operation log was last updated can be determined.
[0076] For example, the operation log corresponding to the update (update) operation is as follows:
[0077]
[0078]
[0079] Among them, op is equal to u, indicating that the write operation corresponding to the operation log is an update operation, and the doc_version field under the o field contains the PID of the MongoDB cluster that generates the operation log. In the above example, the value of the byte corresponding to the PID of the MongoDB cluster is "1001".
[0080] For the delete operation, no special processing is needed; because the MongoDB cluster does not record the operation log for a failed delete operation, if the MongoDB cluster attempts to delete a non-existent operation log, the delete operation will fail because the operation log does not exist, but this failed operation will not be recorded in the operation log, which means that the MongoDB cluster to be synchronized will not receive this failed delete operation, and thus will not attempt to delete anything to avoid unnecessary data loss. Based on this, no special processing is needed for the delete operation.
[0081] After the write operation table marker adds the PIDs of the MongoDB cluster to the corresponding fields based on the above corresponding write operations of different types, when the replicator performs log filtering through the filter 220, for the captured operation log, if the operation type determined based on the op field is an insert operation or an update operation, the PID corresponding to the operation log is extracted from the corresponding field and compared with the PID of the local MongoDB cluster of the source database, if they are the same, the operation log is passed, otherwise, the operation log is filtered out.
[0082] If the operation type determined based on the op field is a delete operation (i.e., op is equal to d), the operation log is passed.
[0083] If the op field is equal to n, indicating that the current operation log is an empty message, the operation log is filtered.
[0084] If the op field is equal to c, when the o field is followed by an applyOps (apply operation) structure, the applyOps data is parsed and processed according to the above rules, otherwise, the operation log is filtered.
[0085] The replay service component 230 in the replicator synchronizes the target operation log obtained after filtering to the target database; after the synchronization of the target operation log is completed in the target database, the synchronization point information in the local synchronization point database in the data center where the target database is located and the synchronization point information in the global synchronization point database corresponding to the service router are updated; the local point database of the data center stores the synchronization point information corresponding to the data log synchronization process performed by the local MongoDB cluster of the data center; the global synchronization point database stores the synchronization point information of the data synchronization process between each data center in the system.
[0086] Illustratively, the synchronization point information includes the PID of the source database, the PID of the target database, and a timestamp, which is the time when the operation log is generated; illustratively, Table 1 shows an example of the data structure of the synchronization point information stored in each synchronization point database according to an example embodiment of the present application:
[0087] Table 1
[0088]
[0089] The service router is configured to provide address query service for the client based on the global sync point library; the service router is a global service router, responsible for service health monitoring and user service routing.
[0090] Optionally, the service router is further configured to detect the state of each data center application service and MongoDB cluster service and maintain a health service table; illustratively, Table 2 shows an example of the data structure of the health service table provided by an example embodiment of the present application:
[0091] Table 2
[0092]
[0093] Table 3 shows an example of the health service data provided by an example embodiment of the present application:
[0094] Table 3
[0095]
[0096]
[0097] That is, the data center, the service address of the data center, and the PID of the local MongoDB cluster are one-to-one corresponding, and one of them can be determined to obtain the other information.
[0098] The data center includes an application layer, which is configured to provide business services for the client based on the received tracking point information and the local sync point library to determine that the local MongoDB cluster contains complete data of the client.
[0099] Among them, the data integrity checker is deployed in the application layer, and when the data integrity is checked in the application layer, the application layer can determine that the local MongoDB cluster contains complete data of the client based on the received tracking point information and the local sync point library through the data integrity checker;
[0100] It should be noted that, Figure 1 The system contains three data centers as an example, but based on different system settings, the system can contain more or less number of data centers, which is not limited by the present application.
[0101] The MongoDB-based data synchronization system provided in the embodiments of the present application can solve the problems of network delay performance reduction, data inconsistency, complex configuration and management, insufficient coverage, and the like, which exist in the cross-region deployment scheme based on the replica set and sharding mechanism. The data synchronization system provided in the embodiments of the present application can implement the cross-region multi-active deployment based on the application layer, which has the following advantages compared with the cross-region multi-active deployment based on the data layer in the related art.
[0102] Flexibility: The cross-region multi-active deployment based on the application layer can be flexibly configured and customized according to business requirements. The application layer can perform scattered processing and decision-making on data according to specific business logic and rules, thereby better adapting to different requirements and scenarios.
[0103] Scalability: The cross-region multi-active deployment based on the application layer can better support the expansion of the system. By deploying different application instances in multiple geographic locations, traffic and load can be dispersed to different nodes, improving the processing capacity and scalability of the system.
[0104] Elasticity and adaptability: The cross-region multi-active deployment based on the application layer can better cope with network interruption, data synchronization delay and other abnormal situations. When a node fails or the network is interrupted, other nodes can still provide services and automatically synchronize data after recovery, ensuring the availability and stability of the system.
[0105] High performance: The cross-region multi-active deployment based on the application layer only relies on local databases to achieve fast access and processing of data. The application layer can use caching, load balancing and other technologies to improve the performance and response speed of the system.
[0106] User data consistency: The cross-region multi-active deployment based on the application layer ensures the consistency of user data by customizing strategies, checking data integrity and guiding users to nodes with complete data.
[0107] Cloud-native support: The cross-region multi-active deployment based on the application layer solves the problem of insufficient cloud-native support of the original ObjectId algorithm by customizing the format of ObjectId.
[0108] Based on the MongoDB-based data synchronization system as shown in Figure 1 FIG. 1, the embodiments of the present application provide a client request processing method, Figure 3A flow chart of a client request processing method provided by an example embodiment of the present application is shown, which is executed by an application layer of a target data center, the target data center being any one of at least two data centers included in a data synchronization system, each data center being deployed with an independent MongoDB cluster, each MongoDB cluster having a unique partition identifier PID; the MongoDB clusters are synchronized in operation logs; as shown in Figure 3 The method comprises the following steps:
[0109] Step 310, receiving a service access request sent by a client; the service access request contains trace point information; the trace point information contains a first PID of a MongoDB cluster where the client performs a latest write operation and a first timestamp.
[0110] In the case that the client has a service address cached locally or acquires a service address, the client can send a service access request to a data center indicated by the service address according to the cached service address, and correspondingly, the data center receives the service access request sent by the client.
[0111] The service access request contains trace point information, and the trace point information contains a cluster PID of a last write operation of the client and a timestamp, and the format of the trace point information can be customized according to actual needs. The trace point information can be created by an application layer of data of a last write operation of the client and returned to the client.
[0112] Further, in order to reduce information redundancy in the client, only the latest trace point information can be saved in the client, that is, the trace point information is updated and replaced after the client receives the client returned by the application layer.
[0113] Step 320, performing data integrity verification based on the trace point information and a local synchronization point library of the target data center to obtain a verification result; the local synchronization point library contains synchronization point information; the synchronization point information is used to identify a PID and a timestamp of a MongoDB cluster corresponding to a log synchronization operation.
[0114] After the trace point information is extracted from the service access request, the application layer can determine whether the complete data of the client has been synchronized to or exists in the local MongoDB cluster of the current data center through the first PID and the first timestamp in the trace point information, and then determine whether to provide service for the client;
[0115] In the case where the data integrity is determined based on the tracking point information, the application layer can verify based on the synchronization point position information stored in the local synchronization point library in combination with the tracking point information, that is, check whether the synchronization point position information in the local synchronization point library contains an entry (that is, synchronization point position information) matching the first PID and the first timestamp. If the matching entry exists, it indicates that the complete data of the client already exists in the target data center. If the matching entry does not exist, it indicates that the complete data of the client has not been completely synchronized to the target data center.
[0116] In the case where the data integrity is determined based on the tracking point information, the application layer can verify based on the synchronization point position information stored in the local synchronization point library in combination with the tracking point information, that is, check whether the synchronization point position information in the local synchronization point library contains an entry (that is, synchronization point position information) matching the first PID and the first timestamp. If the matching entry exists, it indicates that the complete data of the client already exists in the target data center. If the matching entry does not exist, it indicates that the complete data of the client has not been completely synchronized to the target data center.
[0117] In the case where the data integrity is determined based on the tracking point information, the application layer can verify based on the synchronization point position information stored in the local synchronization point library in combination with the tracking point information, that is, check whether the synchronization point position information in the local synchronization point library contains an entry (that is, synchronization point position information) matching the first PID and the first timestamp. If the matching entry exists, it indicates that the complete data of the client already exists in the target data center. If the matching entry does not exist, it indicates that the complete data of the client has not been completely synchronized to the target data center.
[0118] In the case where the data integrity is determined based on the tracking point information, the application layer can verify based on the synchronization point position information stored in the local synchronization point library in combination with the tracking point information, that is, check whether the synchronization point position information in the local synchronization point library contains an entry (that is, synchronization point position information) matching the first PID and the first timestamp. If the matching entry exists, it indicates that the complete data of the client already exists in the target data center. If the matching entry does not exist, it indicates that the complete data of the client has not been completely synchronized to the target data center.
[0119] In the case where the data integrity is determined based on the tracking point information, the application layer can verify based on the synchronization point position information stored in the local synchronization point library in combination with the tracking point information, that is, check whether the synchronization point position information in the local synchronization point library contains an entry (that is, synchronization point position information) matching the first PID and the first timestamp. If the matching entry exists, it indicates that the complete data of the client already exists in the target data center. If the matching entry does not exist, it indicates that the complete data of the client has not been completely synchronized to the target data center. Figure 4A flow chart of a client request processing method provided by another example embodiment of the application is shown, which is executed by a service router in a MongoDB-based data synchronization system; the data synchronization system includes at least two data centers, each of which is deployed with an independent MongoDB cluster, and each MongoDB cluster has a unique partition identifier (PID); the operation logs are synchronized between the MongoDB clusters, as shown in Figure 4 The method can include the following steps:
[0120] Step 410, receiving an address query request sent by a client; the address query request includes tracking point information; the tracking point information includes a first PID of a MongoDB cluster where the client performs a latest write operation and a first timestamp.
[0121] When the service address cached locally by the client is empty or has expired, the client can send an address query request carrying the tracking point information to the service router to obtain the service address from the service router and cache it locally; correspondingly, the service router receives the address query request, performs service address query based on the address query request, and feeds back the query result to the client.
[0122] Step 420, performing address query based on the tracking point information and a global synchronization point library to obtain a target service address; the target service address corresponds to a local MongoDB cluster of a data center that contains complete data of the client.
[0123] The global synchronization library includes synchronization point information between the MongoDB clusters, and the synchronization point information includes a PID of a source library, a PID of a target library, and a timestamp; the service router extracts the tracking point information from the address query request to query the synchronization point information in the global synchronization point library based on the tracking point information, checks whether the synchronization point information in the global synchronization point library includes an entry (i.e., synchronization point information) matching the first PID and the first timestamp, i.e., determines which local MongoDB cluster of a data center contains complete data of the client; wherein the matching entry indicates that the complete data corresponding to the client already exists in the MongoDB cluster of the data center corresponding to the synchronization point information, and the service address of the data center corresponding to the synchronization point information is determined as the target service address.
[0124] In one possible implementation, the service router can perform address query in the global synchronization point library in a certain order, and after the target service address is queried, the query operation can be stopped and the target service address is fed back, so as to reduce resource waste caused by unnecessary query operations.
[0125] Step 430, the target service address is fed back to the client, so that the client accesses the corresponding data center based on the target service address.
[0126] The service router feeds back the target service address to the client, and correspondingly, the client receives the target service address and caches it locally to access the corresponding data center based on the target service address.
[0127] In summary, the client request processing method provided by the embodiments of the application is executed by a service router in a MongoDB-based data synchronization system, the data synchronization system includes at least two data centers, each data center is deployed with an independent MongoDB cluster, and each MongoDB cluster has a unique partition identifier PID; the operation logs are synchronized between each MongoDB cluster; after receiving an address query request sent by a client, the tracking point information contained in the address query request is matched with the synchronization point information contained in the global synchronization point library, the service address of the data center corresponding to the matched synchronization point information is determined as a target service address, and the target service address is fed back to the client, so that the client performs subsequent service access based on the target service address; in the above scheme, by using the tracking point information in combination with each synchronization point information in the global synchronization point library for address query, the service address of the data center containing complete data of the client (i.e. the target service address) can be queried, so that the data center accessed by the client based on the target service address can provide services for the client based on the latest and consistent complete data, thereby reducing the risk of processing exceptions caused by data delay and improving the overall reliability of the system.
[0128] Taking a service access process performed in a case where the service address locally cached by the client is empty or has expired as an example, Figure 5 a flowchart of a client request processing method provided by another exemplary embodiment of the application is shown, the method can be executed by a MongoDB-based data synchronization system, the data synchronization system includes at least two data centers, each data center is deployed with an independent MongoDB cluster, and each MongoDB cluster has a unique partition identifier PID; the operation logs are synchronized between each MongoDB cluster; as Figure 5 shown, the method includes an address query phase and a service access phase;
[0129] In the address query phase, the interaction between the client and the service router is involved, and the process includes:
[0130] Step 510, the service router receives the address query request sent by the client; the address query request contains tracking point information; the tracking point information contains the first PID of the MongoDB cluster where the client performs the latest write operation and the first timestamp.
[0131] Step 520, the service router performs address query based on the tracking point information and the global synchronization point library to obtain a target service address; the target service address corresponds to a local MongoDB cluster in a data center that contains complete data of the client.
[0132] In a possible implementation, the process of address query can be implemented as follows:
[0133] According to the priority order of each service address, the synchronization point information corresponding to each service address is sequentially verified with the tracking point information for data integrity.
[0134] In a case where the second synchronization point information and the tracking point information satisfy the integrity condition, the service address corresponding to the second synchronization point information is determined as the target service address.
[0135] The priority order can be a service priority defined by business; alternatively, the priority order can be defined according to the distance between the client and the data center, and the closer the physical distance between the client and the data center, the higher the priority of the corresponding service address; or, the priority order can be defined according to the province to which the client belongs, and the clients in the same province have a unified service address priority list; Table 4 shows an example of the priority list of the service address provided by an example embodiment of the present application.
[0136] Table 4
[0137] Priority (high to low) Service address PID 1 https: / / 10.10.1.5:9443 / portal / 5 2 https: / / 10.10.1.1:7443 / portal / 1 3 https: / / 10.10.1.2:8443 / portal / 2
[0138] When performing address query based on the tracking point information and the global synchronization point library, the synchronization point information corresponding to each service address is sequentially obtained according to the priority of each service address to verify the data integrity, so as to confirm whether the MongoDB cluster corresponding to the service address contains complete data of the client; if it is determined that the current MongoDB cluster contains complete data of the client, the corresponding service address of the client is immediately returned.
[0139] Wherein, whether the MongoDB cluster has the complete data of the client can be judged according to the tracking point information and the synchronization point information between the clusters; the service router queries the synchronization point information from the global synchronization point library, which is from the MongoDB cluster of the last write operation of the client (namely, the source library) to the MongoDB cluster corresponding to the service address (namely, the target library), and determines that the MongoDB cluster contains the complete data of the client by comparing the two and determining that the integrity condition is met.
[0140] Wherein, the integrity condition comprises:
[0141] The first PID is the same as the third PID of the MongoDB cluster identified by the second synchronization point information;
[0142] Or,
[0143] In the case that the first PID is different from the third PID, the timestamp identified by the second synchronization point information is greater than the first timestamp.
[0144] In the case that the first PID is the same as the third PID of the MongoDB cluster identified by the second synchronization point information, it indicates that the MongoDB cluster corresponding to the second synchronization point information is the cluster of the last write operation of the client, and the MongoDB cluster contains the complete data of the client; in the case that the first PID is different from the third PID, it indicates that the MongoDB cluster corresponding to the second synchronization point information is not the cluster of the last write operation of the client, and the timestamp is judged at this time, if the timestamp identified by the second synchronization point information is greater than the first timestamp, it indicates that the MongoDB cluster of the user write operation and the MongoDB cluster corresponding to the second synchronization point information have completed the synchronization of the data of the client, and the MongoDB cluster corresponding to the second synchronization point information also contains the complete data of the client; if the timestamp identified by the second synchronization point is less than the first timestamp, it indicates that the complete data of the client has not been synchronized in the current MongoDB cluster, and the service router can continue the data integrity verification process of the next service address according to the priority order of the service address, until the synchronization point information meeting the integrity condition is obtained, and the service address corresponding to the synchronization point information is determined as the target service address.
[0145] Step 530, the service router feeds back the target service address to the client, so that the client accesses the corresponding data center for service based on the target service address.
[0146] In the service access stage, the interaction between the client and the application layer of the target data center, which can be the data center indicated by the target service address, is involved; however, due to the possibility of data tampering, forgery and the like in the interaction between the service router and the client, in order to further improve the security of service access, the application layer will verify again whether the complete data of the client is stored in the local MongoDB cluster after receiving the service access request, and determine whether to accept the user service access request based on the verification result; the process includes:
[0147] In step 540, the application layer receives the service access request sent by the client; the service access request contains tracking point information; the tracking point information contains the first PID of the MongoDB cluster on which the client performs the latest write operation and the first timestamp.
[0148] The client carries the tracking point information to access the application service according to the target service address specified by the service router, and correspondingly, the application layer receives the access service request sent by the client based on the target service address; the target service address is the service address fed back to the client by the service router after address query in the global synchronization point library based on the address query request sent by the client; the global synchronization point library contains synchronization point information between at least two MongoDB clusters.
[0149] In step 550, data integrity verification is performed based on the tracking point information and the local synchronization point library of the target data center to obtain a verification result; the local synchronization point library contains synchronization point information; the synchronization point information is used to identify the PID and timestamp of the MongoDB cluster corresponding to the log synchronization operation.
[0150] In one possible implementation, the process of data integrity verification can be implemented as follows:
[0151] In the case where the tracking point information and the local synchronization point library of the target data center satisfy the integrity condition, it is determined that the local MongoDB cluster contains the complete data of the client.
[0152] The application layer queries the synchronization point information from the client's last write operation cluster (i.e. source library) to the local MongoDB cluster (i.e. target library) from the local synchronization point library, and in the case where the synchronization point information and the tracking point information satisfy the integrity condition, it is determined that the local MongoDB cluster contains the complete data of the client; wherein the integrity condition includes:
[0153] The first PID is the same as the second PID of the local MongoDB cluster;
[0154] Or,
[0155] In the case that the first PID is different from the second PID, the timestamp indicated by the first synchronization point information is greater than the first timestamp; the first synchronization point information is the latest synchronization point information between the MongoDB cluster corresponding to the first PID extracted from the local synchronization point library and the local MongoDB cluster.
[0156] In the case that the first PID is the same as the second PID of the local MongoDB cluster, it indicates that the local MongoDB cluster is the cluster where the client last performs the write operation, and the MongoDB cluster contains complete data of the client; in the case that the first PID is different from the second PID, it indicates that the local MongoDB cluster is not the cluster where the client last performs the write operation, and at this time, the timestamp is judged. If the timestamp indicated by the first synchronization point information is greater than the first timestamp, it indicates that the MongoDB cluster where the user performs the write operation and the local MongoDB cluster have completed the synchronization of the data of the client, and the local MongoDB cluster also contains the latest complete data of the client; if the timestamp indicated by the first synchronization point is less than the first timestamp, it indicates that the local MongoDB cluster has not synchronized the complete data of the client.
[0157] Step 560, in the case that the verification result indicates that the local MongoDB cluster of the target data center contains the complete data of the client, providing the service access request corresponding to the business service for the client.
[0158] That is, in the case that it is determined that the local MongoDB cluster of the target data center contains the latest complete data of the client, the application layer of the target data center accepts the service access request sent by the client, and returns a processing result to the client,
[0159] In the case that the local MongoDB cluster of the target data center does not contain the latest complete data of the client, the application layer of the target data center rejects the service access request sent by the client.
[0160] In a possible implementation, the method further includes:
[0161] In the case that the verification result indicates that the local MongoDB cluster of the target data center does not contain the complete data of the client, a redirection instruction is sent to the client; the redirection instruction is used to instruct the client to reacquire a service address.
[0162] For a CS (Client / Server Architecture) architecture application, the redirection instruction can be a redirection error code, and the application layer returns the redirection error code to the client. After receiving the redirection error code, the client re-requests an available address from the router. For a BS (Browser / Server Architecture) architecture application, the request is redirected by the client browser to the service router, and the service router is redirected by the client browser to an available service address.
[0163] Further, in the case where the application layer receives the service access request of the client, the method further comprises:
[0164] updating the tracking point information after completing the business service corresponding to the client;
[0165] issuing the updated tracking point information to the client.
[0166] That is, after completing the business service corresponding to the client, the application layer creates and issues new tracking point information to indicate that the MongoDB cluster where the client last performs a write operation is the current MongoDB cluster, so that the client can perform service access based on the updated tracking point information in subsequent service access requests.
[0167] In summary, the client request processing method provided by the embodiments of the present application is executed by a MongoDB-based data synchronization system, the data synchronization system includes at least two data centers, each data center has an independent MongoDB cluster, each MongoDB cluster has a unique PID, and the operation logs are synchronized between the MongoDB clusters; wherein, after receiving an address query request sent by a client, a service router in the data synchronization system matches the tracking point information contained in the address query request with the synchronization point information contained in the global synchronization point library, determines the service address of the data center corresponding to the matched synchronization point information as a target service address, and feeds back the target service address to the client; after receiving a service access request sent by the client, an application layer of the target data center performs data integrity verification based on the tracking point information contained in the service access request and the synchronization point information contained in the local synchronization point library of the target data center, and provides corresponding business services for the client in the case that the verification result indicates that the local MongoDB cluster of the target data center contains complete data of the client; in the above scheme, the tracking point information and the local synchronization point library are used to verify the data integrity, so that the latest and consistent data is used when processing the client request, the risk of processing exception caused by data delay is reduced, and the overall reliability of the system is improved; the application layer can provide services for the client based on the local MongoDB cluster, the performance problem caused by data synchronization delay can be reduced, and the service response speed can be improved.
[0168] Figure 6 A block diagram of a client request processing apparatus provided by an example embodiment of the present application is shown, the apparatus can be applied in an application layer of a target data center, the target data center is any one of at least two data centers included in a data synchronization system, each data center is deployed with an independent MongoDB cluster, each MongoDB cluster has a unique partition identifier PID; the MongoDB clusters are synchronized in operation logs; as shown in Figure 6 the apparatus includes:
[0169] A first receiving module 610 is configured to receive a service access request sent by a client; the service access request contains tracking point information; the tracking point information contains a first PID of a MongoDB cluster on which the client performs a latest write operation and a first timestamp;
[0170] A verification module 620 is configured to perform data integrity verification based on the tracking point information and a local synchronization point library of the target data center, and obtain a verification result; the local synchronization point library contains synchronization point information; the synchronization point information is used to identify the PID and the timestamp of the MongoDB cluster corresponding to the log synchronization operation.
[0171] The service module 630 is configured to provide a service corresponding to the service access request for the client in a case where the verification result indicates that the local MongoDB cluster of the target data center contains complete data of the client.
[0172] In a possible implementation, the verification module 620 is configured to determine that the verification result indicates that the local MongoDB cluster contains complete data of the client in a case where it is determined that the completeness condition is met based on the tracking point information and the local synchronization point library of the target data center.
[0173] The completeness condition includes:
[0174] The first PID is the same as a second PID of the local MongoDB cluster.
[0175] Or,
[0176] In a case where the first PID is different from the second PID, a timestamp identified by first synchronization point information is greater than the first timestamp; the first synchronization point information is the latest synchronization point information between a MongoDB cluster corresponding to the first PID and the local MongoDB cluster and extracted from the local synchronization point library.
[0177] In a possible implementation, the apparatus further includes:
[0178] The instruction sending module is configured to send a redirection instruction to the client in a case where the verification result indicates that the local MongoDB cluster of the target data center does not contain complete data of the client; the redirection instruction is used to instruct the client to reacquire a service address.
[0179] In a possible implementation, the apparatus further includes:
[0180] The updating module is configured to update the tracking point information after completing the service corresponding to the client.
[0181] The delivery module is configured to deliver the updated tracking point information to the client.
[0182] In a possible implementation, the data synchronization system further contains a service router.
[0183] The first receiving module is configured to receive an access service request sent by the client based on a target service address; the target service address is a service address fed back to the client by the service router based on address query in a global synchronization point library after receiving an address query request sent by the client; and the global synchronization point library contains synchronization point information between at least two MongoDB clusters.
[0184] Figure 7 A block diagram of a client request processing apparatus provided by another example embodiment of the present application is shown, which can be applied to a service router in a MongoDB-based data synchronization system; the data synchronization system contains at least two data centers, each of which is deployed with an independent MongoDB cluster, each of which has a unique partition identifier PID; and each of the MongoDB clusters is synchronized in operation logs; as shown in the figure, the apparatus comprises: Figure 7
[0185] The second receiving module 710 is configured to receive an address query request sent by the client; the address query request contains tracking point information; and the tracking point information contains a first PID of a MongoDB cluster where the client performs a latest write operation and a first timestamp;
[0186] The address query module 720 is configured to perform address query based on the tracking point information and a global synchronization point library to obtain a target service address; and the local MongoDB cluster of the data center corresponding to the target service address contains complete data of the client;
[0187] The address feedback module 730 is configured to feed back the target service address to the client, so that the client performs service access to the corresponding data center based on the target service address.
[0188] In a possible implementation, the address query module 720 comprises:
[0189] The verification submodule is configured to perform data integrity verification between each service address corresponding synchronization point information and the tracking point information in turn according to the priority order of each service address;
[0190] The address confirmation submodule is configured to determine the service address corresponding to the second synchronization point information as the target service address in the case where it is determined that the second synchronization point information and the tracking point information satisfy the integrity condition;
[0191] The integrity condition comprises:
[0192] The first PID is the same as a third PID of the MongoDB cluster identified by the second synchronization point information;
[0193] Or,
[0194] In a case where the first PID is different from the third PID, the timestamp identified by the second synchronization point information is greater than the first timestamp.
[0195] Figure 8 A structural block diagram of a computer device 800 is shown, which is an example embodiment of the present application. The computer device can be implemented as a data center or a service router in the above-mentioned solutions of the present application. The computer device 800 includes a central processing unit (CPU) 801, a system memory 804 including a random access memory (RAM) 802 and a read-only memory (ROM) 803, and a system bus 805 connecting the system memory 804 and the central processing unit 801. The computer device 800 further includes a mass storage device 806 for storing an operating system 809, application programs 810 and other program modules 811.
[0196] Without loss of generality, the computer readable medium can include computer storage media and communication media. The computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. The computer storage media includes RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) flash or other solid state storage technology, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. Of course, those skilled in the art can know that the computer storage medium is not limited to the above several. The system memory 804 and the mass storage device 806 mentioned above can be collectively referred to as memory.
[0197] According to various embodiments of the present disclosure, the computer device 800 can further operate connected to a network on a remote computer through a network, such as the Internet. That is, the computer device 800 can be connected to a network 808 through a network interface unit 807 connected to the system bus 805, or can be connected to other types of networks or remote computer systems (not shown) using the network interface unit 807.
[0198] The memory further includes at least one instruction, at least one program, a code set or an instruction set stored in the memory, and the central processing unit 801 implements all or part of the steps of the client request processing method shown in each of the above embodiments by executing the at least one instruction, at least one program, code set or instruction set.
[0199] In an exemplary embodiment, a computer readable storage medium is also provided, in which at least one computer program is stored, the computer program is loaded and executed by the processor to implement all or part of the steps of the above client request processing method. For example, the computer readable storage medium can be a Read-Only Memory (ROM), a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk and an optical data storage device, etc.
[0200] In an exemplary embodiment, a computer program product is also provided, which includes at least one computer program, the computer program is loaded and executed by the processor to implement all or part of the steps of the client request processing method shown in any of the above embodiments. Figure 3 、 Figure 4 or Figure 5 any of the above embodiments.
[0201] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims.
[0202] It should be understood that the application is not limited to the precise construction that has been described above and shown in the accompanying drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application is limited only by the appended claims.
Claims
1. A client request processing method, characterized in that, The method is executed by the application layer of the target data center, which is any one of at least two data centers included in a MongoDB-based data synchronization system. Each data center deploys an independent MongoDB cluster, and each MongoDB cluster has a unique partition identifier (PID). Operation logs are synchronized between the various MongoDB clusters. The method includes: Receive a service access request sent by a client; the service access request contains trace point information; the trace point information contains the first PID and first timestamp of the MongoDB cluster where the client last performed a write operation. Data integrity is verified based on the tracking point information and the local synchronization point library of the target data center to obtain the verification result; the local synchronization point library contains synchronization point information; the synchronization point information is used to identify the PID and timestamp of the MongoDB cluster corresponding to the log synchronization operation. If the verification result indicates that the local MongoDB cluster of the target data center contains the complete data of the client, the business service corresponding to the service access request is provided to the client.
2. The method according to claim 1, characterized in that, The data integrity verification is performed based on the tracking point information and the local synchronization point library of the target data center to obtain the verification results, including: If the integrity conditions are met based on the tracking point information and the local synchronization point library of the target data center, the verification result indicates that the local MongoDB cluster contains complete data of the client. The integrity conditions include: The first PID is the same as the second PID of the local MongoDB cluster; or, When the first PID is different from the second PID, the timestamp of the first synchronization point information identifier is greater than the first timestamp; the first synchronization point information is the latest synchronization point information between the MongoDB cluster corresponding to the first PID and the local MongoDB cluster extracted from the local synchronization point database.
3. The method according to claim 1, characterized in that, The method further includes: If the verification result indicates that the local MongoDB cluster in the target data center does not contain complete data of the client, a redirection instruction is sent to the client; the redirection instruction is used to instruct the client to reacquire the service address.
4. The method according to claim 1, characterized in that, The method further includes: After completing the business service corresponding to the client, update the tracking point information; The updated tracking point information is sent to the client.
5. The method according to any one of claims 1 to 4, characterized in that, The data synchronization system also includes a service router; The receiving of service access requests sent by the client includes: Receive the service access request sent by the client based on the target service address; The target service address is the service address that the service router returns to the client after performing an address lookup in the global synchronization point database based on the address query request sent by the client; the global synchronization point database contains synchronization point information between at least two MongoDB clusters.
6. A client request processing method, characterized in that, The method is executed by a service router in a MongoDB-based data synchronization system; the data synchronization system includes at least two data centers, each data center deploying an independent MongoDB cluster, and each MongoDB cluster having a unique partition identifier (PID); operation logs are synchronized between the various MongoDB clusters; the method includes: Receive an address query request sent by the client; the address query request contains tracking point information; the tracking point information contains the first PID and first timestamp of the MongoDB cluster where the client last performed a write operation; Based on the tracking point information and the global synchronization point database, an address query is performed to obtain the target service address; the local MongoDB cluster of the data center corresponding to the target service address contains the complete data of the client; The target service address is fed back to the client so that the client can access the corresponding data center for services based on the target service address.
7. The method according to claim 6, characterized in that, The step of obtaining the target service address by querying the address based on the tracking point information and the global synchronization point database includes: According to the priority order of each service address, the data integrity is verified sequentially with the tracking point information based on the synchronization point information corresponding to each service address. If it is determined that the second synchronization point information and the tracking point information meet the integrity condition, the service address corresponding to the second synchronization point information is determined as the target service address; The integrity conditions include: The first PID is the same as the third PID of the MongoDB cluster identified by the second synchronization point information; or, When the first PID and the third PID are different, the timestamp of the second synchronization point information identifier is greater than the first timestamp.
8. A data synchronization system based on MongoDB, characterized in that, The system includes: at least two data centers and a service router; each data center is deployed with an independent MongoDB cluster, and each MongoDB cluster has a unique partition identifier (PID); The MongoDB clusters are configured with replicators in pairs. These replicators synchronize target operation logs from the source database to the target database, and save the synchronization point information between the source and target databases to the local synchronization point database of the data center where the target database is located and the global synchronization point database corresponding to the service router. The synchronization point information identifies the PID and timestamp of the MongoDB cluster corresponding to the log synchronization operation. The target operation log is an operation log containing the PID of the local MongoDB cluster of the source database in a specific field. The service router is used to provide address query services to clients based on the global synchronization point database; The data center includes an application layer, which provides business services to the client based on the received tracking point information and the local synchronization point library, which determines that the local MongoDB cluster contains complete data of the client.
9. The system according to claim 8, characterized in that, The application driver layer corresponding to the application layer includes a write operation marker; the write operation marker is used to add the PID of the local MongoDB cluster to a specific field of the operation log based on the operation type of the write operation.
10. The system according to claim 9, characterized in that, When the write operation is an insert operation, the write operation marker is used to add the PID of the local MongoDB cluster to the object identifier ObjectId of the operation document. The format of the added ObjectId is timestamp-PID-process identifier-increment counter.
11. A client request processing apparatus, characterized in that, The device is applied to the application layer of a target data center, which is any one of at least two data centers included in a MongoDB-based data synchronization system. Each data center deploys an independent MongoDB cluster, and each MongoDB cluster has a unique partition identifier (PID). Operation logs are synchronized between the various MongoDB clusters. The device includes: The first receiving module is used to receive a service access request sent by the client; the service access request includes trace point information; the trace point information includes the first PID and first timestamp of the MongoDB cluster in which the client last performed a write operation. The verification module is used to perform data integrity verification based on the tracking point information and the local synchronization point library of the target data center, and obtain the verification result; the local synchronization point library contains synchronization point information; the synchronization point information is used to identify the PID and timestamp of the MongoDB cluster corresponding to the log synchronization operation. The service module is used to provide the client with the business service corresponding to the service access request when the verification result indicates that the local MongoDB cluster of the target data center contains the complete data of the client.
12. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one computer program, which is loaded and executed by the processor to implement the client request processing method as described in any one of claims 1 to 7.
13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to implement the client request processing method as described in any one of claims 1 to 7.
14. A computer program product, characterized in that, The computer program product includes at least one computer program, which is loaded and executed by a processor to implement the client request processing method as described in any one of claims 1 to 7.
Citation Information
Patent Citations
Method and device for synchronizing account information in cluster system, and storage medium
CN108881231A
Distributed high-fault-tolerance data real-time synchronization method from MongoDB to HBase
CN110287251A