Client request processing method and data synchronization system
By using tracking point information and local synchronous point library to verify data integrity in MongoDB's multi-data center off-site deployment solution, data inconsistency caused by latency between nodes is solved, and system reliability and response speed is improved.
Patent Information
- Application Number
- CN202411144538.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-08-20
AI Technical Summary
In the MongoDB replica set and sharding scheme deployed offsite in multi-data centers, inter-node delays lead to data inconsistency, resulting in client request processing exceptions.
By using tracking point information and local synchronous point library at the application layer of the target data center for data integrity verification, ensuring that the use of up-to-date and consistent data is used, reducing the risk of processing exceptions due to data latency.
Improves overall reliability of the system, reduces performance problems caused by data synchronization latency, and improves service response speed.
Smart Images

Figure CN119094176A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of cloud computing technology, and in particular to a client request processing method and a data synchronization system. Background Art
[0002] In today's cloud computing environment, multi-data center remote active deployment is mainly to improve system reliability, availability, disaster recovery, performance and scalability, as well as to meet regulatory requirements for data location. MongoDB is currently the most popular distributed document-based NoSql database. The multi-data center deployment solution officially recommended by MongoDB mainly uses MongoDB's replica sets and sharding functions.
[0003] However, using MongoDB's replica set and sharding functions for remote deployment will cause latency issues between different nodes, and data inconsistency due to latency, resulting in request processing exceptions when processing client requests. Summary of the invention
[0004] The embodiment of the present application provides a client request processing method and a data synchronization system, which uses tracking point information and a local synchronization point library to confirm data integrity, thereby ensuring that the latest and consistent data is used when processing client requests, reducing the risk of processing exceptions due to data delays, and improving the overall reliability of the system. The technical solution is as follows.
[0005] On the one hand, a client request processing method is provided, the method is executed by an application layer of a target data center, the target data center is any one of at least two data centers included in a MongoDB-based data synchronization system, each of the data centers is deployed with an independent MongoDB cluster, and each of the MongoDB clusters has a unique partition identifier PID; operation logs are synchronized between the MongoDB clusters; the method includes:
[0006] Receive a service access request sent by a client; the service access request includes tracking point information; the tracking point information includes a first PID and a first timestamp of a MongoDB cluster on which the client has performed a latest write operation;
[0007] Perform data integrity verification based on the tracking point information and the local synchronization point library of the target data center to obtain a verification result; the local synchronization point library includes synchronization point information; the synchronization point information is used to identify the PID and timestamp of the MongoDB cluster corresponding to the log synchronization operation;
[0008] When the verification result indicates that the local MongoDB cluster of the target data center contains complete data of the client, the business service corresponding to the service access request is provided to the client.
[0009] On the other hand, a method for processing client requests is provided, the method being executed by a service router in a data synchronization system based on MongoDB; the data synchronization system comprises at least two data centers, each of which is deployed with an independent MongoDB cluster, and each of which has a unique partition identifier PID; operation logs are synchronized between the MongoDB clusters; the device comprises:
[0010] Receive an address query request sent by a client; the address query request includes tracking point information; the tracking point information includes a first PID and a first timestamp of a MongoDB cluster on which the client has performed a latest write operation;
[0011] Performing address query based on the tracking point information and the global synchronization point library to obtain the target service address; the local MongoDB cluster of the data center corresponding to the target service address contains the complete data of the client;
[0012] The target service address is fed back to the client, so that the client accesses the service of the corresponding data center based on the target service address.
[0013] On the other hand, a client request processing device is provided, which can be applied to the application layer of a target data center, where the target data center is any one of at least two data centers included in a data synchronization system, each data center is deployed with an independent MongoDB cluster, and each MongoDB cluster has a unique partition identifier PID; operation logs are synchronized between MongoDB clusters; the device includes:
[0014] A first receiving module, configured to receive a service access request sent by a client; the service access request includes tracking point information; the tracking point information includes a first PID and a first timestamp of the MongoDB cluster where the client performs the latest write operation;
[0015] A verification module, configured to perform data integrity verification based on the tracking point information and the local synchronization point library of the target data center to obtain a verification result; the local synchronization point library includes synchronization point information; the synchronization point information is used to identify the PID and timestamp of the MongoDB cluster corresponding to the log synchronization operation;
[0016] A service module is used to provide the client with a business service corresponding to the service access request when the verification result indicates that the local MongoDB cluster of the target data center contains complete data of the client.
[0017] In a possible implementation, the verification module is used to determine that the verification result indicates that the local MongoDB cluster contains complete data of the client when it is determined that the integrity condition is satisfied based on the tracking point information and the local synchronization point library of the target data center;
[0018] The integrity conditions include:
[0019] The first PID is the same as the second PID of the local MongoDB cluster;
[0020] or,
[0021] When the first PID is different from the second PID, the timestamp identified by the first synchronization point information is greater than the first timestamp; the first synchronization point information is the latest synchronization point information between the MongoDB cluster corresponding to the first PID and the local MongoDB cluster extracted from the local synchronization point library.
[0022] In a possible implementation manner, the device further includes:
[0023] An instruction sending module is used to send a redirection instruction to the client when the verification result indicates that the local MongoDB cluster of the target data center does not contain the complete data of the client; the redirection instruction is used to instruct the client to re-acquire the service address.
[0024] In a possible implementation manner, the device further includes:
[0025] An updating module, used for updating the tracking point information after completing the business service corresponding to the client;
[0026] The sending module is used to send the updated tracking point information to the client.
[0027] In a possible implementation, the data synchronization system further includes a service router;
[0028] The first receiving module is used to receive an access service request sent by the client based on a target service address; the target service address is a service address that is fed back to the client by the service router after performing an address query in a global synchronization point library based on an address query request sent by the client; the global synchronization point library contains synchronization point information between at least two MongoDB clusters.
[0029] On the other hand, a client request processing device is provided, which can be applied to a service router in a data synchronization system based on MongoDB; the data synchronization system includes at least two data centers, each of which is deployed with an independent MongoDB cluster, and each of the MongoDB clusters has a unique partition identifier PID; operation logs are synchronized between the MongoDB clusters; the device includes:
[0030] A second receiving module is used to receive an address query request sent by a client; the address query request includes tracking point information; the tracking point information includes a first PID and a first timestamp of the MongoDB cluster where the client performs the latest write operation;
[0031] An address query module is used to perform address query based on the tracking point information and the global synchronization point library to obtain a target service address; the local MongoDB cluster of the data center corresponding to the target service address contains the complete data of the client;
[0032] The address feedback module is used to feed back the target service address to the client, so that the client can access the service of the corresponding data center based on the target service address.
[0033] In a possible implementation, the address query module includes:
[0034] A verification submodule, for performing data integrity verification based on the synchronization point information corresponding to each service address and the tracking point information in sequence according to the priority order of each service address;
[0035] An address confirmation submodule, configured to determine the service address corresponding to the second synchronization point information as the target service address when it is determined that the second synchronization point information and the tracking point information meet the integrity condition;
[0036] The integrity conditions include:
[0037] The first PID is the same as the third PID of the MongoDB cluster identified by the second synchronization point information;
[0038] or,
[0039] When the first PID is different from the third PID, the timestamp identified by the second synchronization point information is greater than the first timestamp.
[0040] On the other hand, a data synchronization system based on MongoDB is provided, the system comprising: at least two data centers and a service router; each of the data centers is deployed with an independent MongoDB cluster, and each of the MongoDB clusters has a unique partition identifier PID;
[0041] A replicator is deployed between each of the MongoDB clusters; the replicator is used to synchronize the target operation log in the source database to the target database, and save the synchronization point information of the source database and the target database to the local synchronization point database of the data center where the target database is located and the global synchronization point database corresponding to the service router; the synchronization point information is used to identify the PID and timestamp of the MongoDB cluster corresponding to the log synchronization operation; the target operation log is an operation log containing the PID of the local MongoDB cluster of the source database in a specific field;
[0042] The service router is used to provide address query services for the client based on the global synchronization point library;
[0043] The data center includes an application layer, and the application layer is used to provide business services for the client when it is determined that the local MongoDB cluster contains complete data of the client based on the received tracking point information and the local synchronization point library.
[0044] In a possible implementation, the application driver layer corresponding to the application layer includes a write operation marker; the write operation marker is used to add the PID of the local MongoDB cluster to a specific field of the operation log based on the operation type of the write operation.
[0045] In a possible implementation, when the operation type of the write operation is an insert operation, the write operation marker is used to add the PID of the local MongoDB cluster to the object identifier ObjectId of the operation document, and the format of the added ObjectId is timestamp-PID-process identifier-increment counter.
[0046] On the other hand, a computer device is provided, the computer device comprising a processor and a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to implement the above-mentioned client request processing method.
[0047] On the other hand, a computer-readable storage medium is provided, wherein at least one computer program is stored in the computer-readable storage medium, and the computer program is loaded and executed by a processor to implement the above-mentioned client request processing method.
[0048] On the other hand, a computer program product is provided, the computer program product comprising at least one computer program, the computer program being loaded and executed by a processor to implement the client request processing method provided in the above-mentioned various optional implementations.
[0049] The technical solution provided by this application may have the following beneficial effects:
[0050] The client request processing method provided in the embodiment of the present application is executed by the application layer of each data center included in the data synchronization system based on MongoDB, each data center has an independent MongoDB cluster, each MongoDB cluster has a unique PID, and operation logs are synchronized between each MongoDB cluster; after receiving the service access request sent by the client, data integrity verification is performed based on the tracking point information contained in the service access request and the synchronization point information contained in the local synchronization point library of the target data center, and when the verification result indicates that the local MongoDB cluster of the target data center contains the complete data of the client, the client is provided with corresponding business services; in the above scheme, data integrity confirmation is performed by using the tracking point information and the local synchronization point library, which ensures that the latest and consistent data is used when processing client requests, reduces the risk of processing exceptions due to data delays, and improves the overall reliability of the system; the application layer can provide services to the client based on the local MongoDB cluster, can reduce performance problems caused by data synchronization delays, and can also improve service response speed.
[0051] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0053] Figure 1 A schematic diagram of a MongoDB-based data synchronization system provided by an exemplary embodiment of the present application is shown;
[0054] Figure 2 A schematic diagram of a replicator provided by an exemplary embodiment of the present application is shown;
[0055] Figure 3A flowchart of a client request processing method provided by an exemplary embodiment of the present application is shown;
[0056] Figure 4 A flowchart of a method for processing client requests provided by another exemplary embodiment of the present application is shown;
[0057] Figure 5 A flowchart of a method for processing client requests provided by another exemplary embodiment of the present application is shown;
[0058] Figure 6 A block diagram of a client request processing device provided by an exemplary embodiment of the present application is shown;
[0059] Figure 7 A block diagram showing a client request processing device provided by another exemplary embodiment of the present application
[0060] Figure 8 A structural block diagram of a computer device shown in an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION
[0061] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.
[0062] The embodiment of the present application provides a data synchronization system based on MongoDB. The data synchronization system does not rely on the multi-data center deployment solution of the MongoDB database, is completely implemented by the application layer, and has the characteristics of multi-active, remote disaster recovery, user data consistency, and simple application configuration. Figure 1 A schematic diagram of a data synchronization system based on MongoDB provided by an exemplary embodiment of the present application is shown as follows: Figure 1 As shown, the system includes: at least two data centers and a service router; each data center is deployed with an independent MongoDB cluster, and each MongoDB cluster has a unique PID (Partition Identifier).
[0063] Among them, a replicator is deployed between each MongoDB cluster; the replicator is used to synchronize the target operation log in the source library to the target library, and save the synchronization point information of the source library and the target library to the local synchronization point library of the data center where the target library is located and the global synchronization point (Sync Point) library corresponding to the service router; the synchronization point information is used to identify the PID and timestamp of the MongoDB cluster corresponding to the log synchronization operation; the target operation log is an operation log containing the PID of the local MongoDB cluster of the source library in a specific field.
[0064] Among them, the source database is the MongoDB cluster that generates the target operation log, and the target database is the MongoDB cluster to which the target operation log is to be synchronized; schematically, for MongoDB cluster 1 in data center 1 and MongoDB cluster 2 in data center 2, when synchronizing the target operation log in MongoDB cluster 1 to the target database, MongoDB cluster 1 is the source database and MongoDB cluster 2 is the target database.
[0065] Since the source database contains not only the operation log (oplog) generated by itself, but also the operation log synchronized from other MongoDB clusters, in order to avoid redundant replication of the operation log, the replicator includes a filter for filtering out the operation log generated by the source database and synchronizing it to the target database; schematically, Figure 2 A schematic diagram of a replicator provided by an exemplary embodiment of the present application is shown. Figure 2As shown, the replicator includes a pull service component 210, a filter 220, and a replay service component 230; the replicator pulls the operation log from the source library through the pull service component 210. In order to avoid excessive data pulling causing waste of data processing resources in the subsequent processing process, the pull service component 210 can obtain the latest synchronization point information corresponding to the source library and the target library in the target library before pulling the operation log from the source library. When pulling the operation log, the operation log with a timestamp greater than the timestamp of the latest synchronization point information identifier is pulled from the source library; schematically, if the timestamp corresponding to the latest synchronization point information identifier between the source library and the target library in the target library is time point 1, the replicator pulls the operation log from the source library when pulling the operation log. The operation log after time point 1 is obtained; the filter 220 performs operation log filtering based on the PID of the operation log to filter out the target operation log from the pulled operation log; in the embodiment of the present application, when the operation log is generated, the PID of the corresponding MongoDB cluster can be added to the specific field of the operation log, so that when the operation log is filtered, the filter 220 can extract the corresponding PID from the specific field of the operation log, and by comparing the PID of the operation log with the PID of the local MongoDB cluster corresponding to the source library, the target operation log containing the PID of the local MongoDB cluster of the source library can be filtered out; the replay service component 230 writes the target operation log to the target library.
[0066] In an embodiment of the present application, the data center includes an application layer and an application driver layer, wherein the application driver layer includes a write operation marker; the write operation marker is used to add the PID of the local MongoDB cluster to a specific field of the operation log based on the operation type of the write operation. Write operations include insert operations, update operations, and delete operations. Further, different write operations correspond to different specific fields; wherein, for insert operations, the corresponding specific field may be the primary key field "_id"; for update operations, the corresponding specific field may be the document version field "doc_version".
[0067] Wherein, when the operation type of the write operation is an insert operation, the write operation marker is used to add the PID of the local MongoDB cluster to the object identifier ObjectId of the operation document, and the format of the added ObjectId is timestamp-PID-process identifier-increment counter. The ObjectID of the MongoDB cluster is a special data type, which is used to identify the primary key (_id) in the MongoDB cluster; the ObjectID provided in the embodiment of the present application is a hexadecimal string consisting of 12 bytes. Schematically, the format of the ObjectID is:
[0068] The first four bytes: timestamp accurate to seconds, indicating the time when the ObjectID was created;
[0069] The next 2 bytes: This part is the globally unique PID assigned by the system to the MongoDB cluster, indicating the identifier of the MongoDB cluster that created the operation log;
[0070] The next 3 bytes are the ID of the application process, which is used to identify the application process ID that generates the operation log. When the application process starts, it obtains the unique process ID in the cluster from the PID issuance module.
[0071] The last 3 bytes: This part is a counter that is used cyclically within the same application process to ensure the uniqueness of each ObjectID within the same process and within the same second.
[0072] For example, the operation log corresponding to the Insert operation is:
[0073]
[0074] Where op is equal to i, indicating that the write operation corresponding to the operation log is an insert operation. The value of the _id field under the o field is ObjectId. The ObjectId contains the PID of the MongoDB cluster that generates the operation log. In the above example, the value of the byte corresponding to the PID of the MongoDB cluster is "d624".
[0075] When the operation type of the write operation is an update operation, the write operation table and its "PID+operation timestamp" string are saved by defining the doc_version (document version) field in the operation log. Therefore, by querying the doc_version field of the operation log, the PID of the MongoDB cluster can be intercepted, thereby determining the MongoDB cluster where the operation log was most recently updated.
[0076] For example, the operation log corresponding to the update operation is:
[0077]
[0078]
[0079] Wherein, op is equal to u, indicating that the write operation corresponding to the operation log is an update operation. The doc_version field under the o field contains the PID of the MongoDB cluster that generates the operation log. In the above example, the value of the byte corresponding to the PID of the MongoDB cluster is "1001".
[0080] No special processing is done for the delete operation, because the MongoDB cluster does not record the operation log for the failed delete operation. For example, if the MongoDB cluster tries to delete a non-existent operation log, the deletion operation will fail because the operation log does not exist, but the failed operation will not be recorded in the operation log, which means that the MongoDB cluster to be synchronized will not receive the failed deletion operation, and therefore will not try to delete anything to avoid unnecessary data loss. Based on this, no special processing is needed for the delete operation.
[0081] After the write operation table marker adds the PID of the MongoDB cluster to the corresponding field based on the above-mentioned different types of write operations, when the replicator performs log filtering through filter 220, for the captured operation log, if the operation type determined based on the op field is an insert operation or an update operation, the PID corresponding to the operation log is extracted from the corresponding field and compared with the PID of the local MongoDB cluster of the source library. If they are the same, the operation log is released, otherwise, the operation log is filtered out.
[0082] If the operation type determined based on the op field is a delete operation (ie, op is equal to d), the operation log is released.
[0083] If the op field is equal to n, it means that the current operation log is an empty message and the operation log is filtered.
[0084] If the op field is equal to c, and the o field contains an applyOps structure, the applyOps data is parsed and processed according to the above rules. Otherwise, the operation log is filtered.
[0085] The replay service component 230 in the replicator synchronizes the filtered target operation log to the target library; after completing the synchronization of the target operation log in the target library, the synchronization point information in the local synchronization point library in the data center where the target library is located and the synchronization point information in the global synchronization point library corresponding to the service router are updated; the local point library of the data center stores the synchronization point information corresponding to the data log synchronization process performed by the local MongoDB cluster of the data center; the global synchronization point library stores the synchronization point information of the data synchronization process performed between each data center in the system.
[0086] Schematically, the synchronization point information includes the PID of the source library, the PID of the target library, and a timestamp, where the timestamp is the time when the operation log is generated; Schematically, Table 1 shows an example of a data structure for storing synchronization point information in each synchronization point library provided by an exemplary embodiment of the present application:
[0087] Table 1
[0088]
[0089] The service router is used to provide address query services for clients based on the global synchronization point library; the server router is a global service router, responsible for service health monitoring and user service routing.
[0090] Optionally, the service router is also used to detect the status of application services and MongoDB cluster services in each data center and maintain a healthy service table; schematically, Table 2 shows an example of the data structure of a healthy service table provided by an exemplary embodiment of the present application:
[0091] Table 2
[0092]
[0093] Table 3 shows an example of health service data provided by an exemplary embodiment of the present application:
[0094] Table 3
[0095] Service Address PID https: / / 10.10.1.1:7443 / portal / 1 https: / / 10.10.1.2:8443 / portal / 2 https: / / 10.10.1.5:9443 / portal / 5
[0096] That is to say, the data center, the service address of the data center, and the PID of the local MongoDB cluster are one-to-one corresponding. By determining one of them, the other information can be obtained.
[0097] The data center includes an application layer, and the application layer is used to provide business services to the client when it is determined that the local MongoDB cluster contains complete data of the client based on the received tracking point information and the local synchronization point library.
[0098] Wherein, a data integrity checker is deployed in the application layer. When the data integrity check is performed in the application layer, the application layer can determine the complete data of the client contained in the local MongoDB cluster based on the received tracking point information and the local synchronization point library through the data integrity checker;
[0099] It should be noted that Figure 1 In the example, a system including three data centers is taken, but based on different system settings, the system may include more or fewer data centers, and this application does not limit this.
[0100] The MongoDB-based data synchronization system provided in the embodiment of the present application can solve the shortcomings of the cross-region deployment solution based on the replica set and sharding mechanism, such as reduced network latency performance, inconsistent data, complex configuration and management, and insufficient coverage. The data synchronization system provided in the embodiment of the present application can realize multi-active deployment in different locations based on the application layer, which has the following advantages compared with the multi-active deployment in different locations based on the data layer in the related art:
[0101] Flexibility: The multi-site active-active deployment based on the application layer can be flexibly configured and customized according to business needs. The application layer can perform decentralized data processing and decision-making based on specific business logic and rules, so as to better adapt to different needs and scenarios;
[0102] Scalability: Multi-site active-active deployment based on the application layer can better support system expansion. By deploying different application instances in multiple geographical locations, traffic and load can be dispersed to different nodes, improving the system's processing power and scalability;
[0103] Elasticity and adaptability: The multi-active deployment based on the application layer can better cope with abnormal situations such as network interruption and data synchronization delay. When a node fails or the network is interrupted, other nodes can still continue to provide services and automatically synchronize data after recovery to ensure the availability and stability of the system;
[0104] High performance: The multi-active deployment based on the application layer only relies on the local database to achieve fast access and processing of data. The application layer can use caching, load balancing and other technologies to improve the performance and response speed of the system;
[0105] User data consistency: The multi-active deployment implemented at the application layer uses customized strategies to verify data integrity and guide users to nodes with complete data, ensuring the consistency of user data.
[0106] Cloud-native support: Based on the application layer's multi-active deployment, the custom format ObjectId is used to solve the problem of insufficient support for cloud-native native by the native ObjectId algorithm.
[0107] Based on Figure 1 The MongoDB-based data synchronization system shown in the figure, the embodiment of the present application provides a client request processing method, Figure 3 A flowchart of a client request processing method provided by an exemplary embodiment of the present application is shown, the method is executed by the application layer of a target data center, the target data center is any one of at least two data centers included in the data synchronization system, each data center is deployed with an independent MongoDB cluster, each MongoDB cluster has a unique partition identifier PID; operation logs are synchronized between MongoDB clusters; Figure 3 As shown, the method includes:
[0108] Step 310, receiving a service access request sent by a client; the service access request includes tracking point information; the tracking point information includes the first PID and the first timestamp of the MongoDB cluster where the client has performed the latest write operation.
[0109] When the client has a service address cached locally or obtains the service address, it can send a service access request to the data center indicated by the service address according to the cached service address. Correspondingly, the data center receives the service access request sent by the client.
[0110] The service access request contains trace point information, which contains the cluster PID and timestamp of the client's last write operation. The format of the trace point information can be customized according to actual needs. The trace point information can be created by the application layer of the client's last write operation and returned to the client.
[0111] Furthermore, in order to reduce information redundancy in the client, the client may only store the latest tracking point information, that is, after the client receives the message returned by the application layer, the tracking point information is updated and replaced.
[0112] Step 320, perform data integrity verification based on the tracking point information and the local synchronization point library of the target data center to obtain a verification result; the local synchronization point library contains synchronization point information; the synchronization point information is used to identify the PID and timestamp of the MongoDB cluster corresponding to the log synchronization operation.
[0113] After extracting the tracking point information from the service access request, the application layer can determine whether the complete data of the client has been synchronized to or exists in the local MongoDB cluster of the current data center through the first PID and the first timestamp in the tracking point information, and then determine whether to provide services to the client;
[0114] Among them, when determining data integrity based on tracking point information, the application layer can perform verification based on the synchronization point information stored in the local synchronization point library in combination with the tracking point information, that is, check whether the synchronization point information of the local synchronization point library contains an entry (that is, the synchronization point information) that matches the first PID and the first timestamp. If there is a matching entry, it means that the client's complete data already exists in the target data center. If there is no matching entry, it means that the client's complete data has not yet been fully synchronized to the target data center.
[0115] Step 330 , when the verification result indicates that the local MongoDB cluster of the target data center contains the complete data of the client, provide the client with a business service corresponding to the service access request.
[0116] Accordingly, when the verification result indicates that the local MongoDB cluster of the target center does not contain the complete data of the client, the service access request sent by the client is rejected.
[0117] In summary, the client request processing method provided in the embodiment of the present application is executed by the application layer of each data center included in the data synchronization system based on MongoDB, each data center has an independent MongoDB cluster, each MongoDB cluster has a unique PID, and operation logs are synchronized between each MongoDB cluster; after receiving the service access request sent by the client, data integrity verification is performed based on the tracking point information contained in the service access request and the synchronization point information contained in the local synchronization point library of the target data center, and when the verification result indicates that the local MongoDB cluster of the target data center contains the complete data of the client, the corresponding business service is provided to the client; in the above scheme, data integrity confirmation is performed by using tracking point information and local synchronization point library, which ensures that the latest and consistent data is used when processing client requests, reduces the risk of processing exceptions due to data delays, and improves the overall reliability of the system; the application layer can provide services to the client based on the local MongoDB cluster, can reduce performance problems caused by data synchronization delays, and can also improve service response speed.
[0118] In the embodiment of the present application, the data synchronization system further includes a service router for providing a service address query service for the client. Figure 4 A flowchart of a client request processing method provided by another exemplary embodiment of the present application is shown, and the method is executed by a service router in a data synchronization system based on MongoDB; the data synchronization system includes at least two data centers, each data center is deployed with an independent MongoDB cluster, and each MongoDB cluster has a unique partition identifier PID; operation logs are synchronized between each MongoDB cluster, such as Figure 4 As shown, the method may include the following steps:
[0119] Step 410, receiving an address query request sent by a client; the address query request includes tracking point information; the tracking point information includes the first PID and the first timestamp of the MongoDB cluster where the client has performed the latest write operation.
[0120] When the service address cached locally is empty or expired, the client can send an address query request carrying tracking point information to the service router to obtain the service address from the service router and cache it locally; accordingly, the service router receives the address query request, performs a service address query based on the address query request and feeds back the query result to the client.
[0121] Step 420, perform address query based on the tracking point information and the global synchronization point library to obtain the target service address; the local MongoDB cluster of the data center corresponding to the target service address contains the complete data of the client.
[0122] The global synchronization library contains synchronization point information between various MongoDB clusters, and the synchronization point information includes the PID of the source library, the PID of the target library and the timestamp; the service router extracts the tracking point information from the address query request, queries the synchronization point information in the global synchronization point library based on the tracking point information, and checks whether the synchronization point information in the global synchronization point library contains an entry (i.e., synchronization point information) that matches the first PID and the first timestamp, that is, determines which data center's local MongoDB cluster contains the client's complete data; wherein the matching entry indicates that the complete data corresponding to the client already exists in the MongoDB cluster of the data center corresponding to the synchronization point information, and determines the service address of the data center corresponding to the synchronization point information as the target service address.
[0123] In a possible implementation, the service router may perform address queries in a global synchronization point library in a certain order. After finding the target service address, the query operation may be stopped and the target service address may be fed back, thereby reducing resource waste caused by unnecessary query operations.
[0124] Step 430: Feedback the target service address to the client, so that the client can access the service of the corresponding data center based on the target service address.
[0125] The service router feeds back the target service address to the client. Correspondingly, the client receives the target service address and caches it locally to access the service of the corresponding data center based on the target service address.
[0126] In summary, the client request processing method provided in the embodiment of the present application is executed by a service router included in a MongoDB-based data synchronization system, wherein the data synchronization system includes at least two data centers, each data center is deployed with an independent MongoDB cluster, and each MongoDB cluster has a unique partition identifier PID; operation logs are synchronized between each MongoDB cluster; after receiving an address query request sent by the client, matching is performed based on the tracking point information included in the address query request and the synchronization point information included in the global synchronization point library, and the service address of the data center corresponding to the matched synchronization point information is determined as the target service address and fed back to the client, so that the client performs subsequent service access based on the target service address; in the above scheme, by using the tracking point information combined with the synchronization point information in the global synchronization point library to perform an address query, the service address of the data center containing the complete data of the client (i.e., the target service address) can be queried, so that the data center accessed by the client based on the target service address can provide services to the client based on the latest and consistent complete data, thereby reducing the risk of processing abnormalities due to data delays and improving the overall reliability of the system.
[0127] Take the service access process when the service address cached locally by the client is empty or expired as an example. Figure 5 A flowchart of a client request processing method provided by another exemplary embodiment of the present application is shown. The method can be executed by a data synchronization system based on MongoDB, wherein the data synchronization system includes at least two data centers, each data center is deployed with an independent MongoDB cluster, and each MongoDB cluster has a unique partition identifier PID; operation logs are synchronized between the various MongoDB clusters; Figure 5 As shown, the method includes an address query phase and a service access phase;
[0128] Among them, in the address query stage, it involves the interaction between the client and the service router. The process includes:
[0129] Step 510: The service router receives an address query request sent by the client; the address query request includes tracking point information; the tracking point information includes the first PID and the first timestamp of the MongoDB cluster where the client performs the latest write operation.
[0130] Step 520: The service router performs an address query based on the tracking point information and the global synchronization point library to obtain a target service address; the local MongoDB cluster of the data center corresponding to the target service address contains the complete data of the client.
[0131] In a possible implementation, the address query process can be implemented as follows:
[0132] According to the priority order of each service address, the data integrity is verified based on the synchronization point information corresponding to each service address and the tracking point information in turn;
[0133] When it is determined that the second synchronization point information and the tracking point information satisfy the integrity condition, the service address corresponding to the second synchronization point information is determined as the target service address.
[0134] The priority order may be based on a customized service priority of the business; for attributes, the priority order may be defined according to the distance between the client and the data center. The closer the physical distance between the client and the data center, the higher the priority of the corresponding service address; or, the priority order may be defined according to the province to which the client belongs, and clients in the same province have a unified service address priority list; Table 4 shows an example of a priority list of service addresses provided by an exemplary embodiment of the present application:
[0135] Table 4
[0136] Priority (high to low) Service Address PID 1 https: / / 10.10.1.5:9443 / portal / 5 2 https: / / 10.10.1.1:7443 / portal / 1 3 https: / / 10.10.1.2:8443 / portal / 2
[0137] When performing address queries based on tracking point information and the global synchronization point library, the synchronization point information and tracking point information corresponding to the corresponding service address are obtained in turn according to the priority of each service address to verify data integrity, so as to confirm whether the MongoDB cluster corresponding to the service address has the complete data of the client. If it is determined that the current MongoDB cluster has the complete data of the client, the corresponding service address is immediately returned to the client.
[0138] Among them, whether the MongoDB cluster has the complete data of the client can be judged based on the tracking point information and the synchronization point information between the clusters; the service router queries the synchronization point information from the MongoDB cluster of the client's last write operation (i.e., the source database) to the MongoDB cluster corresponding to the service address (i.e., the target database) from the global synchronization point database, and by comparing the two and determining that the integrity conditions are met, it is determined that the MongoDB cluster contains the complete data of the client.
[0139] The integrity condition includes:
[0140] The first PID is the same as the third PID of the MongoDB cluster identified by the second synchronization point information;
[0141] or,
[0142] When the first PID is different from the third PID, the timestamp identified by the second synchronization point information is greater than the first timestamp.
[0143] When the first PID is the same as the third PID of the MongoDB cluster identified by the second synchronization point information, it means that the MongoDB cluster corresponding to the second synchronization point information is the cluster to which the client last performs a write operation, and the MongoDB cluster contains the complete data of the client; and when the first PID is different from the third PID, it means that the MongoDB cluster corresponding to the second synchronization point information is not the cluster to which the client last performs a write operation, and at this time, a timestamp judgment is performed. If the timestamp identified by the second synchronization point information is greater than the first timestamp, it means that the MongoDB cluster to which the user performs a write operation and the MongoDB cluster corresponding to the second synchronization point information have completed data synchronization for the client, and the MongoDB cluster corresponding to the second synchronization point information also contains the latest complete data of the client; if the timestamp identified by the second synchronization point information is less than the first timestamp, it means that the complete data of the client has not yet been synchronized in the current MongoDB cluster, and the service router can continue the data integrity verification process of the next service address according to the priority order of the service address until the synchronization point information that meets the integrity condition is obtained, and the service address corresponding to the synchronization point information is determined as the target service address.
[0144] Step 530: The service router feeds back the target service address to the client, so that the client accesses the service of the corresponding data center based on the target service address.
[0145] In the service access phase, the client interacts with the application layer of the target data center, which can be the data center indicated by the target service address. However, since there may be data tampering and fraud in the interaction between the service router and the client, in order to further improve the security of service access, after receiving the service access request, the application layer will verify whether the complete data of the client is stored in the local MongoDB cluster, and determine whether to accept the user service access request based on the verification result. This process includes:
[0146] Step 540, the application layer receives a service access request sent by the client; the service access request includes tracking point information; the tracking point information includes the first PID and the first timestamp of the MongoDB cluster where the client has performed the latest write operation.
[0147] The client carries the tracking point information and accesses the application service according to the target service address specified by the service router. Correspondingly, the application layer receives the access service request sent by the client based on the target service address; the target service address is the service address that the service router feeds back to the client after performing an address query in the global synchronization point library based on the address query request sent by the client; the global synchronization point library contains the synchronization point information between at least two MongoDB clusters.
[0148] Step 550, perform data integrity verification based on the tracking point information and the local synchronization point library of the target data center to obtain a verification result; the local synchronization point library contains synchronization point information; the synchronization point information is used to identify the PID and timestamp of the MongoDB cluster corresponding to the log synchronization operation.
[0149] In a possible implementation, the data integrity verification process can be implemented as follows:
[0150] When it is determined that the integrity condition is satisfied based on the tracking point information and the local synchronization point library of the target data center, it is determined that the verification result indicates that the local MongoDB cluster contains complete data of the client.
[0151] The application layer queries the local synchronization point database to obtain the synchronization point information from the cluster where the client last wrote the operation (i.e., the source database) to the local MongoDB cluster (i.e., the target database). When it is determined that the synchronization point information and the tracking point information meet the integrity condition, it is determined that the local MongoDB cluster contains the complete data of the client; wherein the integrity condition includes:
[0152] The first PID is the same as the second PID of the local MongoDB cluster;
[0153] or,
[0154] When the first PID and the second PID are different, the timestamp identified by the first synchronization point information is greater than the first timestamp; the first synchronization point information is the latest synchronization point information between the MongoDB cluster corresponding to the first PID and the local MongoDB cluster extracted from the local synchronization point library.
[0155] When the first PID is the same as the second PID of the local MongoDB cluster, it indicates that the local MongoDB cluster is the cluster to which the client last performed write operations, and the MongoDB cluster contains the complete data of the client; and when the first PID is different from the second PID, it indicates that the local MongoDB cluster is not the cluster to which the client last performed write operations. At this time, a timestamp judgment is performed. If the timestamp identified by the first synchronization point information is greater than the first timestamp, it indicates that the data synchronization of the client has been completed between the MongoDB cluster to which the user performed write operations and the local MongoDB cluster, and the local MongoDB cluster also contains the latest complete data of the client; if the timestamp identified by the first synchronization point is less than the first timestamp, it indicates that the complete data of the client has not yet been synchronized in the local MongoDB cluster.
[0156] Step 560: When the verification result indicates that the local MongoDB cluster of the target data center contains the complete data of the client, the business service corresponding to the service access request is provided to the client.
[0157] That is, when it is determined that the local MongoDB cluster of the target data center contains the latest and complete data of the client, the application layer of the target data center accepts the service access request sent by the client and returns the processing result to the client.
[0158] In the case that the local MongoDB cluster of the target data center does not contain the latest complete data of the client, the application layer of the target data center rejects the service access request sent by the client.
[0159] In a possible implementation, the method further includes:
[0160] When the verification result indicates that the local MongoDB cluster of the target data center does not contain the complete data of the client, a redirection instruction is sent to the client; the redirection instruction is used to instruct the client to re-acquire the service address.
[0161] Among them, for CS (Client / Server Architecture) architecture applications, the redirection instruction can be a redirection error code, and the application layer returns the redirection error code to the client. After receiving the redirection error code, the client re-requests an available address from the router; for BS architecture (Browser / Server Architecture) applications, the request is redirected to the service router through the client browser, and the service router redirects it to the available service address through the client browser.
[0162] Furthermore, when the application layer accepts the service access request of the client, the method further includes:
[0163] After completing the business service corresponding to the client, update the tracking point information;
[0164] Send updated tracking point information to the client.
[0165] That is to say, after completing the business service corresponding to the client, the application layer creates and issues new tracking point information to indicate that the MongoDB cluster where the client last performed a write operation is the current MongoDB cluster, so that the client can access the service based on the updated tracking point information in subsequent service access requests.
[0166] In summary, the client request processing method provided in the embodiment of the present application is executed by a data synchronization system based on MongoDB, wherein the data synchronization system includes at least two data centers, each data center has an independent MongoDB cluster, each MongoDB cluster has a unique PID, and operation logs are synchronized between each MongoDB cluster; wherein, after receiving the address query request sent by the client, the service router in the data synchronization system matches the tracking point information contained in the address query request and the synchronization point information contained in the global synchronization point library, determines the service address of the data center corresponding to the matched synchronization point information as the target service address and feeds it back to the client; the application layer of the target data center receives the address query request sent by the client, and sends the matching service address to the client. After a service access request is sent, data integrity verification is performed based on the tracking point information contained in the service access request and the synchronization point information contained in the local synchronization point library of the target data center. When the verification result indicates that the local MongoDB cluster of the target data center contains the complete data of the client, the client is provided with corresponding business services. In the above scheme, data integrity confirmation is performed by using tracking point information and the local synchronization point library, which ensures that the latest and consistent data is used when processing client requests, reduces the risk of processing exceptions due to data delays, and improves the overall reliability of the system. The application layer can provide services to clients based on the local MongoDB cluster, which can reduce performance issues caused by data synchronization delays. In addition, the service response speed can also be improved.
[0167] Figure 6A block diagram of a client request processing device provided by an exemplary embodiment of the present application is shown, and the device can be applied to the application layer of a target data center, and the target data center is any one of at least two data centers included in the data synchronization system, and each data center is deployed with an independent MongoDB cluster, and each MongoDB cluster has a unique partition identifier PID; operation logs are synchronized between MongoDB clusters; Figure 6 As shown, the device comprises:
[0168] The first receiving module 610 is used to receive a service access request sent by a client; the service access request includes tracking point information; the tracking point information includes a first PID and a first timestamp of the MongoDB cluster where the client performs the latest write operation;
[0169] Verification module 620, used to perform data integrity verification based on the tracking point information and the local synchronization point library of the target data center to obtain a verification result; the local synchronization point library includes synchronization point information; the synchronization point information is used to identify the PID and timestamp of the MongoDB cluster corresponding to the log synchronization operation;
[0170] The service module 630 is used to provide the client with the business service corresponding to the service access request when the verification result indicates that the local MongoDB cluster of the target data center contains the complete data of the client.
[0171] In a possible implementation, the verification module 620 is configured to determine that the verification result indicates that the local MongoDB cluster contains complete data of the client when it is determined that the integrity condition is satisfied based on the tracking point information and the local synchronization point library of the target data center;
[0172] The integrity conditions include:
[0173] The first PID is the same as the second PID of the local MongoDB cluster;
[0174] or,
[0175] When the first PID is different from the second PID, the timestamp identified by the first synchronization point information is greater than the first timestamp; the first synchronization point information is the latest synchronization point information between the MongoDB cluster corresponding to the first PID and the local MongoDB cluster extracted from the local synchronization point library.
[0176] In a possible implementation manner, the device further includes:
[0177] An instruction sending module is used to send a redirection instruction to the client when the verification result indicates that the local MongoDB cluster of the target data center does not contain the complete data of the client; the redirection instruction is used to instruct the client to re-acquire the service address.
[0178] In a possible implementation manner, the device further includes:
[0179] An updating module, used for updating the tracking point information after completing the business service corresponding to the client;
[0180] The sending module is used to send the updated tracking point information to the client.
[0181] In a possible implementation, the data synchronization system further includes a service router;
[0182] The first receiving module is used to receive an access service request sent by the client based on a target service address; the target service address is a service address that is fed back to the client by the service router after performing an address query in a global synchronization point library based on an address query request sent by the client; the global synchronization point library contains synchronization point information between at least two MongoDB clusters.
[0183] Figure 7 A block diagram of a client request processing device provided by another exemplary embodiment of the present application is shown, which can be applied to a service router in a data synchronization system based on MongoDB; the data synchronization system includes at least two data centers, each of which is deployed with an independent MongoDB cluster, and each of the MongoDB clusters has a unique partition identifier PID; operation logs are synchronized between the MongoDB clusters; Figure 7 As shown, the device comprises:
[0184] The second receiving module 710 is used to receive an address query request sent by a client; the address query request includes tracking point information; the tracking point information includes a first PID and a first timestamp of the MongoDB cluster where the client performs the latest write operation;
[0185] The address query module 720 is used to perform address query based on the tracking point information and the global synchronization point library to obtain the target service address; the local MongoDB cluster of the data center corresponding to the target service address contains the complete data of the client;
[0186] The address feedback module 730 is used to feed back the target service address to the client, so that the client can access the service of the corresponding data center based on the target service address.
[0187] In a possible implementation, the address query module 720 includes:
[0188] A verification submodule, for performing data integrity verification based on the synchronization point information corresponding to each service address and the tracking point information in sequence according to the priority order of each service address;
[0189] An address confirmation submodule, configured to determine the service address corresponding to the second synchronization point information as the target service address when it is determined that the second synchronization point information and the tracking point information meet the integrity condition;
[0190] The integrity conditions include:
[0191] The first PID is the same as the third PID of the MongoDB cluster identified by the second synchronization point information;
[0192] or,
[0193] When the first PID is different from the third PID, the timestamp identified by the second synchronization point information is greater than the first timestamp.
[0194] Figure 8 The structural block diagram of a computer device 800 shown in an exemplary embodiment of the present application is shown. The computer device can be implemented as a data center or service router in the above-mentioned scheme of the present application. The computer device 800 includes a central processing unit (CPU) 801, a system memory 804 including a random access memory (RAM) 802 and a read-only memory (ROM) 803, and a system bus 805 connecting the system memory 804 and the central processing unit 801. The computer device 800 also includes a mass storage device 806 for storing an operating system 809, an application program 810 and other program modules 811.
[0195] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer storage media include RAM, ROM, Erasable Programmable Read Only Memory (EPROM), Electronically-Erasable Programmable Read-Only Memory (EEPROM) flash memory or other solid-state storage technology, CD-ROM, Digital Versatile Disc (DVD) or other optical storage, cassettes, tapes, disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that the computer storage medium is not limited to the above. The above-mentioned system memory 804 and mass storage device 806 can be collectively referred to as memory.
[0196] According to various embodiments of the present disclosure, the computer device 800 can also be connected to a remote computer on the network through a network such as the Internet. That is, the computer device 800 can be connected to the network 808 through the network interface unit 807 connected to the system bus 805, or the network interface unit 807 can be used to connect to other types of networks or remote computer systems (not shown).
[0197] The memory also includes at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is stored in the memory. The central processing unit 801 implements all or part of the steps in the client request processing method shown in the above-mentioned embodiments by executing the at least one instruction, at least one program, code set or instruction set.
[0198] In an exemplary embodiment, a computer-readable storage medium is also provided, in which at least one computer program is stored, and the computer program is loaded and executed by a processor to implement all or part of the steps in the above-mentioned client request processing method. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.
[0199] In an exemplary embodiment, a computer program product is also provided, the computer program product comprising at least one computer program, the computer program being loaded and executed by a processor Figure 3 , Figure 4 or Figure 5 All or part of the steps of the client request processing method shown in any embodiment.
[0200] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the claims.
[0201] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. A client request processing method, characterized in that: The method is executed by the application layer of the target data center, the target data center is any one of at least two data centers included in the MongoDB-based data synchronization system, each of the data centers is deployed with an independent MongoDB cluster, and each of the MongoDB clusters has a unique partition identifier PID; operation logs are synchronized between the MongoDB clusters; the method includes: Receive a service access request sent by a client; the service access request includes tracking point information; the tracking point information includes a first PID and a first timestamp of a MongoDB cluster on which the client has performed a latest write operation; Perform data integrity verification based on the tracking point information and the local synchronization point library of the target data center to obtain a verification result; the local synchronization point library includes synchronization point information; the synchronization point information is used to identify the PID and timestamp of the MongoDB cluster corresponding to the log synchronization operation; When the verification result indicates that the local MongoDB cluster of the target data center contains complete data of the client, the business service corresponding to the service access request is provided to the client.
2. The method according to claim 1, characterized in that The data integrity verification is performed based on the tracking point information and the local synchronization point library of the target data center to obtain a verification result, including: In a case where it is determined based on the tracking point information and the local synchronization point library of the target data center that the integrity condition is satisfied, determining that the verification result indicates that the local MongoDB cluster contains complete data of the client; The integrity conditions include: The first PID is the same as the second PID of the local MongoDB cluster; or, When the first PID is different from the second PID, the timestamp identified by the first synchronization point information is greater than the first timestamp; the first synchronization point information is the latest synchronization point information between the MongoDB cluster corresponding to the first PID and the local MongoDB cluster extracted from the local synchronization point library.
3. The method according to claim 1, characterized in that: The method further comprises: When the verification result indicates that the local MongoDB cluster of the target data center does not contain the complete data of the client, a redirection instruction is sent to the client; the redirection instruction is used to instruct the client to re-acquire the service address.
4. The method according to claim 1, characterized in that: The method further comprises: After completing the business service corresponding to the client, updating the tracking point information; The updated tracking point information is sent to the client.
5. The method according to any one of claims 1 to 4, characterized in that: The data synchronization system also includes a service router; The receiving a service access request sent by a client comprises: Receiving a service access request sent by the client based on a target service address; The target service address is the service address fed back to the client by the service router after performing an address query in the global synchronization point library based on the address query request received from the client; the global synchronization point library contains synchronization point information between at least two MongoDB clusters.
6. A client request processing method, characterized in that: The method is executed by a service router in a data synchronization system based on MongoDB; the data synchronization system comprises at least two data centers, each of which is deployed with an independent MongoDB cluster, and each of which has a unique partition identifier PID; operation logs are synchronized between the MongoDB clusters; the method comprises: Receive an address query request sent by a client; the address query request includes tracking point information; the tracking point information includes a first PID and a first timestamp of a MongoDB cluster on which the client has performed a latest write operation; Performing address query based on the tracking point information and the global synchronization point library to obtain the target service address; the local MongoDB cluster of the data center corresponding to the target service address contains the complete data of the client; The target service address is fed back to the client, so that the client accesses the service of the corresponding data center based on the target service address.
7. The method according to claim 6, characterized in that The performing address query based on the tracking point information and the global synchronization point library to obtain the target service address includes: According to the priority order of each service address, data integrity verification is performed in sequence based on the synchronization point information corresponding to each service address and the tracking point information; In the case where it is determined that the second synchronization point information and the tracking point information meet the integrity condition, determining the service address corresponding to the second synchronization point information as the target service address; The integrity conditions include: The first PID is the same as the third PID of the MongoDB cluster identified by the second synchronization point information; or, When the first PID is different from the third PID, the timestamp identified by the second synchronization point information is greater than the first timestamp.
8. A data synchronization system based on MongoDB, characterized in that: The system includes: at least two data centers and a service router; each data center is deployed with an independent MongoDB cluster, and each MongoDB cluster has a unique partition identifier PID; A replicator is deployed between each of the MongoDB clusters; the replicator is used to synchronize the target operation log in the source database to the target database, and save the synchronization point information of the source database and the target database to the local synchronization point database of the data center where the target database is located and the global synchronization point database corresponding to the service router; the synchronization point information is used to identify the PID and timestamp of the MongoDB cluster corresponding to the log synchronization operation; the target operation log is an operation log containing the PID of the local MongoDB cluster of the source database in a specific field; The service router is used to provide address query services for the client based on the global synchronization point library; The data center includes an application layer, and the application layer is used to provide business services for the client when it is determined that the local MongoDB cluster contains complete data of the client based on the received tracking point information and the local synchronization point library.
9. The system according to claim 8, characterized in that The application driver layer corresponding to the application layer includes a write operation marker; the write operation marker is used to add the PID of the local MongoDB cluster to a specific field of the operation log based on the operation type of the write operation.
10. The system according to claim 9, characterized in that When the operation type of the write operation is an insert operation, the write operation marker is used to add the PID of the local MongoDB cluster to the object identifier ObjectId of the operation document, and the format of the added ObjectId is timestamp-PID-process identifier-increment counter.
Citation Information
Patent Citations
Method and device for synchronizing account information in cluster system, and storage medium
CN108881231A
Distributed high-fault-tolerance data real-time synchronization method from MongoDB to HBase
CN110287251A
MongoDB data synchronization method based on converged media platform
CN112667698A
Cluster authority authentication method, client, server and storage medium
CN115801319A
Authorized client application deployment method and device, computer equipment and storage medium
CN118245091A
