Data processing method, device and server

By recording data processing requests in the global log and obtaining incremental logs in a distributed system separated by log mirroring, the front-end server performs operations in sequence, solving the confusion and inconsistency of data processing operations and achieving efficient and orderly data processing.

CN113297233BActive Publication Date: 2025-05-09ALIBABA GROUP HOLDING LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202010903226.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-09-01
Publication Date
2025-05-09
Estimated Expiration
2040-09-01

AI Technical Summary

Technical Problem

In a distributed system with log mirror separation, when the front-end server processes data processing requests, the data processing operation may be confusing and inconsistent due to the separation of the log and the image.

Method used

By sending data processing requests to the global server, recording them in the global log, and obtaining incremental logs through the global server, the front-end server performs the operations contained in the incremental log in sequence to ensure the orderly and consistency of data processing.

Benefits of technology

It realizes efficient and orderly data processing operations in a distributed system with log mirror separation, avoids confusion and inconsistency in data processing, and reduces the data processing burden of the global server.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113297233B_ABST
    Figure CN113297233B_ABST
Patent Text Reader

Abstract

This specification provides a data processing method, device and server. In one embodiment, based on the above data processing method, in a distributed system, when the front-end server receives a data processing request containing an operation initiated by a terminal device, it will not immediately respond to the data processing request to perform the operation, but will first send the data processing request to the global server; after receiving the data processing request, the global server will not perform the operation, but will record the operations in the data processing requests from different front-end servers in the global log in sequence; the front-end server then obtains the incremental log through the global server, and according to the incremental log, executes the operations contained in the incremental log in sequence. This makes the data processing operations in the distributed system with log mirror separation efficient and orderly.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This specification belongs to the field of Internet technology, and in particular to data processing methods, devices and servers. Background Art

[0002] In a distributed data system with separated logs and mirrors, when a front-end server receives a data processing request initiated by the current terminal device, it often needs to obtain the current latest data related to the data processing request in order to perform the corresponding operation processing. However, since the log and mirror are separated in the system, this part of the current latest data may have been stored in the local mirror data, or it may also be stored in the global log of the remote global server. At this time, the front-end server must still pull the above-mentioned current latest data from the remote global server first. However, in the process of the front-end server pulling the above-mentioned current latest data, the front-end servers in other regions may have accessed the global server and modified the above-mentioned data, which will in turn affect the data processing of the front-end server, resulting in confusion, inconsistency and other problems in the data processing operations in the distributed system.

[0003] Therefore, there is an urgent need for a data processing method that can make data processing operations in a distributed system with log mirror separation efficient and orderly. Summary of the invention

[0004] This specification provides a data processing method, device and server to make data processing operations in a distributed system with log mirror separation efficient and orderly.

[0005] A data processing method, device and server provided in this specification are implemented as follows:

[0006] A data processing method comprises: receiving a data processing request sent by a terminal device; wherein the data processing request contains an operation; sending the data processing request to a global server; wherein the global server is used to receive the data processing request sent by a front-end server, and record the operations contained in the data processing request in sequence in a global log; obtaining an incremental log through the global server; and executing the operations contained in the incremental log in sequence according to the incremental log and local mirror data.

[0007] A data processing method includes: obtaining a data processing request; and recording the operations contained in the data processing request in sequence in a global log.

[0008] A data processing method comprises: obtaining proposal data; initiating a vote on the proposal data in multiple global servers according to a consistency protocol; and recording the operations contained in the proposal data in sequence in a global log when it is determined that more than a preset number of global servers have voted in favor of the proposal data.

[0009] A data processing system comprises a plurality of front-end servers and a plurality of global servers, wherein: the front-end server receives a data processing request sent by a terminal device; wherein the data processing request comprises an operation; the front-end server encapsulates the data processing request into proposal data based on a consistency protocol, and sends the proposal data to a global server; the global server obtains the proposal data; according to the consistency protocol, a vote on the proposal data is initiated in a plurality of global servers; when it is determined that more than a preset number of global servers have voted in favor of the proposal data, the operations contained in the proposal data are recorded in sequence in a global log; the front-end server also obtains an incremental log through the global server; and according to the incremental log and local mirror data, the operations contained in the incremental log are executed in sequence.

[0010] A data processing device comprises: a receiving module, used for receiving a data processing request sent by a terminal device; wherein the data processing request contains an operation; a sending module, used for sending the data processing request to a global server; wherein the global server is used for receiving the data processing request sent by a front-end server, and recording the operations contained in the data processing request in sequence in a global log; a first acquisition module, used for acquiring an incremental log through the global server; and an execution module, used for executing the operations contained in the incremental log in sequence according to the incremental log and local mirror data.

[0011] A data processing device includes: a second acquisition module for acquiring proposal data; a processing module for initiating a vote on the proposal data among multiple global servers according to a consistency protocol; and a recording module for recording the operations contained in the proposal data in sequence in a global log when it is determined that more than a preset number of global servers have voted in favor of the proposal data.

[0012] A server comprises a processor and a memory for storing processor executable instructions, wherein the processor executes the instructions to receive a data processing request sent by a terminal device; wherein the data processing request contains an operation; the data processing request is sent to a global server; wherein the global server is used to receive the data processing request sent by a front-end server, and record the operations contained in the data processing request in a global log in sequence; obtain an incremental log through the global server; and execute the operations contained in the incremental log in sequence according to the incremental log and local mirror data.

[0013] A computer-readable storage medium stores computer instructions, which, when executed, implement a data processing request sent by a terminal device; wherein the data processing request includes an operation; the data processing request is sent to a global server; wherein the global server is used to receive the data processing request sent by a front-end server, and record the operations included in the data processing request in a global log in sequence; obtain an incremental log through the global server; and execute the operations included in the incremental log in sequence according to the incremental log and local mirror data.

[0014] The data processing method, device and server provided in this specification, in a distributed system with log mirror separation, when the front-end server receives a data processing request containing an operation from a terminal device, it will not immediately respond to the data processing request to perform the operation, but will first send the data processing request to the global server; after receiving the data processing request, the global server will not consume processing resources to perform the operation, but will record the operations contained in the data processing requests from different front-end servers in the global log in sequence. The front-end server then obtains the incremental log through the global server, and executes the operations contained in the incremental log in sequence according to the incremental log. This can make the data processing operations in the distributed system with log mirror separation efficient and orderly. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of this specification, the drawings required for use in the embodiments will be briefly introduced below. The drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0016] Figure 1 It is a schematic diagram of an embodiment of the structure of a system using the data processing method provided in the embodiments of this specification;

[0017] Figure 2is a schematic diagram of an embodiment of a data processing method provided by an embodiment of this specification, in a scenario example;

[0018] Figure 3 is a schematic diagram of an embodiment of applying the data processing method provided by the embodiments of this specification in a scenario example;

[0019] Figure 4 It is a schematic diagram of an embodiment of applying the data processing method provided by the embodiments of this specification in a scenario example;

[0020] Figure 5 It is a schematic diagram of an embodiment of applying the data processing method provided by the embodiments of this specification in a scenario example;

[0021] Figure 6 is a schematic diagram of an embodiment of a data processing method provided by an embodiment of this specification, in a scenario example;

[0022] Figure 7 is a flowchart of a data processing method provided by an embodiment of this specification;

[0023] Figure 8 is a flowchart of a data processing method provided by an embodiment of this specification;

[0024] Fig. 9 It is a schematic diagram of the structure of a server provided by an embodiment of this specification;

[0025] Fig.10 It is a schematic diagram of the structural composition of a data processing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0027] The embodiment of this specification provides a data processing method, which can be applied to a system structure based on a distributed system including multiple global servers and multiple front-end servers. Figure 1 shown.

[0028] The system is specifically a distributed system with separated log mirroring.

[0029] Specifically, the system may include multiple global servers. The multiple global servers may constitute a protocol group (e.g., a Paxos protocol group), including: a master node server (recorded as Leader) deployed in City S, and slave node server 1 (recorded as Follower 1) deployed in City B, and slave node server 2 (recorded as Follower 2) deployed in City Z. The three global servers constitute a consistency protocol (e.g., Paxos protocol) group, responsible for global log services.

[0030] The system also includes multiple front-end servers. Among them, the above-mentioned multiple front-end servers (which can be recorded as FrontEnd) include servers deployed in different areas for connecting to terminal devices of users in the area. For example, including: front-end server 1 deployed in area 1 (which can be recorded as FrontEnd1), front-end server 2 deployed in area 2 (which can be recorded as FrontEnd2), front-end server 3 deployed in area 3 (which can be recorded as FrontEnd3)... front-end server n deployed in area N (which can be recorded as FrontEndN), etc. In this system, the mirror service can be provided by the front-end server or by a local third-party server in the area where the front-end server is located.

[0031] Based on the above system, during specific implementation, when different front-end servers receive a data processing request containing an operation sent by a user in the connected area through a terminal device, they will not immediately respond to the data processing request to perform the operation; instead, they can first encapsulate the above data processing request into proposal data based on the consistency protocol, and then send the proposal data to the global server.

[0032] After receiving the above-mentioned proposal data, the global server will not execute the operations involving computing logic contained in the proposal data, but will initiate data processing based on the consistency protocol for the proposal data in multiple global servers. Specifically, for example, multiple global servers can vote on the proposal data, and reach a consensus when most global servers (for example, at least two global servers) vote in favor. Then, the global server can record the operations contained in each proposal data in the global log in order according to the consensus time when the consensus is reached on the proposal data.

[0033] Each front-end server can obtain incremental logs from the global server at preset time intervals. The incremental logs can be specifically understood as the new log data added to the current global log relative to the global log when the incremental log was last pulled. The front-end server can also save the obtained incremental logs in the local mirror data and update the local mirror data to make the local mirror data of each front-end server consistent.

[0034] Furthermore, the front-end server may perform operations according to the incremental log, in combination with the local mirror data, and in the execution order of the operations in the incremental log to obtain the corresponding target processing result.

[0035] In this way, when the front-end server performs an operation, based on the local mirror data, the front-end server already has the full amount of data required to process the operation, so there is no need to pull new data from the remote global server, so the front-end server can directly perform the operation locally, making the data processing operation relatively efficient. At the same time, because the different front-end servers in the distributed data processing system perform operations according to the same execution order based on the incremental log, it is possible to avoid confusion and inconsistency when different front-end servers in the distributed system perform operations, ensuring the orderly data processing on each front-end server and the consistency of the local mirror data of each front-end server. Thus, the data processing operation in the distributed system with log mirror separation can be made efficient and orderly. In addition, the above method can make the global server not participate in the specific operations involving the calculation logic, so that the global server can be more focused on the processing of the consistency protocol, thereby reducing the data processing burden of the global server, making the division of labor of different servers in the entire distributed system more clear and clear, and improving the overall data processing performance of the system.

[0036] In this embodiment, the global server and the front-end server may specifically include a server that is applied to one side of the network platform and is responsible for data processing in the background and can realize functions such as data transmission and data processing. Specifically, the global server and the front-end server may be, for example, an electronic device with data calculation, storage and network interaction functions. Alternatively, the global server and the front-end server may also be a software program running in the electronic device to provide support for data processing, storage and network interaction. In this embodiment, the number of servers included in the global server and the front-end server is not specifically limited. The global server and the front-end server may specifically be one server, or several servers, or a server cluster formed by several servers.

[0037] In this embodiment, the terminal device may specifically include an electronic device applied to the user side and capable of realizing functions such as data collection and data transmission. Specifically, the terminal device may be, for example, a desktop computer, a tablet computer, a laptop computer, a smart phone, a smart wearable device, etc. Alternatively, the terminal device may also be a software application that can be run in the above electronic devices. For example, it may be an APP running on a mobile phone.

[0038] For an example scenario, see Figure 2 As shown, a shopping website can apply the data processing method provided in the embodiments of this specification to process transaction data in different regions using a distributed system with log mirror separation.

[0039] In this scenario example, when a user in Q City uses a terminal device (e.g., a mobile phone, etc.) to place an order for online shopping, the terminal device can generate and send a corresponding bill payment request to the front-end server (referred to as FrontEndQ) deployed in Q City. The above-mentioned bill payment request is used to request the use of the amount in the user's electronic account to pay the payment amount of the bill (e.g., 100 yuan). Specifically, the above-mentioned payment request includes a comparison and then set operation (i.e., CAS operation), and the execution of this operation involves the use of calculation logic, which will be described in detail later.

[0040] After receiving the bill payment request, the front-end server may first determine the physical address of the terminal device that sent the bill payment request as a device identifier for indicating the terminal device. Figure 3 As shown, a UUID field is added as a preset field in the internal request field of the bill payment request, and the above device identification is filled in the above UUID field.

[0041] See also Figure 4 As shown, the front-end server encapsulates the payment request carrying the device identifier into proposal data (which can be recorded as proposal data a) based on a consistency protocol (e.g., Paxos, etc.), and sends the proposal data a to one of the multiple global servers at the remote end. The total number of the multiple global servers is an odd number.

[0042] After receiving the proposal data a, the global server may add the proposal data a to a queue to be voted on. The queue to be voted on may also include proposal data sent by different front-end servers in other regions (such as P city, R city, etc.). The queue to be voted on may also include proposal data sent by the same front-end server in the same region that has not yet been voted on.

[0043] Furthermore, the global server can vote on the proposed data in the queue in turn. When the global server processes the proposed data a, the global server can initiate a vote on the proposed data a in multiple global servers according to the consistency protocol. In the case where the majority of global servers, for example, more than half of the global servers vote in favor of the proposed data a, the master node server among the multiple global servers can record the operations contained in the proposed data a (hereinafter referred to as operation A) in a certain order in the global log. The above-mentioned global log is used to store all the data in the system. In addition, when the proposal data a is voted in favor, the global server can also feedback corresponding confirmation information to the front-end server, such as "OK", to inform the front-end server in advance that the proposal data has been voted in favor and will be recorded in the global log.

[0044] Specifically, the master node server can arrange the operations contained in different proposal data along the time axis according to the order of the time when the proposal data was voted through, and record the above operations in the global log in order. Figure 5 As shown, the voting time of the proposed data b to which operation B (a creation operation, Creat) belongs is earlier than the voting time of the proposed data a, while the voting time of the proposed data c to which operation C (a compare and set operation, CAS) belongs is later than the voting time of the proposed data a. At this time, when recording the above operations, the master node server can record the above operations in order along the time axis in the global log according to the order of the voting time. In this way, among the operations stored in the global log, operation B will be arranged in front of operation A (a compare and set operation, CAS), and operation A will be arranged in front of operation C.

[0045] In the above process, the global server will not perform operations involving computational logic, nor will it compare or merge the written data with the original data. The global server is mainly responsible for the data based on the consistency protocol, and is only responsible for storing the received written data and the operation records contained in the proposal data in the global log.

[0046] See also Figure 4 The front-end server can periodically send incremental log acquisition requests to the global server at preset time intervals (e.g., 10 minutes). The incremental logs can be specifically understood as the new log data added to the current global log relative to the global log when the incremental log was last pulled.

[0047] The global server receives and responds to the incremental log acquisition request, and feeds back the corresponding incremental log to the front-end server.

[0048] After receiving the above-mentioned incremental log, the front-end server can first send the above-mentioned incremental log to the local mirror server in the area, and the local mirror server will perform mirroring processing according to the above-mentioned incremental log to update the local mirror data, so that the updated local mirror data is consistent with the global log, and contains the full data of the global server before the time point of sending the incremental log.

[0049] For example, the local mirror server may merge duplicate data in the local mirror data according to the data contained in the incremental log, retain only the latest value of the data in the local mirror data, and thus update the local mirror data.

[0050] For further information, see Figure 4 As shown, the mirror server can execute the operations included in the incremental log in order according to the local mirror data. For example, the mirror server first executes operation B, then operation A, and finally operation C according to the arrangement order in the incremental log.

[0051] When the mirror server executes operation A, it can obtain the data carried by operation A, that is, the amount to be paid: 10 yuan. At the same time, the mirror server can also query the remaining amount of the user's payment account from the local mirror data as the target data, and then compare the above target data with the amount to be paid (an operation involving calculation logic) to obtain the corresponding comparison result. According to the comparison result, the corresponding setting operation is performed to obtain the final target processing result.

[0052] It should be noted that when the mirror server executes operation A, the local mirror data used is the mirror data that has been updated according to the incremental log, and contains the full data of the global server before the time point when the incremental log is sent. That is, at this time, the local mirror data already contains the latest target data required to execute operation A (for example, the most detailed remaining amount in the user's payment account, etc.). Therefore, when the mirror server executes operation A, it does not need to call the latest data from the remote global server separately, and can accurately execute operation A only by using the local mirror data.

[0053] In addition, it should be noted that since the incremental logs obtained by different front-end servers in different regions are the same, the data contained in the corresponding incremental logs and the execution order of the operations contained therein are also the same. Therefore, when executing the incremental logs, the local mirror servers in different regions also execute them in the same order, and the processing results obtained are also the same. In this way, after executing the operations contained in the incremental logs, the local mirror data in different regions are also the same and consistent.

[0054] When the specific operation is performed, if the remaining balance in the user's payment account is 15 yuan, the corresponding comparison result is: the amount to be paid is less than the remaining balance in the user's payment account. According to the above comparison result, it can be determined that the remaining balance in the user's payment account is sufficient to pay the bill, and then the user's payment account can be used to complete the payment. At the same time, the remaining balance in the user's payment account is set to 5 yuan. Thus, the comparison and setting operation included in the user's bill request is completed. At this time, the mirror server can generate a prompt message of successful payment as the target processing result, and then send the target processing result to the front-end server. The front-end server then feeds back the above target processing result to the terminal device to prompt the user that the payment is successful.

[0055] If, again, the remaining balance in the user's payment account is 5 yuan, the corresponding comparison result is: the amount to be paid is greater than the remaining balance in the user's payment account. According to the above comparison result, it can be determined that the remaining balance in the user's payment account is insufficient to pay the bill, and the user's payment account cannot be used to complete the payment. At the same time, the remaining balance in the user's payment account remains unchanged at 5 yuan. Thus, the comparison and then setting operation contained in the user's bill request is completed. At this time, the mirror server can generate a prompt message of insufficient account balance and payment failure as the target processing result, and then send the target processing result to the front-end server. The front-end server then feeds back the above target processing result to the terminal device to prompt the user that the balance in the account is insufficient and the payment has failed.

[0056] Of course, in other cases, the region where the front-end server is located may not have the mirror server. In this case, the front-end server itself can be responsible for the local mirroring service.

[0057] For details, please refer to Figure 6 As shown, when the front-end server receives the incremental log, the front-end server can perform mirroring processing based on the incremental log to update the local mirror data. Then, based on the local mirror data, the operations contained in the incremental log are executed in sequence to obtain the corresponding target processing result. Finally, the target processing result is fed back to the terminal device.

[0058] As can be seen from the above scenario examples, based on the data processing method provided by this specification, when executing the operation contained in the incremental log, based on the local mirror data, the local already has the full amount of data required to process the operation, therefore, there is no need to pull new data from the remote global server, so that the operation can be directly performed locally (for example, the local front-end server or the local mirror server), and the corresponding target processing result obtained makes the data processing operation relatively efficient. And because different front-end servers or mirror servers located in different regions are based on the same incremental log and perform operations in the same execution order. Therefore, it can avoid confusion and inconsistency in different regions when performing operations, ensuring the order of data processing operations in different regions and the consistency of local mirror data in different regions. In addition, based on the above method, the global server does not need to consume processing resources and processing time to perform specific operations involving computing logic, and the global server can be more focused on data processing on the consistency protocol, thereby reducing the data processing burden of the global server, making the division of labor of the servers in the entire distributed system more clear and clear, and improving the overall data processing performance of the system.

[0059] See also Figure 7 As shown, the embodiment of this specification provides a data processing method. Among them, the method is specifically applied to the front-end server side of the service. When implemented specifically, the method may include the following contents.

[0060] S701: Receive a data processing request sent by a terminal device; wherein the data processing request includes an operation.

[0061] In one embodiment, the front-end server may be specifically understood as a server deployed in a specific area in a distributed system with log mirror separation to serve users in the area.

[0062] The above-mentioned log mirror separation can be specifically understood as the separation of mirror service and log service in a distributed system. Specifically, a remote global server in the system can be responsible for the global log service, and a regional local server in the system (for example, a regional local front-end server, or a local mirror server) can be responsible for the mirror service. Front-end servers in different regions can make local mirror data in different regions consistent by pulling incremental logs from the global server.

[0063] In one embodiment, the incremental log may be specifically understood as the log data newly added to the current global log relative to the global log when the incremental log was last pulled.

[0064] In one embodiment, the above-mentioned data processing request can be specifically understood as request data generated and sent by the user through the terminal device, which contains the operation to be performed. Specifically, the above-mentioned data processing request can be a transaction data processing request, a bill payment request, an information query request, and so on. Of course, the data processing requests listed above are only a schematic illustration. During specific implementation, according to the specific application scenario, the above-mentioned data processing request may also include other types of data processing requests in addition to the requests listed above. This specification does not limit this.

[0065] In one embodiment, the above operation can be specifically understood as an operation involving calculation logic contained in a data processing request. Specifically, the above operation may include a compare and set operation (Compare And Set, CAS). The specific implementation of this operation may include comparing the data (such as amount, version, etc.) input (or written) by the user through the terminal device with the existing relevant data, and then performing corresponding setting processing operations according to the comparison results. In addition, the above operation may also include a compare and switch operation (Compare And Swap), a compare and delete operation (Compare And Delete), and other operations involving calculation logic during the implementation process. Of course, it should be noted that the operations listed above are only a schematic description. During specific implementation, other operations involving calculation logic may also be used as the above operations according to specific application scenarios and processing requirements. This specification does not limit this.

[0066] In one embodiment, the data processing request may include operations not involving computing logic, such as creation operations, update operations, etc., in addition to the operations involving computing logic.

[0067] In addition, the above data processing request may also include other content data such as data to be written by the user.

[0068] In one embodiment, during specific implementation, the terminal device can generate a corresponding data processing request in response to the user's operation, and send the data processing request to the front-end server in the area via wired or wireless means. Correspondingly, the front-end server can receive the data processing request sent by the terminal device.

[0069] S702: Send the data processing request to the global server; wherein the global server is used to receive the data processing request sent by the front-end server, and record the operations included in the data processing request in sequence in the global log.

[0070] In one embodiment, in a distributed system with separated log mirrors, the global server may specifically perform a consistency vote on the data carried in the user's data processing request based on a consistency protocol (e.g., Paxos protocol, Raft protocol, zab protocol, etc.), and then persist the data in the global log.

[0071] In view of the above situation, the front-end server can first encapsulate the data processing request into proposal data based on the consistency protocol, and then send the proposal data to the global server.

[0072] In one embodiment, the global server may include multiple (e.g., an odd number) global servers. The multiple global servers may also be deployed in different regions. The front-end server may send the proposal data to any one of the multiple global servers.

[0073] In one embodiment, after receiving the above-mentioned proposal data, the global server may initiate a vote on the proposed number among multiple global servers according to a consistency protocol; when it is determined that more than a preset number (for example, half) of global servers have voted in favor of the proposal data, the operations contained in the proposal data are recorded in sequence in the global log.

[0074] In one embodiment, the global server may receive multiple different proposal data at the same time. The multiple proposal data may be sent by the same front-end server or different front-end servers. The global server may queue the multiple proposal data for voting. Then, the time when each proposal data is voted through may be determined, and then, based on the time when the vote is passed, the operations contained in the proposal data may be recorded in sequence in the global log.

[0075] Specifically, for example, the global server may arrange operations 1, 2, 3, and 4 in different proposal data in chronological order along the time axis according to the voting time of the corresponding proposal data, and record them in the global log.

[0076] In one embodiment, in a system of other scenarios, the front-end server may also directly send the data processing request to the global server. The global server may arrange the operations included in different data processing requests in chronological order along the time axis according to the receiving time of the data processing request, and record them in the global log.

[0077] S703: Obtain incremental logs through the global server.

[0078] In one embodiment, during specific implementation, the front-end server may send a request to obtain the incremental log to the global server at a preset time interval (e.g., 10 minutes). The global server receives and responds to the request to obtain the incremental log, and feeds back the corresponding incremental log to the front-end server. Thus, the front-end server can obtain the corresponding incremental log through the global server.

[0079] In one embodiment, during specific implementation, the global server may detect whether a trigger condition occurs, and actively push the incremental log to the front-end server when the trigger condition is detected.

[0080] The trigger condition may specifically be that the global log has been updated; or that the global server has accumulated multiple records of the global log since the last incremental log was pushed, where the number of records is greater than the trigger number threshold, etc.

[0081] Of course, the above-listed methods for obtaining incremental logs are only schematic illustrations. In specific implementation, the front-end server may also obtain incremental logs through a combination of the above-listed methods, or through other methods other than the above-listed methods.

[0082] In one embodiment, after the front-end server obtains the incremental log through the global server, it can also store the incremental log as mirror data through the mirror service and update the local mirror data so that the local mirror data is consistent with the local mirror data of other front-end servers and the global log of the global server.

[0083] In one embodiment, the mirror service can be performed locally on the front-end server, that is, the front-end server can be responsible for the mirror service and manage the local mirror data. In this case, the front-end server can directly convert the incremental log into mirror data locally for storage to update the local mirror data.

[0084] In one embodiment, the mirror service can be performed on a third-party server (referred to as a mirror server) in the area where the front-end server is located, that is, a mirror server different from the front-end server can be responsible for the mirror service in the area and manage the local mirror data. In this case, the front-end server can send the incremental log to the mirror server, and the mirror server converts the incremental log into mirror data for storage to update the local mirror data.

[0085] S704: According to the incremental log and the local mirror data, the operations included in the incremental log are executed in sequence.

[0086] In one embodiment, the front-end server can specifically determine the execution order of operations based on the incremental log; and then obtain data matching the operation from the local mirror data as target data; perform operations according to the target data in accordance with the execution order to obtain corresponding target processing results.

[0087] In one embodiment, the data matching the operation can be understood as data required to perform the operation. For example, when the operation is to set the operation after comparison, the target data can be existing data used for comparison.

[0088] In one embodiment, after receiving the incremental log, the operation is performed in the order of execution in the incremental log; and when performing the operation, the target data obtained is based on the local mirror data updated after obtaining the incremental log. Therefore, when the operation is executed, the local mirror data already contains all the required full data before the global server records the operation. In this case, the front-end server does not need to access the global server to pull the latest data; there is no need to worry about the data of the global server changing due to the operations of other front-end servers in the process of pulling new data, and there is no need to set a global lock, etc. Instead, the local mirror data can be directly used to obtain the target data required for the processing operation from the local mirror data, and then the operation is performed according to the target data to obtain the corresponding target processing result.

[0089] In one embodiment, since multiple different front-end servers in the system all obtain and perform operations according to the same incremental log and in the same execution order, it is possible to avoid confusion and inconsistency when different front-end servers in the distributed system perform operations, ensuring the order of data processing on each front-end server and the consistency of local mirror data of each front-end server.

[0090] In one embodiment, when the mirror service is provided by the mirror server, the mirror server can send the incremental log to the mirror server, and the mirror server replaces the front-end server to sequentially execute the operations contained in the incremental log according to the incremental log and the local mirror data.

[0091] In one embodiment, taking the operation including comparison followed by setting as an example, in the operation performed according to the target data to obtain the corresponding target processing result, the specific implementation may include: comparing the data carried in the operation (i.e., the data input by the user or the data written) with the target data to obtain the comparison result; and performing the corresponding setting operation according to the comparison result to obtain the corresponding target processing result. The operation is completed.

[0092] In one embodiment, when there is a mirror server (for example, a third-party server) in the area where the front-end server is located that is different from the front-end server and is responsible for the local mirror service, after the front-end server obtains the incremental log through the global server, the method may also include the following contents when it is implemented: the front-end server sends the incremental log to the mirror server; the mirror server updates the local mirror data according to the incremental log, and executes the operations contained in the incremental log according to the local mirror data.

[0093] In one embodiment, after the mirror server performs an operation and obtains a corresponding target processing result, the mirror server may send the target processing result to the front-end server. The front-end server may feed back the target processing result to the corresponding terminal device.

[0094] It can be seen from the above embodiments that based on the data processing method provided by the embodiments of this specification, in a distributed system with separated log images, when the front-end server receives a data processing request containing an operation from a terminal device, it will not immediately respond to the data processing request and perform the operation, but will first send the data processing request to the global server; after receiving the data processing request, the global server will not perform the operation, but will record the operations in the data processing requests from different front-end servers in the global log in sequence. The front-end server obtains the incremental log through the global server, and then executes the operations contained in the incremental log in sequence according to the incremental log. This can make the data processing operations in the distributed system with separated log images efficient and orderly.

[0095] In one embodiment, before encapsulating the data processing request into proposal data based on the consistency protocol, the method may also include the following contents when implemented: determining the device identification of the terminal device; adding a preset field in the data processing request, and setting the device identification of the terminal device in the preset field.

[0096] In this embodiment, the device identification can be specifically understood as identification information of a terminal device that sends a data processing request. Specifically, the identification information can be a device number of the terminal device, or a physical address of the terminal device, etc. The specific content of the device identification is not limited in this specification.

[0097] In this embodiment, during specific implementation, the front-end server may determine the above-mentioned device identification by parsing the data processing request.

[0098] In this embodiment, after receiving a data processing request, the front-end server can add a preset field inside the data processing request, and then set the corresponding device identifier in the above preset field, so that when subsequently executing operations in the incremental log, the terminal device corresponding to the operation can be determined according to the device identifier in the above preset field.

[0099] In one embodiment, the above-mentioned preset field may specifically include a UUID field. Figure 3 As shown. The above UUID field can be a part of the internal request field (or protocol field) of the data processing request. Of course, it should be noted that the UUID fields listed above are only a schematic illustration. In specific implementation, other suitable fields can also be selected as the above preset fields. This specification does not limit this.

[0100] In one embodiment, after executing the operations in sequence and obtaining the corresponding target processing results, the method may also include the following contents when implemented: obtaining and determining the device identification of the terminal device based on the preset field corresponding to the operation in the incremental log; and sending the target processing result to the terminal device based on the device identification of the terminal device.

[0101] In this embodiment, during specific implementation, the front-end server can also read the preset field in the data processing request corresponding to the operation through the incremental log, and determine the terminal device corresponding to the operation according to the device identifier in the preset field. Then, the target processing result obtained by executing the operation can be sent to the terminal device through a wired or wireless method, so as to timely feedback the corresponding target processing result to the user through the terminal device, thereby improving the user's experience.

[0102] In one embodiment, the operations contained in the incremental log are executed in sequence according to the incremental log and the local mirror data. When implemented specifically, the following contents may be included: determining the execution order of the operations according to the incremental log; obtaining target data matching the operations from the local mirror data; and executing the operations according to the target data in the execution order to obtain corresponding target processing results.

[0103] In this embodiment, since the incremental log is obtained based on the global log, the operations recorded in the incremental log are all arranged in chronological order. The front-end server can determine the execution order of each operation based on the arrangement order of each operation in the incremental log. Then, the operation can be executed based on the execution order.

[0104] In one embodiment, the operation may specifically include an operation involving computing logic. Among them, the operation involving computing logic may specifically include at least one of the following: compare and then set operation (CAS operation), compare and then delete operation, compare and convert operation, etc. Of course, the operations involving computing logic listed above are only a schematic illustration. During specific implementation, according to specific application scenarios and processing requirements, the above operations may also include other types of operations involving computing logic. This specification does not limit this. In addition, the operation may specifically include operations that do not involve computing logic. For example, create operations, update operations, etc.

[0105] In one embodiment, during specific implementation, the specific type of the operation may be determined according to identification information of the operation included in the data processing request.

[0106] For example, when it is detected that the identification information of the operation matches an example operation in a preset operation list, the operation can be determined to be an operation involving computing logic. The preset operation list may store identification information of a plurality of pre-determined common example operations involving computing logic (e.g., CAS operations, etc.).

[0107] For another example, when it is detected that the identification information of the operation does not match any of the example operations in the preset operation list, it can be determined that the operation is an operation that does not involve computing logic.

[0108] In one embodiment, in the case where the operation includes a comparison followed by a setting operation, the operation is performed according to the target data to obtain a corresponding target processing result. When implemented specifically, the following contents may be included: comparing the data carried in the operation with the target data to obtain a comparison result; and performing a corresponding setting operation according to the comparison result to obtain a corresponding target processing result.

[0109] In one embodiment, the above method can be applied to a transaction data processing scenario. Accordingly, the above data processing request can specifically include a bill payment request, the data carried in the above operation can specifically include the amount to be paid, and the above target data can specifically include the remaining amount in the payment account.

[0110] Of course, the transaction data processing scenarios listed above are only schematic illustrations. In specific implementation, the above method can also be applied to other application scenarios. Accordingly, the above data processing request, target data, etc. can also contain other types of data that match the application scenario. For example, the above method can also be applied to identity authentication scenarios. Accordingly, the above data processing request can include an identity authentication request, and the data carried in the above operation can specifically include biometric features to be verified (for example, fingerprints or head portraits of users to be verified, etc.), and the above target data can specifically include biometric features pre-stored in a database and bound to the user, etc.

[0111] As can be seen from the above, the data processing method provided by the embodiment of this specification, based on the data processing method provided by the embodiment of this specification, the front-end server in the distributed system with log mirror separation will not immediately respond to the data processing request and perform the operation when receiving the data processing request containing the operation from the terminal device, but will first send the data processing request to the global server; after receiving the data processing request, the global server will not perform the operation, but will record the operations in the data processing requests from different front-end servers in the global log in sequence. The front-end server obtains the incremental log through the global server, and then executes the operations contained in the incremental log in sequence according to the incremental log. Thus, the data processing operation in the distributed system with log mirror separation can be made efficient and orderly. At the same time, since different front-end servers deployed in different regions all obtain and perform operations according to the same incremental log and in the same execution order, it can avoid confusion and inconsistency when different front-end servers in different regions perform operations, ensuring the orderly data processing on the front-end servers in each region and the consistency of the local mirror data in each region. Moreover, based on the above method, the global server does not need to participate in specific operations involving computing logic, so that the global server can focus more on data processing related to the consistency protocol, thereby reducing the data processing burden of the global server, making the division of labor of different servers in the entire data processing system clear and clear, and improving the overall data processing performance of the data processing system. In addition, the front-end server also adds a preset field inside the data processing request before encapsulating the data processing request into the proposal data, and sets the device identification of the terminal device that sends the data processing request in the above preset field. In this way, when the front-end server performs the operation in the incremental log, it can determine the corresponding terminal device by reading the device identification in the preset field, and then can promptly feedback the target processing result to the corresponding terminal device, so that the user on the terminal device side can obtain the target processing result in a timely manner, thereby improving the user experience.

[0112] See also Figure 8As shown, the embodiment of this specification also provides another data processing method. Among them, the method is applied to the global server side. When implemented specifically, the method may include the following contents.

[0113] S801: Acquire proposal data.

[0114] S802: Initiate a vote on the proposed data in multiple global servers according to a consistency protocol.

[0115] S803: When it is determined that more than a preset number of global servers have voted in favor of the proposed data, operations included in the proposed data are recorded in sequence in a global log.

[0116] In one embodiment, the global server may be a server in a distributed system with log mirror separation. Specifically, the global server may include multiple global servers, wherein different global servers may be deployed in different regions or in the same region.

[0117] In one embodiment, the global server is responsible for the global log service in the system, focusing on data processing based on the consistency protocol, and will not participate in specific operations involving computing logic.

[0118] In one embodiment, when the above-mentioned global server is implemented, a vote can be initiated among multiple global servers for the proposed data according to a consistency protocol. When more than a preset number of global servers agree, the proposed data is determined to be voted through, and the data and / or operations carried by the proposed data are recorded in the global log.

[0119] In one embodiment, the operations included in the proposed data are recorded in sequence in a global log. When implemented specifically, the following contents may be included: determining the time when the proposed data is voted through; and based on the time when the vote is passed, recording the operations included in the proposed data in sequence in a global log.

[0120] In one embodiment, when recording a global log, the global server can arrange different operations along the timeline according to the order of the voting and approval time of the proposal data corresponding to the operations, and record the above operations in sequence in the global log, so that the subsequent front-end server can execute each operation on the front-end server side according to the incremental log and in the same execution order as in the log.

[0121] In one embodiment, the operation may specifically include an operation involving calculation logic. The calculation logic may specifically include calculation logic related to comparison. Specifically, the operation may include a comparison and then set operation (ie, CAS operation).

[0122] In one embodiment, after the operations contained in the proposal data are recorded in sequence in the global log, the method may also include the following contents when implemented: detecting whether a trigger condition occurs; and if it is determined that the trigger condition is detected, pushing an incremental log to the front-end server according to the current global log.

[0123] In one embodiment, the trigger condition may specifically be that the global log has been updated; or that the global server has accumulated multiple records of the global log since the last incremental log was pushed, where the number of records is greater than the trigger number threshold, etc.

[0124] In one embodiment, when the global server detects a trigger condition, it can actively send the incremental log to each front-end server. Of course, when the global server receives the incremental log acquisition request sent regularly from the front-end server, responds to the incremental log acquisition request, and sends the incremental log to the front-end device.

[0125] Based on the above data processing method, the global server does not need to participate in specific operations involving computing logic and can focus more on data processing related to the consistency protocol, thereby reducing the data processing burden of the global server and making the division of labor of different servers in the entire data processing system clear and clear, thereby improving the overall data processing performance of the data processing system.

[0126] The embodiment of this specification also provides another data processing method. The method can be specifically applied to the global server side, and when implemented, it can include the following contents.

[0127] S1: Get data processing request.

[0128] S2: Record the operations included in the data processing request in the global log in sequence.

[0129] In one embodiment, the front-end server in the distributed system may not encapsulate the data processing request into proposal data and then send it to the global server, but directly send the data processing request containing the operation to the global server. Accordingly, the global server may receive the data processing request sent from the front-end server. However, the global server does not need to execute the operation contained in the data processing request, but records the operations contained in the data processing request in sequence in the global log.

[0130] In one embodiment, the operations included in the data processing request are recorded in sequence in a global log. When implemented specifically, the following contents may be included: determining the time when the data processing request is received; and recording the operations included in the data processing request in sequence in a global log according to the time when the processing request is received.

[0131] In one embodiment, when recording a global log, the global server can arrange different operations along the timeline according to the order in which the data processing requests corresponding to the operations are received by the global server, and record the above operations in sequence in the global log, so that the subsequent front-end server can execute each operation on the front-end server side according to the incremental log and in the same execution order as in the log.

[0132] The embodiment of this specification also provides a server, including a processor and a memory for storing processor executable instructions. The processor can perform the following steps according to the instructions when it is implemented: receiving a data processing request sent by a terminal device; wherein the data processing request includes an operation; sending the data processing request to a global server; wherein the global server is used to receive the data processing request sent by a front-end server, and record the operations included in the data processing request in a global log in sequence; obtaining an incremental log through the global server; and executing the operations included in the incremental log in sequence according to the incremental log and the local mirror data. The server can be specifically used as a front-end server in a distributed system.

[0133] See also Fig. 9 As shown, the embodiment of this specification also provides another specific server, wherein the server includes a network communication port 901, a processor 902 and a memory 903, and the above structures are connected through internal cables so that each structure can perform specific data interaction.

[0134] The network communication port 901 may be specifically used to receive a data processing request sent by a terminal device; wherein the data processing request includes an operation.

[0135] The processor 902 can be specifically used to send the data processing request to the global server; wherein the global server is used to receive the data processing request sent by the front-end server, and record the operations contained in the data processing request in sequence in the global log; obtain the incremental log through the global server; and execute the operations contained in the incremental log in sequence according to the incremental log and the local mirror data.

[0136] The memory 903 may be specifically used to store corresponding instruction programs.

[0137] In this embodiment, the network communication port 901 can be a virtual port that is bound to different communication protocols so that different data can be sent or received. For example, the network communication port can be a port responsible for web data communication, a port responsible for FTP data communication, or a port responsible for email data communication. In addition, the network communication port can also be a physical communication interface or communication chip. For example, it can be a wireless mobile network communication chip, such as GSM, CDMA, etc.; it can also be a Wifi chip; it can also be a Bluetooth chip.

[0138] In this embodiment, the processor 902 may be implemented in any appropriate manner. For example, the processor may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) executable by the (micro)processor, a logic gate, a switch, an application specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, etc. This specification does not limit this.

[0139] In this embodiment, the memory 903 may include multiple levels. In a digital system, anything that can store binary data can be a memory; in an integrated circuit, a circuit with a storage function that has no physical form is also called a memory, such as RAM, FIFO, etc.; in a system, a storage device with a physical form is also called a memory, such as a memory stick, TF card, etc.

[0140] The embodiment of this specification also provides another server, including a processor and a memory for storing processor executable instructions. The processor can perform the following steps according to the instructions when it is implemented: obtaining proposal data; initiating a vote on the proposal data in multiple global servers according to a consistency protocol; and recording the operations contained in the proposal data in sequence in a global log when it is determined that more than a preset number of global servers have voted in favor of the proposal data. The server can be used as a front-end server in a distributed system.

[0141] The embodiment of the present specification also provides a data processing system, including multiple front-end servers and multiple global servers, wherein: the front-end server receives a data processing request sent by a terminal device; wherein the data processing request includes an operation; the front-end server encapsulates the data processing request into proposal data based on a consistency protocol, and sends the proposal data to a global server; the global server obtains the proposal data; according to the consistency protocol, a vote on the proposal data is initiated among multiple global servers; when it is determined that more than a preset number of global servers have voted in favor of the proposal data, the operations included in the proposal data are recorded in sequence in a global log; the front-end server also obtains an incremental log through the global server; and according to the incremental log and the local mirror data, the operations included in the incremental log are executed in sequence.

[0142] In one embodiment, the data processing system may be a distributed system. Further, the data processing system may be a distributed system with log mirroring separation. The global server may include multiple global servers. The number of global servers may be an odd number.

[0143] The embodiment of this specification also provides a computer storage medium based on the above-mentioned data processing method, wherein the computer storage medium stores computer program instructions, and when the computer program instructions are executed, the following are achieved: receiving a data processing request sent by a terminal device; wherein the data processing request includes an operation; sending the data processing request to a global server; wherein the global server is used to receive the data processing request sent by a front-end server, and record the operations included in the data processing request in sequence in a global log; obtaining an incremental log through the global server; and executing the operations included in the incremental log in sequence according to the incremental log and local mirror data.

[0144] In this embodiment, the storage medium includes, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a cache, a hard disk (HDD), or a memory card. The memory may be used to store computer program instructions. The network communication unit may be an interface for network connection communication set in accordance with the standard specified by the communication protocol.

[0145] In this embodiment, the functions and effects specifically implemented by the program instructions stored in the computer storage medium can be explained in comparison with other implementations and will not be described in detail here.

[0146] See also Fig.10 As shown, at the software level, the embodiments of this specification also provide a data processing device, which may specifically include the following structural modules.

[0147] The receiving module 1001 may be specifically configured to receive a data processing request sent by a terminal device; wherein the data processing request includes an operation;

[0148] The sending module 1002 may be specifically used to send the data processing request to the global server; wherein the global server is used to receive the data processing request sent by the front-end server, and record the operations included in the data processing request in sequence in the global log;

[0149] The first acquisition module 1003 may be specifically used to acquire incremental logs through a global server;

[0150] The execution module 1004 may be specifically configured to sequentially execute the operations included in the incremental log according to the incremental log and the local mirror data.

[0151] The embodiments of this specification also provide another data processing device, which may specifically include the following structural modules: a second acquisition module, which may be specifically used to acquire proposal data; a processing module, which may be specifically used to initiate a vote on the proposal data in multiple global servers according to a consistency protocol; and a recording module, which may be specifically used to record the operations contained in the proposal data in sequence in a global log when it is determined that more than a preset number of global servers have voted in favor of the proposal data.

[0152] It should be noted that the units, devices or modules described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above devices are described separately by functions divided into various modules. Of course, when implementing this specification, the functions of each module can be implemented in the same or more software and / or hardware, or the modules that implement the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0153] It can be seen from the above that the data processing device provided in the embodiments of this specification can make the data processing operations in the distributed system with log mirror separation efficient and orderly.

[0154] Although the present specification provides method operation steps as described in the embodiments or flow charts, more or less operation steps may be included based on conventional or non-creative means. The order of steps listed in the embodiments is only one way of executing the order of many steps, and does not represent a unique execution order. When the device or client product in practice is executed, it can be executed in sequence or in parallel according to the method shown in the embodiments or the drawings (for example, a parallel processor or a multi-threaded processing environment, or even a distributed data processing environment). The term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, product or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, product or device. In the absence of more restrictions, it is not excluded that there are other identical or equivalent elements in the process, method, product or device including the elements. The first, second, etc. words are used to represent the name, and do not represent any particular order.

[0155] Those skilled in the art also know that, in addition to implementing the controller in a purely computer-readable program code, the controller can be made to implement the same function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered as a hardware component, and the devices for implementing various functions included therein can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules for implementing the method and structures within the hardware component.

[0156] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media including storage devices.

[0157] Through the description of the above embodiments, it can be known that those skilled in the art can clearly understand that the present specification can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present specification can essentially be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a mobile terminal, a server, or a network device, etc.) to execute the methods described in each embodiment of the present specification or some parts of the embodiments.

[0158] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. This specification can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0159] Although the present specification is described through embodiments, those skilled in the art will appreciate that there are many modifications and changes to the present specification without departing from the spirit of the present specification, and it is intended that the appended claims include these modifications and changes without departing from the spirit of the present specification.

Claims

1. A data processing method, comprising: Receiving a data processing request sent by a terminal device; wherein the data processing request includes an operation; Sending the data processing request to a global server; wherein the global server is used to receive the data processing request sent by the front-end server, and record the operations included in the data processing request in a global log in sequence; Get incremental logs through the global server; According to the incremental log and the local mirror data, the operations contained in the incremental log are executed in sequence; The step of sending the data processing request to the global server includes: Encapsulating the data processing request into proposal data based on a consistency protocol; The proposed data is sent to a global server, and according to a consistency protocol, a vote on the proposed data is initiated among multiple global servers; when it is determined that more than a preset number of global servers have voted in favor of the proposed data, the operations contained in the proposed data are recorded in sequence in a global log.

2. The method according to claim 1, before encapsulating the data processing request into proposal data based on the consistency protocol, the method further comprises: Determining a device identification of the terminal device; A preset field is added to the data processing request, and the device identification of the terminal device is set in the preset field. 3 . The method according to claim 2 , wherein the preset field comprises a UUID field.

4. The method according to claim 2, wherein the operations contained in the incremental log are sequentially executed according to the incremental log and the local mirror data, comprising: Determine the execution order of the operations according to the incremental log; Get the target data that matches the operation from the local mirror data; According to the execution order, operations are performed according to the target data to obtain corresponding target processing results.

5. The method according to claim 4, after performing the operations in sequence and obtaining the corresponding target processing results, the method further comprises: Obtaining and determining a device identification of the terminal device according to a preset field corresponding to the operation in the incremental log; The target processing result is sent to the terminal device according to the device identification of the terminal device.

6. The method of claim 4, wherein the operations include operations involving computational logic, wherein: The operation involving the calculation logic includes at least one of the following: a compare-and-set operation, a compare-and-delete operation, and a compare-and-convert operation.

7. The method according to claim 6, wherein the step of performing an operation according to the target data to obtain a corresponding target processing result comprises: Compare the data carried in the operation with the target data to obtain a comparison result; According to the comparison result, corresponding setting operations are performed to obtain corresponding target processing results.

8. The method according to claim 7, wherein the data processing request comprises a bill payment request, the data carried in the operation comprises an amount to be paid, and the target data comprises a remaining amount in the payment account.

9. The method according to claim 1, obtaining incremental logs through a global server, comprises: Receive incremental logs pushed by the global server when a trigger condition is detected; and / or, At preset time intervals, a request for obtaining incremental logs is sent to the global server to obtain the incremental logs fed back by the global server.

10. A data processing method, comprising: Obtain a data processing request; record the operations included in the data processing request in sequence in a global log; Among them, incremental logs are obtained through the global server; According to the incremental log and the local mirror data, the operations contained in the incremental log are executed in sequence; Encapsulating the data processing request into proposal data based on a consistency protocol; The proposed data is sent to a global server, and according to a consistency protocol, a vote on the proposed data is initiated among multiple global servers; when it is determined that more than a preset number of global servers have voted in favor of the proposed data, the operations contained in the proposed data are recorded in sequence in a global log.

11. The method according to claim 10, wherein recording the operations included in the data processing request in sequence in a global log comprises: determining a time of receipt of the data processing request; The operations included in the data processing request are recorded in sequence in a global log according to the receiving time of the processing request.

12. A data processing method, comprising: Get proposal data; Initiate a vote on the proposed data in multiple global servers according to a consensus protocol; If it is determined that more than a preset number of global servers have voted in favor of the proposed data, the operations included in the proposed data are recorded in sequence in a global log; wherein, receiving a data processing request sent by a terminal device, encapsulating the data processing request into proposal data based on a consistency protocol; and sending the proposal data to a global server; Get incremental logs through the global server; According to the incremental log and the local mirror data, the operations contained in the incremental log are executed in sequence.

13. The method according to claim 12, wherein recording the operations included in the proposal data in sequence in a global log comprises: Determine the time for voting on the proposed data; According to the time when the vote is passed, the operations included in the proposal data are recorded in sequence in the global log.

14. The method of claim 12, the operations comprising operations involving computational logic.

15. The method according to claim 12, after recording the operations included in the proposal data in sequence in a global log, the method further comprises: Detect whether the trigger condition occurs; When it is determined that the trigger condition is detected, the incremental log is pushed to the front-end server according to the current global log.

16. A data processing system, comprising a plurality of front-end servers and a plurality of global servers, wherein: The front-end server receives a data processing request sent by a terminal device; wherein the data processing request includes an operation; the front-end server encapsulates the data processing request into proposal data based on a consistency protocol, and sends the proposal data to a global server; The global server obtains the proposed data; according to the consistency protocol, initiates a vote on the proposed data among multiple global servers; when it is determined that more than a preset number of global servers have voted in favor of the proposed data, records the operations contained in the proposed data in sequence in a global log; The front-end server also obtains the incremental log through the global server; and executes the operations contained in the incremental log in sequence according to the incremental log and the local mirror data.

17. A data processing device, comprising: A receiving module, used for receiving a data processing request sent by a terminal device; wherein the data processing request includes an operation; A sending module, used to send the data processing request to a global server; wherein the global server is used to receive the data processing request sent by the front-end server, and record the operations included in the data processing request in a global log in sequence; A first acquisition module is used to acquire the incremental log through the global server; an execution module is used to sequentially execute the operations contained in the incremental log according to the incremental log and the local mirror data; The device is further used to send the data processing request to a global server, including: Encapsulating the data processing request into proposal data based on a consistency protocol; The proposed data is sent to a global server, and according to a consistency protocol, a vote on the proposed data is initiated among multiple global servers; when it is determined that more than a preset number of global servers have voted in favor of the proposed data, the operations contained in the proposed data are recorded in sequence in a global log.

18. A data processing device, comprising: A second acquisition module is used to acquire proposal data; A processing module, configured to initiate a vote on the proposed data in a plurality of global servers according to a consistency protocol; A recording module, configured to record the operations included in the proposal data in sequence in a global log when it is determined that more than a preset number of global servers have voted in favor of the proposal data; The device is further configured to receive a data processing request sent by a terminal device, encapsulate the data processing request into proposal data based on a consistency protocol, and send the proposal data to a global server; Get incremental logs through the global server; According to the incremental log and the local mirror data, the operations contained in the incremental log are executed in sequence.

19. A server, comprising a processor and a memory for storing processor-executable instructions, wherein the processor implements the steps of the method according to any one of claims 1 to 9 when executing the instructions.

20. A computer-readable storage medium having computer instructions stored thereon, wherein the instructions, when executed, implement the steps of the method according to any one of claims 1 to 9.

Citation Information

Patent Citations

  • Synchronization method and device for master database and slave database

    CN105912628A

  • Large-scale cloud storage copy server consistency processing method and system

    CN106603645A